Develop > Develop Content > Extend Content > IDE > IDE Advanced Tasks > Data Source Extraction Customization

Data Source Extraction Customization

Integration with data sources is performed using the Data Collection Service (DCS). In order to generate the query executed for the data source, the Content Flow Manager tells the DCS framework which source entity to extract. The DCS then extracts the correlating metadata definitions, and uses them in order to formulate a query.

ClosedCustomize the Source Model

You can customize the generated query by modifying the source entity .XML file. This updates the source metadata in the Data Warehouse and matches it to the customized source.

  1. Modify the .XML file and load it into the Data Warehouse metadata repository by logging on DWH and using the user interface.

  2. Change the SQL query from the DCS log and run the query on the source database make sure the query is correct and can run without errors.

  3. Identify all of the changes.

There are various types of customizations to the source models supported by the DWH. This document emphasizes cases where the data source that you extract information from is based on RDBMS, and entities which are already modeled in the DWH have had field names or table names changed in the data source’s structure.

The Data Source’s source model metadata xml files are located under:

$HPBA_HOME/ContentPacks/<CPname>/INBUILT/DATAMODEL/DATAINTEGRATION/SOURCE/<product_version>/<DB_type>/<entity_name>.xml

Example

$HPBA_HOME/ContentPacks/SM/INBUILT/DATAMODEL/DATAINTEGRATION/SOURCE/9.2/oracle/SM_INCIDENTSM1_SRC.xml

ClosedMake changes in a field name

Open the entity xml source model file and update the source field names in the datasourcecolumnname property in the ColumnMapping tag:

Example

<ns1:ColumnMapping extractioncolumnidentifier="<identifier>" notapplicable="N" >

<ns1:DataSourceMapping datasourcetablenameoraliasname=" <name>" datasourcecolumnname=" source field name" datasourcecolumntype="XXXXX"/>

</ns1:ColumnMapping>

ClosedMake changes in a table name

  1. Open the entity xml source model file and update the DataSourceJoin and DataSourceWhere tags with the changes.

Examples

<ns1:DataSourceJoin>table name</ns1:DataSourceJoin>

<ns1:DataSourceWhere>field name=1</ns1:DataSourceWhere>

<ns1:DataSourceJoin>table1 A LEFT JOIN table2 B ON A.AAAAA = B.BBBBB</ns1:DataSourceJoin>

<ns1:DataSourceWhere>A.AAAAAAAA IS NOT NULL</ns1:DataSourceWhere>

  1. Change the datasourcetablenameoraliasname property in the ColumnMapping tag to the new value.

Example

<ns1:ColumnMapping extractioncolumnidentifier="XXXX" notapplicable="N" >

<ns1:DataSourceMapping datasourcetablenameoraliasname=" A" datasourcecolumnname=" XXXX" datasourcecolumntype="XXXXX"/>

</ns1:ColumnMapping>

ClosedUpdate the data warehouse metadata

  1. Go to Admin > Data Management >Install Content Pack, and upgrade the changes.

  2. Start the ETL again using the DW ABC Streams Management UI. For details, see Data Management - Run ETL - Content Flow Management

Send Help Center feedback