Data Model Types

Content packs contain all the artifacts needed to connect to the relevant data source and gather data from that data source. The mapping between the raw data from the data source and the target database in the Data Warehouse is included in the ETL.

ClosedData Model Types

Data models describe the source, extraction, consolidation, and target components for a single entity:

  • Target Entity. Fact or Dimension: Entity name and column names defined according to the Star-Schema architecture.
  • Source Entity. The actual representation of an entity extracted from the source. Generated automatically according to the integration entity design.
  • Integration Entity. Entity name, column names, and data types defined according to source table design.
  • Column Mapping. Integration to Target column mapping.

ClosedLogical Data Models

DWH supports the following types of logical data models:

  • Target Data Models:

    • Target Model – Dimension or Fact. The target data models describe the format of the data that produces business intelligence analytics. The instantiated data models are dimension, fact, and hierarchy tables.

  • iCP Data Models:

    • Extraction Model. ETL processes obtain the data in native format and stages it in intermediate tables that conform to the extraction data model for that entity. Extraction also validates the data to ensure that it is usable.

    • Source Model. The source entity models represent the design of DII (Data Integration Interface).

ClosedPhysical Data Models

DWH logical Data Models generate multiple physical tables that support the ETL processing logic.

  • Source entity models. The source entity models are the data produced by an external application. Used for a specific version of the product.

  • Extraction models. ETL processes obtain the data in native format and stage it in intermediate tables that conform to the extraction data model for that entity. Extraction also validates the data to ensure that it is usable.

  • Consolidation models. Consolidation models transform the data further by merging extracted data from diverse sources into meaningful data required by applications.

  • Target models. The target models describe the final version of the data that adheres to the design required by applications.

Data models describe the source and extraction format so that the data warehouse can consume this data and integrate it into a common view that you can use to populate comprehensive business intelligence reports and dashboards.

Each table contains both designed columns and static columns, also known as metadata columns, so that the administrator can have the visibility of ETL processing over the entity’s data.

ClosedData Warehouse Internal Fields

The following are the internal Data Warehouse fields, in addition to those in the IDE, that are added to the model.

Field Name Prefixes Description
MD_BATCH_ID

ETL stream ID displayed on all Physical Models.

MD_PROCESS_ID ETL Process ID running the specific job.
MD_DURABLE_KEY

Entity’s unique identifier populated with MD_QUALIFIER which should be considered as the identifier to consolidate records.

Displayed only in Consolidation Physical Models

MD_ENTERPRISE_KEY

Entity’s unique identifier in the DWH system.

Displayed only in Target Physical Models

MD_BUSINESS_KEY

Entity’s unique key as it comes from the source systems.

Displayed only in Target and Extraction Physical Models

MD_TRANSLASTIND

Indicator whether the line is the last transaction of a specific MD_ENTERPRISE_KEY.

Displayed only in Target Physical Models

MD_ACTIVESTATUSIND

Indicator whether the line is active (simulates delete operation).

Displayed only in Target Physical Models

MD_TRANSENDDATE

Transaction end date.

Displayed only in Target Physical Models

MD_DELETEDDATE

Delete date (date record was marked as inactive).

Displayed only in Target Physical Models

MD_CREATEDDATE

Date record was created.

Displayed only in Target Physical Models

MD_LASTMODDATE

Date record was last modified and updated.

Displayed only in Target Physical Models

MD_FLAG

Flag that indicates whether row is new, updated, or requires a delete.

Displayed only in Extractions Physical Models

MD_PF_FLAG

Flag that indicates whether row is pre-filled or not.

Displayed only in Extractions Physical Models

MD_CP_ID Source instance ID