Integration Studio

The Integration Studio is where you manage your UCMDB integration points and connect and share information with external repositories, such as other CMDBs, IT Performance Suite products, or third-party products.

Integration with other products is performed over secure communication channels through Data Flow Probes.

Alternatively, if your remote managed data repositories are accessible from the UCMDB server machine, non-Jython-based integrations can be performed using the UCMDB Integration Service, enabling the Data Flow Probe resources to be used for other discovery tasks.

Integration points in the CMDB are based on adapters, which are entities that are capable of communicating with external data repositories. A basic set of adapters is provided with the CMDB; however, you can create additional adapters using the Federation Framework SDK. For details, see Add an Adapter for a New External Data Source.

You can also create adapters in the Adapter Management module. For details, see Resources Pane.

For details about how to set up integration points for data integrations, see Integration Studio Page.

Integration points can be of one of the following types:

Population

An integration of Population type copies data from an external data repository into the CMDB, so that the CMDB now controls the data.

You use population in one of the following scenarios:

• When you need to track changes made by the CMDB at the CI level.

• When a remote repository is not reliable in terms of response time; for example, a network delay prohibits you from setting up run-time federation with the repository.

• When a remote repository does not support federation capabilities (no appropriate adapter exists).

Federation

An integration of Federation type includes data in the CMDB from other sources in such a way that the source of the data still retains control of the data.

You use the CMDB's federation capabilities to extend the scope of the existing Topology Query Language (TQL) capabilities to encompass data that is stored and maintained in an external repository. The ability to include such information is important as it prevents you from having to copy large amounts of data and, instead, bring it into your CMDB only when it is really needed.

Federation also has the benefit that the federated data does not burden the CMDB in terms of capacity; theoretically, you can set up an integration that federates trillions of CIs and relationships. Federated data is fetched at runtime, as requested, which lessens the impact on system performance.

Federated integration creates a federated integration point, which can then be used when defining TQL queries. For details on TQL, see Topology Query Language.

Retrieving Data from Multiple Federated Data Sources

During TQL query calculation, you can retrieve data for the same CIT from several federated data sources. The data is retrieved from the local CMDB, as well as from other federated data sources, according to how you have configured integration points. As data arrives at the CMDB, it is identified and reconciled, with the end result determined according to the configured reconciliation priority given to the various integrations.

Each CI that is retrieved from an external data repository includes an attribute (Created By) to show from which federated data source the CI has been retrieved.

For limitations, see Limitations – Integration Studio.

Retrieving Attributes from an External Data Repository

• You can retrieve the attributes of a CI from an external data repository, when the core CI data is stored in the CMDB.

• The core data repository must be the CMDB.

• The CIT must be located in a data repository for its attributes to be defined.

• The same attributes can be retrieved from multiple data repositories.

• For details on retrieval options, see the CI Type Retrieval Mode field on the Federation Tab.

• When you configure an integration point to include federated CIs, you must select full federation of a CI or the federation of an attribute alone. You cannot set up two integrations for the same CIT where one is mapped to an external CIT and the other is mapped to that same CIT with an external attribute.

• A CIT can support external attributes if the adapter (which is federating the CIT data) supports mapping information (reconciliation) for this CIT.

Reconciling Information

Federated queries should use the mapping file to reconcile the CI from the CMDB with the attributes from the external data repository.

For details on the Mapping Engine, see Federation Framework for Federated TQL Queries.

For details on selecting attributes to be included in the federation, see Federation Tab.

For details on how reconciliation is performed, see Data Reconciliation.

Use Cases

• You need to discover the SMS or Altiris desktops in your system. The desktop CIT is a core CIT and is already synchronized with the CMDB. However, you do not want to store all the desktop data in the CMDB as this is inefficient and unnecessary. It is enough to store core attributes such as name and MAC address in the CMDB, and to define the other details of the desktops as external attributes in two data repositories: SMS and Altiris.
• VMware creates virtual machines that contain a virtual machine monitor (hypervisor) that allocates hardware resources dynamically and transparently. Multiple operating systems can run concurrently on a single physical computer. Since the allocation resources—for example, memory—are dynamic, DFM cannot discover these resources (DFM runs once every 24 hours and the resource data can change hourly). To enable UCMDB to always be updated with real-time data, the solution is to divide the data into two: the core data of the virtual hosts should be discovered and placed in the CMDB; the resource attributes should be retrieved from the external source. In this use case, the data for these attributes is retrieved from two data repositories: CMDB and VMware.

Calculating Federated TQL Queries

When defining an adapter, you can specify whether calculation of TQL queries must be performed first on the CMDB (default) or whether it can start on the adapter.

As an example of a one-node adapter, if you have a TQL query of Node > CPU (with conditions on the federated CPU):

• If the calculation is performed first on the data in the CMDB:

  • The TQL query for Node is calculated in the CMDB, which retains all of the Node's data.

  • Then Node > CPU is calculated by the adapter, which uses the reconciliation data from the previous step.

• If the calculation is performed first by the adapter:

  • The adapter calculates the TQL query for CPU, and returns the connected nodes as the reconciliation data.

  • Then the calculated data is sent to the CMDB, where the TQL query for Node is calculated according to the reconciliation data from the previous step.

The option to set the adapter as the starting point of the TQL query calculation is specified in the Adapter Management module. For details, see Adapter Source Editor Window.

Data Push

An integration of Data Push type copies data from the CMDB to an external data repository, so that the CMDB no longer retains control over this data.

You use data push integrations to feed important data from your CMDB into an external system to facilitate your necessary business processes. An example of this is pushing data discovered by DFM into  Service Manager, where tickets may be opened that are connected to the actual CIs in your IT infrastructure.

If an authorized state has been defined, you can perform data push from the authorized or actual state.

Data push architecture enhancement

In version 10.30, data push architecture is enhanced to separate the internal resources used by different integrations, localizing at integration point level, resources otherwise shared between the integration points, which, when run in parallel with large amount of data may prove slow. By reducing such shared resources, big improvements can be observed in a concurrent running scenario. Now the engine parallelism can be better exploited through the number of threads allocated. A new JMX setting com.hp.ucmdb.synchronizer.manager.SynchronizerManagerFactory is introduced, which allows you to increase the number of threads for data push jobs from the out-of-the-box value 3 to any desired value. For more information, see How to Increase the Number of Threads for Data Push Jobs.

In version 10.30, the number of CIs from a data push TQL query is increased from 5 million to 20 million on Oracle.

dal.object.condition.max.result.size=99999999
dal.link.condition.max.result.size=99999999
dal.use.memory.instead.temp.table.high.threshold.oracle=60000000
model.change.percent.statistics.threshold=30
replication.timeout=180000000
tql.dfs.max.temp.results=200000000
tql.max.objects.visit.model.calc.task=200000000
push.remote.action.timeout=12000000

For limitations on Data Push jobs, see Limitations – Integration Studio.