Reference > Event Based Discovery

Event Based Discovery

Background

In UCMDB, the regular Universal Discovery jobs are scheduled to run, such as once a day or once a week. During the discovery interval, UCMDB cannot detect what have been changed in remote nodes. For the traditional IT infrastructure, this kind of discovery is good enough to reflect the topology of the data center.

In recent years, the Cloud is becoming quite popular among the IT world. Amazon Web Services, OpenStack, VMware vCloud, Cloud Foundry, and Docker are rapidly adopted by many companies. One of common characters of these technologies is that changes happen easily and frequently. However, the regular Universal Discovery cannot accurately handle such changes in real time. For example:

  • No clues of the existence of nodes. One virtual machine is created for testing at 9:00, and then it is terminated at 16:00 after the testing. Its administrator is not even aware of its existence because most of regular discovery jobs are scheduled to run at midnight.
  • Topology in UCMDB cannot reflect the current status of the Cloud environment. The typical aging time in UCMDB is more than one month. In traditional data centers, nodes are rarely changed. Generally, regular discovery jobs do not delete CI. Therefore, the topology in UCMDB is consistent with the actual environment in most time. However, in the Cloud environment, nodes are created and deleted at any time. After a while, many nodes that are deleted in the Cloud environment still exist in UCMDB. These nodes become a kind of "noise" before they are aged.

Overview

In order to adjust to the Cloud environment, the event based discovery is introduced. This discovery is based on the event that is sent out from Cloud providers in real time. The event based discovery uses a framework called Event Hub to capture such events and report them into UCMDB. Thus, the topology in UCMDB can reflect the actual environment as much as possible.

So far, the following event based discoveries for Cloud environments are introduced:

Discovery Mechanism

This section describes the discovery mechanism for the event based discovery.

ClosedEvent Hub

Event Hub is a new framework designed for the event based discovery. It includes the following:

  • Event Source. Collects events from event providers and puts them in Event Queue.
  • Event Filter. Filters events to only allow specific events to pass through.
  • Event Handler. Handles events and reports them as CIs to UCMDB.

When Event Source starts, it continuously listens to or fetches events from event providers. After events arrive, the events are buffered in a memory queue. Meanwhile, there is a thread pool that is waiting for new events to pull them from the queue. After a new event is pulled, a thread uses Event Filter to filter the event. Then, the filtered event is passed to Event Handler. In Event Handler, one typical handler parses the event and constructs the CI topology to report to UCMDB.

ClosedContinuously running jobs

One important difference between an event based discovery job and a regular job is that the event based discovery job runs continuously until the job is manually deactivated or terminated by Data Flow Probe due to exceeding the maximum execution time.

In UCMDB 10.22, the maximum execution time of jobs is 2147483647 milliseconds (about 24.86 days). Thus, the job needs to be rerun after the timeout. This is a limitation. In UCMDB 10.30, if the maximum execution time of jobs is set to 0 (zero), the job will not reach the time limit.

ClosedEvent providers

So far, the following Event Source providers are implemented:

  • Cloud Foundry
  • OpenStack
  • VMwarevCloud
  • Docker Swarm

Limitation

Limitation. In UCMDB 10.30 and earlier versions, using multiple threads in the event-based discovery may cause the missing of events.

Workaround. It is recommended to upgrade UCMDB to 10.31 or a later version. For UCMDB 10.30 and earlier versions, setting useMultiThreadForEventHub to false in globalsetting.xml; however, this operation may affect the performance of event-base discovery jobs. For details about this setting, see globalSettings.xml File.