Administer > Configure streaming of activity and windows event logs

Configure streaming of activity and windows event logs

OMi MP for Microsoft Azure enables data collection from the activity and windows event log file. The log data is structured in a specific format to provide information about timestamp, host, and message. The structured data is further processed by Operations Agent and forwarded to required targets (example, Operations Bridge Analytics). Operations Bridge Analytics can use these logs to predict trends and pinpoint issues in your IT environment.

OMi MP for Microsoft Azure provides Microsoft Azure Log Streaming aspect that enables the streaming of azure logs for both activity and windows event logs.

The log streaming capability is supported from Operations Manager i 10.63 (or later) and Operations Agent 12.05 (or later) versions.

Microsoft Azure Log Streaming aspect contains MSAzure_ActivitylogStreaming, MSAzure_WindowsEventlogStreaming and Data Forwarding policies that must be configured prior to deploying the aspect. Data from the log file is converted to structured data based on the mapping and conditions defined in the MSAzure_ActivitylogStreaming and MSAzure_WindowsEventlogStreaming policies. Operations Agent processes the data and forwards it to the target specified in the Data forwarding policy.

Before you deploy the Microsoft Azure Log Streaming aspect, ensure the following:

  • The Microsoft Azure Activity Logs aspect is deployed.

  • The Microsoft Azure Windows Event Log Monitoring aspect is deployed.
  • The MSAzure_ActivitylogStreaming and MSAzure_WindowsEventlogStreaming policies are configured.

Deploy Aspect and Policies for Azure Log Streaming

You must deploy the aspects and policies in the following order for Azure log streaming:

  1. Configure the MSAzure_ActivitylogStreaming or MSAzure_WindowsEventlogStreaming policy as follows. Else, go to step 3.

    The MSAzure_ActivitylogStreaming policy collects data from the Azure Cloud and will create activity log on the Remote Node at the following location %ovdatadir%/tmp/MSAzure/MSAzure_Activity_Log.xml file at an interval of 5 minutes. This policy normalizes all the MSAzure Activitylog and converts to JSON data.

    The MSAzure_WindowsEventlogStreaming policy collects data from the Azure Cloud and will create windows event log on the Remote Node at the following location %OvAgentDir%log\MSAzure\MSAzure_WindowsEventLog.log at an interval of the 5 minutes frequency. This policy normalizes the Azure Windows Event log Messages and converts to JSON data.

    An event is generated if the log file does not exist in the specified location. The data collected is structured as <custom_string1><custom_string2><custom_string3><custom_string4><message><timestamp>. This log file structure must not be modified.

    In this instance:

    For Windows Event Log:

    <custom_string1> is channel
    <custom_string 2> is instance name
    <custom_string3> is Event ID

    For Activity Log:

    <custom_string1> is Category
    <custom_string 2> is Subscription ID
    <custom_string3> is Event ID
    <custom_string4> is properties

    By default, data log file is always read from the last position. If you need to load the data from the first time, then you can configure the policy.

    Follow the steps to configure the policy:

    1. Open the Management Templates & Aspects pane:

      Click Administration > Monitoring > Management Templates & Aspects.

    2. In the Configuration Folders pane, expand Cloud Management > Microsoft Azure > Aspects.
    3. In the Management Templates & Aspects pane, select the Microsoft Azure Log Streaming Aspect and click Edit.
    4. In the Policy Templates tab, select the MSAzure_ActivitylogStreaming or MSAzure_WindowsEventlogStreaming policy and click > Edit Policy Template. The Edit window is displayed.
    5. In the Source tab, you can set the Read Mode as either Read from beginning (always) or Read from beginning (first time). By default, the Read Mode is set Read from last position to option.

      Make sure not to modify any other default setting related to data field in source, schema or mapping tab.

    6. Click Save and Close.

    The policy with version incremented by 0.1 is created.

  2. To specify the target and filter conditions, configure the MSAzure_DataForwarding policy as follows.

    This policy forwards the data received from log file to a target server.

    1. Open the Management Templates & Aspects pane:

      Click Administration > Monitoring > Management Templates & Aspects.

    2. In the Configuration Folders pane, expand Cloud Management > Microsoft Azure > Aspects.
    3. In the Management Templates & Aspects pane, select the Microsoft Azure Log Streaming Aspect and click Edit Item.
    4. In the Policy Templates tab, select the MSAzure_DataForwarding policy and click > Edit Policy Template.
    5. In the Targets tab, select the Target server in the Data Forwarding Targets pane. Specify the FQDN along with port for the target in the URL field.

      Follow the steps to add multiple target server:

      1. In the Name field, specify a name for the target.
      2. In the Description field, specify description.
      3. In the URL field, specify the FQDN of the endpoint target to forward the log data.
      4. Select the Wire Format. By default, JSON is supported in OMi MP for Microsoft Azure.
    6. In the Structured input tab, specify the following:

      1. Click Create New Rule > Forward on matched. The Forwarding Rule Details section is enabled.
      2. In the Property tab, specify a description for the condition.
      3. In the Condition tab, follow the steps:

        1. Click Create New Expression. The equals option is displayed.
        2. Expand the equals and in the Property field, enter a value.
        3. In the Operator, select the operator.
        4. In Operand field, enter a value.

        By default, device_product=Azure is the out of the box defined expression.

        Create new expression with different values and operator. You can enter the name of the property or drag it from the Meta Data tab (right pane).

      4. In the Targets tab, select the required target.
    7. Click Save and Close.

    The policy with version incremented by 0.1 is created.

  3. Edit the Microsoft Azure Log Streaming aspect to include the latest version of MSAzure_ActivitylogStreaming, MSAzure_WindowsEventlogStreaming and MSAzure_DataForwarding policies and click Save.
  4. Deploy the latest version of Microsoft Azure Log Streaming aspect.