Troubleshoot > General troubleshooting

General troubleshooting

The following section provides information about troubleshooting scenarios:

Discovery

Operations Agent times out before discovery is complete

Problem: Operations agent times out before discovery is complete.

Solution: Increase the Operations Agent time-out on the node.

  1. On the managed node, run the following in the command prompt:

    ovconfchg -edit 
  2. Increase the timeout as following:

    [agtrep]

    ACTION_TIMEOUT=1800

    RESEND_RELATIONSHIP_INSTANCES=TRUE

    [conf.server]

Or

  1. On the managed node, run the following in the command prompt:

    ovconfchg -ns agtrep -set ACTION_TIMEOUT 5
  2. Run the following command to restart :

    ovc -restart agtrep

CIs are not populated for the view MS Azure Classic Topology

Problem: CIs are not populated for the view MS Azure Classic Topology.

Solution: To resolve this problem, follow these steps:

  1. Verify if all prerequisites are met. For more information, see Installation Prerequisite.

  2. Verify if the MSAzure_Classic_Discovery policy is deployed, run the command: ovpolicy -l.

    If the policy is not deployed, then deploy the Azure Classic Discovery Aspect from the OMi Server. For more information about deploying, see Getting Started.

  3. Run the following command and note down the policy ID:

    C:\Windows\System32\ovpolicy -polname "MSAzure_Classic_Discovery" -list -level 1

  4. Check for <policy_id>.out file at the %OvDataDir%\tmp\agtrep folder. If the output file does not exist, then run the following command:

    ovagtrep –run MSAzure_Classic_Discovery

    If discovery runs successfully, then <policy_id>.out file is created at the %OvDataDir%\tmp\agtrep location.

  5. Sync the Classic Azure discovery information available in %OvDataDir%\datafiles\agtrep.xml file to the OMi Server, run the following command:

    ovagtrep –publish –all

    Check if MS Azure Classic Topology view is updated with the latest CIs.

  6. If the CIs are not updated in RTSM, see the following log files:

    On the managed node:

    Azure MP discovery log : %OvDataDir%\log\MSAzure\MSAzure_Run_Topo.log

    Agent log : %OvDataDir%\log\System.txt

    Discovery log on OMi Server:

    • %topaz_home%/log/wde/opr-svcdiscserver.log
    • %topaz_home%/log/wde/opr-svcdiscserver-citrace.log
    • %topaz_home%/log/odb/odb/cmdb.reconciliation.log
    • %OvDataDir %/shared/server/log/OvSvcDiscServer*.log
  7. For detailed trace, perform the following to run the discovery in the debug mode:

    1. From the command prompt, navigate to %OvDataDir%/bin/instrumentation location.
    2. Run MS_Azure_Perl MS_Azure_Run_Topo.pl 1 policy.

    For more information, see the MSAzure_Run_Topo.log file available at %OvDataDir%\log\MSAzure folder.

CIs are not populated for the view MS Azure RM Topology

Problem: CIs are not populated for the view MS Azure RM Topology.

Solution: To resolve this problem, follow these steps:

  1. Verify if all prerequisites are met. For more information, see Installation Prerequisite.

  2. To verify if the MSAzure_RM_Discovery policy is deployed, run the command: ovpolicy -l.

    If the policy is not deployed, then deploy the Azure RM Discovery Aspect from the OMi Server. For more information about deploying, see Getting Started.

  3. To note the policy ID, run the following command:

    C:\Windows\System32\ovpolicy -polname "MSAzure_RM_Discovery" -list -level 1

  4. Check for <policy_id>.out file at the %OvDataDir%\tmp\agtrep\ folder. If the output file does not exist, then run the following command:

    ovagtrep –run MSAzure_RM_Discovery
  5. To sync the Classic Azure discovery information available in %OvDataDir%\datafiles\agtrep.xml to the OMi Server, run the following command:

    ovagtrep –publish –all
  6. If the CIs are not updated in RTSM, see below log files:

    On the managed node:

    Azure MP discovery log : %OvDataDir%\log\MSAzure\MSAzure_Run_Topo.log

    Agent log : %OvDataDir%\log\System.txt

    Discovery log on OMi Server:

    • %topaz_home%/log/wde/opr-svcdiscserver.log
    • %topaz_home%/log/wde/opr-svcdiscserver-citrace.log
    • %topaz_home%/log/odb/odb/cmdb.reconciliation.log
    • %OvDataDir %/shared/server/log/OvSvcDiscServer*.log
  7. For detailed trace, perform the following to run the discovery in the debug mode:

    1. From the command prompt, navigate to %OvDataDir%/bin/instrumentation/ location.
    2. Run MS_Azure_Perl MS_Azure_Run_Topo.pl 1 policy.

Any discovery related errors in the following log files are forwarded to the OMi:

Azure MP discovery log : %OvDataDir%\log\MSAzure\MSAzure_RM_Run_Topo.log

Agent log : %OvDataDir%\log\System.txt

Monitoring

No events and no graphs are shown on the Performance Dashboard

Classic Monitoring

Problem: No events and no graphs shown on the Performance Dashboard for classic monitoring.

Solution: To resolve this problem, follow these steps:

  1. Verify if all prerequisites are met. For more information, see Installation Prerequisite.
  2. Make sure the Microsoft Azure Classic Performance Monitoring Aspect is deployed on the managed node.
  3. To verify if the data related to Azure MP is logged into MS_Azure datasource, run the following from the command prompt: ovcodautil –obj
  4. Verify if both MS_Azure_VMMON and VM_METADATA data class are listed.
  5. If the datasource is not created, follow the steps to create the datasource:

    1. In the command prompt, change directory to the %OvDataDir%\bin\instrumentation directory.
    2. Run the MS_Azure_Perl MS_Azure_MonConfig.pl perl file.

    For more information about the data source, see the MSAzure_MonConfig.log file available at %OvDataDir%\log\MSAzure folder.

  6. Verify if the data is logged in MS_AZURE_VMMON and VM_METADATA data classes.

    If data is not logged, then follow the steps to enable trace and log data:

    1. From the command prompt, change directory to %OvDataDir%\bin\instrumentation folder.
    2. Run the following command: MS_Azure_Perl MS_Azure_Run_Metrics.pl –d

    For more information about the data logging and also errors encountered while running the command, see the MSAzure_Run_Metrics.log file available at the %OvDataDir%\tmp\MSAzure folder.

RM Monitoring

Problem: No events and no graphs shown on the Performance Dashboard for RM monitoring.

Solution: To resolve this problem, follow these steps:

  1. Verify if all prerequisites are met. For more information, see Installation Prerequisite.

  2. Make sure the Microsoft Azure RM Performance Monitoring Aspect is deployed on the managed node.
  3. Verify if the MS_Azure data source is created and data related to Azure MP is logged, run following from the command prompt: ovcodautil –obj
  4. Verify if both MS_Azure_RMVMMON, MS_AZURE_RMVMSTATUS and VM_METADATA data class are listed.

    For more information about metrics for each data class, see the section Performance Metrics.

  5. If the data classes are not created, then follow the steps to recreate the data source:

    1. In the command prompt, change directory to %OvDataDir%\bin\instrumentation directory.
    2. Run the MS_Azure_Perl MS_Azure_MonConfig.pl perl file.

    For more information about the data source, see the MSAzure_MonConfig.log file available at %OvDataDir%\log\MSAzure folder.

  6. Verify if data is logged in MS_AZURE_VMMON and VM_METADATA classes.

    If data is not logged, follow the steps to enable trace file and log data:

    1. From the command prompt, change directory to %OvDataDir%\bin\instrumentation folder.
    2. Run the following command:

    For more information about data logging and also errors encountered while running the command, see the MetricsCollectorLog.txt file available at %OvDataDir%\tmp\MSAzure folder.

    The MSAzure_RMVMCollector_Mon_LogFile policy forwards the error logs to OMi.

Storage Account Monitoring

Problem: No data is logged in MS_AZURE_STORAGEMON or alerts received for Storage Account Monitoring.

Solution: To resolve this problem, follow these steps:

  1. Verify if all prerequisites are met. For more information, see Installation Prerequisite.

  2. Make sure the Microsoft Azure Performance and Microsoft Azure Storage Account Monitoring Aspects are deployed on the managed node.
  3. Verify if the MS_Azure data source is created and data related to Azure MP is logged, run following from the command prompt: ovcodautil –obj.
  4. Verify if both MS_Azure _STORAGEMON data class is listed.

    For more information about metrics for each data class, see the section Performance Metrics.

  5. If the data classes are not created, then follow the steps to recreate the data source:

    1. In the command prompt, change directory to %OvDataDir%\bin\instrumentation directory.
    2. Run the MS_Azure_Perl MS_Azure_MonConfig.pl perl file.

    For more information about the data source, see the MSAzure_MonConfig.log file available at %OvDataDir%\log\MSAzure folder.

High Availability

Problem: Issues related to HA Configuration and Setup

Solution: To resolve this problem, follow these steps:

  1. Verify if all prerequisites are met. For more information, see Installation Prerequisite.

    • Supported Node platforms
    • Supported Azure Powershell
    • Internet Connectivity
    • All nodes of the Operations Connector should belong to the same domain
    • Check the entry in the XPL setting in OvConfg file, run the following command:

      ovconfchg -edit
      [BSMC]
      BSMC_HA_SERVERS=<host1>,<hostb>
      BSMC_HA_SERVERS_TIME=1473754732

      Make sure, BSMC_HA_SERVERS must be a list of hosts building the cluster.

  2. Make sure the Microsoft Azure HA Config Aspect is deployed on the managed nodes individually.
  3. On each managed node, run the ovc command and check if Azure MP HA Sync (Running) and BSMC Zero Downtime (Stopped) components are available.
  4. If the bsmczedo or azurempha component is not available, then follow the steps to configure the component:

    1. Launch command prompt and go to %OvDataDir%/bin/instrumentation location.
    2. Run Azure_bscmzedo_Reg.bat
  5. After the deployment of management template, BSMCZEDO component will be running on all nodes of the HA Setup.

Windows Event Log

Problem: No data for event log monitoring.

Solution: To resolve this problem, follow these steps:

  1. Verify the configuration specified in the MSAzure_WindowsEventLogMon_Config policy are the preferences set in the Diagnostics Setting in Azure portal.
  2. Verify if XML is well-formed after configuring the MSAzure_WindowsEventLogMon_Config policy. If the validation fails, an error is logged in the log file.

Problem: Solution for errors reported in log files

If following errors are logged in the %ovdatadir%log\MSAzureMP_ARM_WindowsEventLog_Mon.log file, then perform corresponding solution steps.

Error 1: Validation failed. Further Processing stops.Please check the format of Contents of Config File policy and deploy again.

Solution 1: Check the if the contents of ConfigFile policy - MSAzure_WindowsEventLogMon_Config is well formed.

Error 2: Exception Errors occured during Processing of Config File. Check the deployment of policies. For more information enable debug.

Solution 2: Check if the Policy config file policy is deployed.

Error 3: Exception Occured while reading and creating filter settings object. Enable debug for more information on the error.

Solution 3: The filter settings are not correct. Follow to enable debug:

MP_ARM_WindowsEventLog_Mon.exe  freq=%%FREQ%% debug=true