Troubleshooting and Limitations – Layer2 Discovery

Most of Layer2 discovery jobs are SNMP-based, you must meet the following conditions to perform the actual discovery:

• Device must support the used Management Information Bases (MIBs).
• Discovery user (or community for SNMP v1-v2) must have read permission for the required Object Identifiers (OIDs). The list of used OIDs is specified in the Required Permissions section of the corresponding adapter and in the Universal CMDB Discovery and Integration Content Permissions.

To identify whether you deal with a permission issue, you should examine the communication log and check the fetched results. If SNMP does not clearly state that you are not allowed to get some data, and it returns nothing or claims that "required OID doesn't exist", it is strongly recommended that you use the snmpwalk tool to manually check if the data is reachable.

• Problem: The Layer2 Topology VLAN-based by SNMP job is missing triggers for a particular switch.

  Cause: The VLANs by SNMP or VLAN ports by SNMP job fails to consistently run against the Switch.

  Solution:

  1. Check the corresponding communication logs if the permissions are set correctly.
  2. If that is SNMPv3, the Content Pack version is prior to CP17, and those are Cisco devices, it is a limitation because "snmp v3 vlan context handling in CISCO style" is not supported. This issue is solved in CP17.
  3. On some SNMP agents, there is a bug for SNMPv2 for "Community Postfix" and interface state related MIB, the relevant requests will return nothing even though the permissions are set correctly. This issue can be solved by the agent upgrade.

• Problem: The Layer2 Topology VLAN-based by SNMP or Layer2 Topology Bridge-based by SNMP job has run but no Switch-to-Switch relationships are present.

  Solution: Run the Process Layer2 Saved Files job.

• Problem: Some particular switch port is connected to a server or another switch but the required Layer2 Connection CI is not there in UCMDB.

  Solution:

  • For the Layer2 Topology CDP-LLDP based by SNMP job, you can simply check the communication logs and find out what data is fetched for the problematic port.

  • If this is the default flow, do the following:

    1. Prerequisites – the following information is required:

       • Switch Interface MAC address
       • Switch Interface VLAN if any
       • Which Data Flow Probe has performed the discovery and has access to its File System
    2. Go to the Data Flow Probe File System and locate the l2process folder.

    3. Search the file context using the OS native search to look for the files coating the specific MAC address (file names should be in hexadecimal format without extensions)

    4. Among the found files, pick up the one whose first line looks like the following:

       <The MAC address that you look for >:::<Interface Name>:::<digits>:::<digits>:::<digits>:::<digits>:::<digits>:::<Name>:::<Your VLAN ID or None>

       This file contains the interface data in the first line, and the ARP cache for the interface in the specific VLAN.

    5. Check how many MAC addresses are present in the second line and if the server side MAC address is present there. As a rule, the referenced MAC address is present but along with other MAC addresses, which prevents the job from creating a Layer2 relationship because it is considered to be a Switch-to-Switch relationship.

    6. If the second attached device is also a switch, you must repeat the above-mentioned steps to find the file for that interface. In case of the Switch-to-Switch relationship, the issue might appear only if those switches and the relevant topology are discovered by different Data Flow Probes. Therefore, the Process Layer2 Saved Files job does not simply have all the required information in one place.

       To fix this issue, copy the content of the l2process directory to a single Data Flow Probe and run the Process Layer2 Saved Files job. Such copy must be done on a scheduler basis. This is the limitation of this Flow.

    7. If the second attached device is a server, it means that one of the following happens:

       • There is some other network device in the middle that is not discovered.

         Solution: Find out what it is and discover it if possible.

       • The remote box is some kind of virtualization solution box, such as X-Frame, or VMware ESX Server. The default flow does not support the virtualization solution hardware boxes. This is a limitation.

       • In addition to the real MAC address, some virtual ones are visible from that server. It might happen for some Microsoft based Cluster heartbeat virtual interfaces.

         Solution: Filter out the virtual MAC address to keep only one physical MAC address present.

Problem: Layer2 Connection CI is reported but the following reconciliation error occurs "CIs were ignored due to Multiple Match ...".

Cause: This issue occurs if UCMDB fails to properly identify one of the reported nodes. Most often, this issue occurs when the Layer2 Topology VLAN-based by SNMP job runs. The fact is that the only thing that is known about the remote server is the MAC address, which means that if there are more than one interface with such a MAC address in UCMDB, the reported Layer2 topology will fail to reconcile.

Possible Solution: Run the Report Linux with Duplicated MAC Layer2 job if the "remote" server is Linux and it is discovered. Another option to get the ignored relationship back is to run the Build Layer 2 CIs using PortNextMAC enrichment rule, which will report the same topology but based on CI IDs thus solving the reconciliation problem.