Loading

×Sorry to interrupt
CSS Error

Refresh

Cookie Preference Center

General Information on cookies

When you visit our website, we use cookies to ensure that we give you the best experience. This information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies by clicking on the different category headings to find out more and change your settings. However, blocking some types of cookies may impact your experience on the site and the services we are able to offer. Further information can be found in our

Cookie Policy.

Strictly Necessary

Strictly Necessary

Always Active

Strictly Necessary

Strictly necessary cookies are always enabled since they are essential for our website to function. They enable core functionality such as security, network management, and website accessibility. You can set your browser to block or alert you about these cookies, but this may affect how the website functions. For more information please visit www.aboutcookies.org or www.allaboutcookies.org.

Cookie Details‎

Performance

Performance

Performance

Performance cookies are used to analyze the user experience to improve our website by collecting and reporting information on how you use it. They allow us to know which pages are the most and least popular, see how visitors move around the site, optimize our website and make it easier to navigate.

Cookie Details‎

Please confirm your settings by refreshing the page.

Functional

Functional

Functional

Functional cookies help us keep track of your past browsing choices so we can improve usability and customize your experience. These cookies enable the website to remember your preferred settings, language preferences, location and other customizable elements such as font or text size. If you do not allow these cookies, then some or all of these services may not function properly.

Cookie Details‎

Advertising

Advertising

Advertising

Advertising cookies are used to send you relevant advertising and promotional information. They may be set through our site by third parties to build a profile of your interests and show you relevant advertisements on other sites. These cookies do not directly store personal information, but their function is based on uniquely identifying your browser and internet device.

Cookie Details‎

Advertising Cookies

Consent Leg.Interest

Select All Vendors

Select All Vendors

Select All Hosts

Select All

Clear Filters

Information storage and access

Apply

Confirm My Choices Reject All Allow All

My Subscriptions

Loading

Knowledge Base

Network adapter (vmnic) is down or fails with a failed criteria code (2014553)

Last Updated: 2/26/2024Categories: TroubleshootingTotal Views: 129441Language:
Chinese (Simplified)JapaneseEnglish

58 like this article

Subscribe to get an email
newsletter with new updates

58 people found this helpful Please provide article feedback:

Learn how VMware Skyline Advisor and Skyline Health Diagnostics work together to provide proactive intelligence and self-service log analysis.

Use Skyline Advisor to avoid issues before they occur across VCF, vSphere, vSAN, NSX, vROps & Horizon. Skyline Findings are based on top KBs, VMSAs/CVEs and design best practices.

Use Skyline Health Diagnostics to root cause vSphere & vSAN logs when an issue occurs. Skyline Health Diagnostic Findings are based on top KBs and VMSA/CVEs. You can use the Skyline Health Diagnostics tool in online or offline mode.

If you require additional help from technical support, please use Skyline Advisor to initiate a Log Assist to reduce time to upload logs. TSEs can also initiate logs to reduce effort.

Symptoms

ESXi host may show "network redundancy lost" or "network redundancy degraded" alarm.

The ESXi host may show some vmnics as "DOWN" currently.

If no vmnics are down unexpectedly, yet you see redundancy lost messages, there may be NIC flapping.

You may see an error similar to:

Lost network connectivity on virtual switch "vSwitch0". Physical NIC vmnic1 is down. Affected portgroups: "VM Network".

When the NIC Teaming network adapter fails with a failed criteria code, you may get this error in the vobd log similar to:

Nov 28 22:25:19 10.2.0.103 Nov 29 03:25:19 vobd: Nov 29 03:25:19.927: 8596737035540us: [vob.net.dvport.uplink.transition.down] Uplink: vmnic2 is down. Affected dvPort: 132/5d 0d 2c 50 4a 71 82 e7-54 c7 69 ed b2 01 6f bc. 1 uplinks up. Failed criteria: 130.

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

A physical NIC may be brought down due to frequent link status flapping

When a physical NIC experiences frequent link status flapping, ESXi may turn off the physical NIC to avoid unnecessary link failover and network instability.

For example, a physical NIC driver may reset the physical NIC when it detects "tx Hang". A reset causes a link down/up event to be generated. If the physical NIC is itself bad, this causes periodic reset and link status flapping. The vmkernel.log log file on the ESXi host will show that one or more network cards and constantly changing from a Down to Up state and visa versa. During link status flapping, traffic may migrate between different physical NICs. If the flapping is too frequent, you experience network issues, such as bandwidth reduction or an unstable network.

Purpose

This article provides information on troubleshooting issues when a network adapter fails. The ESXi/vCenter UI and ESXi logs show NIC adapter alerts and messages.  This KB goes over typical checks that can be done for troubleshooting.

For more adapter troubleshooting steps, please open a support ticket with the hardware vendor.

Impact / Risks

The services associated with the affected portgroups are disconnected from the associated physical networks: virtual machines, service console, VMotion, NFS, iSCSI, and Management Services.

Resolution

To determine the cause of the failure or eliminate common NIC issues:

Check the current status of the vmnic from either the VMware vSphere Client or the ESXi service console:

To check the status from the vSphere Client:

Select the ESX host and click the Configuration tab.

Click Networking.

The vmnics currently assigned to virtual switches are displayed in the diagrams. If a vmnic displays a red X, that link is currently down.

To check the status from the ESXi service console, run this command:

esxcli network nic list

The output appears similar to this:

Name     PCI Device    Driver  Admin Status  Link Status   Speed  Duplex  MAC Address         MTU  Description
-------  ------------  ------  ------------  -----------  ------  ------  -----------------  ----  -----------
vmnic0   0000:01:00.0  ixgben  Up            Up             1000  Full    ec:f4:bb:da:b6:c8  1500  Intel(R) Ethernet Controller X540-AT2
vmnic1   0000:01:00.1  ixgben  Up            Up             1000  Full    ec:f4:bb:da:b6:ca  1500  Intel(R) Ethernet Controller X540-AT2

Note: The Admin Status is the only portion of the output that ESXi controls. This is done by using the these commands:

esxcli network nic down -n vmnicX
esxcli network nic up -n vmnicX

The Link Status column specifies the status of the link between the physical network adapter and the physical switch. The status can be either Up or Down. If there are several network adapters, with some being up and some down, you might have to verify that they are connected to the intended physical switch ports. This can be done by bringing down each of the ESXi host's ports on the physical switch and running the command to observe which vmnic is affected.

2. Check that the vmnic referred to in the event message is still connected to the switch and configured properly:

Make sure that the network cable is still connected to the switch and to the host.

Check that the switch connected to the system is still functioning properly and has not been misconfigured. Refer to the switch documentation for details.

Check for activity between the physical switch and the vmnic. This might be indicated either by a network trace or activity LEDs.

Check that the NIC driver is up to date: Determining Network/Storage firmware and driver version in ESXi.

3. Search for the word "vmnic" in the vobd log file.

If you see "vmnic down" or "vmnic up" messages, the NIC may be flapping. Note: Some NICs report the NIC link up state only, not down. If the NIC is reported as "up" and the host was not rebooting, this is an indication that the NIC is flapping and not reporting the down state to ESXi.

Check for a failed criteria code with the vmnic messages. If there is a failed criteria code listed, please see step 4 below.

If there is no failed criteria code, and everything was checked in step 2 above, call the hardware vendor about this flapping.

4. In the vobd file,the vmnic failure may be classified with a Failed criteria code in this log. This code explains the reason for the vmnic failure.

Example:

2020-11-17T15:37:00.330Z: [netCorrelator] 4836107000843us: [vob.net.dvport.uplink.transition.down] Uplink: vmnic4 is down. Affected dvPort: 28/50 24 e2 d9 41 e2 48 58-7d da b4 fd 4a bd 37 92. 3 uplinks up. Failed criteria: 128

Time - Event - Uplink# - State - Port - vSwitch - # Active Uplinks left - Failed Criteria

Note: # Active Uplinks left is indication of a failover which identifies the number of active uplinks left in the teaming policy of the virtual switch.

The following are the failed criteria codes.

1 – Link speed reported by the driver (exact match for compliance)
2 – Link speed reported by the driver (equal or greater for compliance)
4 – Link duplex reported by the driver

8 – Link LACP state down
32 – Beacon probing
64 – Errors reported by the driver or hardware
128 – Link state reported by the driver

256 – The port is blocked
512 – The driver has registered the device

Note: Failed Criteria 128 is driver reporting link state down. This can be caused by unplugging the network cable or administratively downing the physical switchport. If this was not an intended link outage it will likely be an issue with the driver, firmware, SFP+ module, cable, and/or switchport of the physical switch. Check the driver by following the below KB, and call the host hardware vendor for further troubleshooting when Failed criteria 128's are seen in the vobd log. For more information, see Determining Network/Storage firmware and driver version in ESXi.

Note: The failure codes are accumulative so can be added together when multiple criteria are met.

When there are multiple failures, you see entries similar to these in the vobd.log file:

2012-04-05T11:22:10.449Z: [netCorrelator] 1123644995238us: [vob.net.pg.uplink.transition.down] Uplink: vmnic3 is down. Affected portgroup: iSCSI-Netz2. 0 uplinks up. Failed criteria: 130

The failed criteria here is 130, which is 2 + 128. This is a combination of these two failure codes:

Link speed reported by the driver (equal or greater for compliance)

Link state reported by the driver

Related Information

VMware Skyline Health Diagnostics for vSphere - FAQ
For more information, see the vSphere API/SDK Documentation.

The criteria that are used to determine if a network adapter in a network adapter team has failed include:

checkBeacon – By default, this check is disabled. This check becomes active when Beacon Probing is enabled on a virtual switch.

checkDuplex – By default, this check is disabled.

If checkDuplex is true, the configured duplex mode is fullDuplex and the link is considered to be bad if the link duplex reported by driver is not the same as fullDuplex

If checkDuplex is false, fullDuplex is unused and the link duplexity is not used as a detection method.

checkErrorPercent – By default, this check is disabled.

If checkErrorPercent is true, the percentage mentioned in the criteria is the configured error percentage that is tolerated. The link is considered to be bad if error rate exceeds percentage.

If checkErrorPercent is false, percentage is unused, and error percentage is not used as a detection method.

checkSpeed – The default setting is Minimum and has a default value of 10Mbps.

To use link speed as the criteria, checkSpeed must be one of these values:

exact – Use exact speed to detect link failure. Speed is the configured exact speed in megabits per second.

minimum – Use minimum speed to detect failure. Speed is the configured minimum speed in megabits per second.

empty string – Do not use link speed to detect failure. Speed is unused in this case.

The Failed criteria code of 32 indicates the link has failed due to Beacon Probing detecting a problem. Beacon Probing sends beacons per VLAN between physical NICs in a team. When these are not received by other NICs this means that there is a problem in the physical network.

Detectable by VMware SkylineTM

Actions

Copy

link to clipboard
copied!

Print

Language:
Chinese (Simplified)JapaneseEnglish

Attachments

Additional Resources

No results for undefined

Ask The Community

Get answers quickly from VMware experts in the community

Post Subject

CONTINUE IN COMMUNITIES

Related Products:

VMware vSphere ESXi

VMware vSphere ESXi

Related Versions:

VMware vSphere ESXi 7.0.0

VMware vSphere ESXi 6.7

VMware vSphere ESXi 6.5

VMware vSphere ESXi 6.0

VMware vSphere ESXi 5.5

VMware vSphere ESXi 5.0

Take Our Survey

Support Assistant