Loading

×Sorry to interrupt
CSS Error

Refresh

Cookie Preference Center

General Information on cookies

When you visit our website, we use cookies to ensure that we give you the best experience. This information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies by clicking on the different category headings to find out more and change your settings. However, blocking some types of cookies may impact your experience on the site and the services we are able to offer. Further information can be found in our

Cookie Policy.

Strictly Necessary

Strictly Necessary

Always Active

Strictly Necessary

Strictly necessary cookies are always enabled since they are essential for our website to function. They enable core functionality such as security, network management, and website accessibility. You can set your browser to block or alert you about these cookies, but this may affect how the website functions. For more information please visit www.aboutcookies.org or www.allaboutcookies.org.

Cookie Details‎

Performance

Performance

Performance

Performance cookies are used to analyze the user experience to improve our website by collecting and reporting information on how you use it. They allow us to know which pages are the most and least popular, see how visitors move around the site, optimize our website and make it easier to navigate.

Cookie Details‎

Please confirm your settings by refreshing the page.

Functional

Functional

Functional

Functional cookies help us keep track of your past browsing choices so we can improve usability and customize your experience. These cookies enable the website to remember your preferred settings, language preferences, location and other customizable elements such as font or text size. If you do not allow these cookies, then some or all of these services may not function properly.

Cookie Details‎

Advertising

Advertising

Advertising

Advertising cookies are used to send you relevant advertising and promotional information. They may be set through our site by third parties to build a profile of your interests and show you relevant advertisements on other sites. These cookies do not directly store personal information, but their function is based on uniquely identifying your browser and internet device.

Cookie Details‎

Advertising Cookies

Consent Leg.Interest

Select All Vendors

Select All Vendors

Select All Hosts

Select All

Clear Filters

Information storage and access

Apply

Confirm My Choices Reject All Allow All

My Subscriptions

Loading

Knowledge Base

Enabling NetQueue on Intel Gigabit Network Devices Using the igb Driver in ESX/ESXi 4.x and ESXi 5.x (1026094)

Last Updated: 3/24/2017Categories: TroubleshootingTotal Views: 22035Language:
English

Subscribe to get an email
newsletter with new updates

Please provide article feedback:

Details

The asynchronous version of the ESX/ESXi 4.x and ESXi 5.0/5.1 igb driver uses VMware's NetQueue technology to enable Intel Virtual Machine Device Queues (VMDq) support for Ethernet devices based on the Intel 82576 and 82580 Gigabit Ethernet Controllers. VMDq is optional and disabled by default.

Solution

Enabling VMDq

To enable VMDq:

Ensure the correct version of the driver is installed and enabled to load automatically at boot:

# esxcfg-module -e igb

Notes:

Currently this is supported on Intel’s ESX/ESXi 4.x and ESXi 5.0/5.1 drivers (igb version 400.2.4.10 and higher) for 82576 and 82580 Gigabit Ethernet Controllers.

This method does not work for ESXi 5.5.x. Intel has discontinued VMDq in the igb driver versions 4.2.16.8 and newer for ESXi 5.5. for more information, see Discontinuing NetQueue Support in the igb Driver Since ESXi 5.5 and Later (2090693).

Set per each port the optional VMDq load parameters for the igb module.

Configure IntMode=2. Setting a value of 2 for this option specifies using MSI-X, which enables the Ethernet controller to direct interrupt messages to multiple processor cores. MSI-X must be enabled in order for NetQueue to work with VMDq.

Set a value for the parameter to indicate the number of transmit and receive queues. The parameter value ranges from 1 to 8 since Intel 82576 and 82580 based network devices provide a maximum of 8 transmit queues and 8 receive queues per port. The value used sets both the transmit and receive queues to the same number.

For a quad-port adapter, the following configuration turns on VMDq in full on all four ports:

# esxcfg-module -s "IntMode=2,2,2,2 VMDQ=8,8,8,8" igb

The VMDq configuration is flexible. Systems with multiple ports are enabled and configured by comma-separated lists. The values are applied to the ports in the order they are enumerated on the PCI bus.

For example:

# esxcfg-module -s IntMode=0,0,2,2, ... ,2,2 VMDQ=1,1,8,8, ... ,4,4 igb

Shows:

The values configured for ports 1 and 2 are: IntMode=0 and VMDQ=1

The values configured for ports 3 and 4 are: IntMode=2 and VMDQ=8

The values configured for the last two ports are: IntMode=2 and VMDQ=4

Reboot the ESX host.

Limitation Notes:

With standard sized Ethernet packets (MTU = 1500 or less), the maximum number of ports supported in VMDq mode is 8, with each port using 8 transmit and receive queues.

When using Jumbo Frames (MTU between 1500 and 9000) and VMDq, the maximum number of supported ports is 4, and also the number of transmit and receive queues per port must be reduced to 4.

Verifying that VMDq is enabled

To verify that VMDq is enabled:

Check the options configured for the igb module:

# esxcfg-module -g igb

The output should appear similar to:

igb enabled = 1 options = 'IntMode=2,2,2,2,2,2,2,2 VMDQ=8,8,8,8,8,8,8,8'

The enabled value must equal 1, which indicates the igb module will load automatically. IntMode and VMDQ must be set for each port. The example above shows a configuration with 8 ports, where all interfaces are configured in full VMDq mode.

Determine which ports use the igb driver using esxcfg-nics. Confirm the driver successfully claimed all supported devices present in the system (enumerate them using lspci and compare the list with the output of esxcfg-nics -l). Query the statistics on each interface using ethtool. If VMDq has been enabled successfully, statistics for multiple transmit and receive queues are shown (see tx_queue_0 through tx_queue_7 and rx_queue_0 through rx_queue_7 in the example below).

# esxcfg-nics -l

Name PCI Driver Link Speed Duplex MAC Address MTU Description
vmnic0 04:00.00 bnx2 Up 1000Mbps Full xx:xx:xx:xx:xx:xw 1500 Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T
vmnic1 08:00.00 bnx2 Down 0Mbps Half xx:xx:xx:xx:xx:xx 1500 Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T
vmnic2 0d:00.00 igb Up 1000Mbps Full xx:xx:xx:xx:xx:xy 1500 Intel Corporation 82576 Gigabit Network Connection
vmnic3 0d:00.01 igb Up 1000Mbps Full xx:xx:xx:xx:xx:xz 1500 Intel Corporation 82576 Gigabit Network Connection
vmnic4 0e:00.00 igb Up 1000Mbps Full xx:xx:xx:xx:xx:x1 1500 Intel Corporation 82576 Gigabit Network Connection
vmnic5 0e:00.01 igb Up 1000Mbps Full xx:xx:xx:xx:xx:x2 1500 Intel Corporation 82576 Gigabit Network Connection
vmnic6 10:00.00 igb Up 1000Mbps Full xx:xx:xx:xx:xx:x3 1500 Intel Corporation 82580 Gigabit Network Connection
vmnic7 10:00.01 igb Up 1000Mbps Full xx:xx:xx:xx:xx:x4 1500 Intel Corporation 82580 Gigabit Network Connection
vmnic8 10:00.02 igb Up 1000Mbps Full xx:xx:xx:xx:xx:x5 1500 Intel Corporation 82580 Gigabit Network Connection
vmnic9 10:00.03 igb Up 1000Mbps Full xx:xx:xx:xx:xx:x6 1500 Intel Corporation 82580 Gigabit Network Connection

# lspci | grep -e 82576 -e 82580

0d:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
0d:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
0e:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
0e:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
10:00.0 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)
10:00.1 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)
10:00.2 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)
10:00.3 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)

# ethtool -S vmnic6

NIC statistics:
rx_packets: 0
tx_packets: 0
rx_bytes: 0
tx_bytes: 0
rx_broadcast: 0
tx_broadcast: 0
rx_multicast: 0
tx_multicast: 0
multicast: 0
collisions: 0
rx_crc_errors: 0
rx_no_buffer_count: 0
rx_missed_errors: 0
rx_aborted_errors: 0
tx_carrier_errors: 0
tx_window_errors: 0
tx_abort_late_coll: 0
tx_deferred_ok: 0
tx_single_coll_ok: 0
tx_multi_coll_ok: 0
tx_timeout_count: 0
rx_long_length_errors: 0
rx_short_length_errors: 0
rx_align_errors: 0
tx_tcp_seg_good: 0
tx_tcp_seg_failed: 0
rx_flow_control_xon: 0
rx_flow_control_xoff: 0
tx_flow_control_xon: 0
tx_flow_control_xoff: 0
rx_long_byte_count: 0
tx_dma_out_of_sync: 0
tx_smbus: 0
rx_smbus: 0
dropped_smbus: 0
rx_errors: 0
tx_errors: 0
tx_dropped: 0
rx_length_errors: 0
rx_over_errors: 0
rx_frame_errors: 0
rx_fifo_errors: 0
tx_fifo_errors: 0
tx_heartbeat_errors: 0
tx_queue_0_packets: 0
tx_queue_0_bytes: 0
tx_queue_0_restart: 0
tx_queue_1_packets: 0
tx_queue_1_bytes: 0
tx_queue_1_restart: 0
tx_queue_2_packets: 0
tx_queue_2_bytes: 0
tx_queue_2_restart: 0
tx_queue_3_packets: 0
tx_queue_3_bytes: 0
tx_queue_3_restart: 0
tx_queue_4_packets: 0
tx_queue_4_bytes: 0
tx_queue_4_restart: 0
tx_queue_5_packets: 0
tx_queue_5_bytes: 0
tx_queue_5_restart: 0
tx_queue_6_packets: 0
tx_queue_6_bytes: 0
tx_queue_6_restart: 0
tx_queue_7_packets: 0
tx_queue_7_bytes: 0
tx_queue_7_restart: 0
rx_queue_0_packets: 0
rx_queue_0_bytes: 0
rx_queue_0_drops: 0
rx_queue_0_csum_err: 0
rx_queue_0_alloc_failed: 0
rx_queue_1_packets: 0
rx_queue_1_bytes: 0
rx_queue_1_drops: 0
rx_queue_1_csum_err: 0
rx_queue_1_alloc_failed: 0
rx_queue_2_packets: 0
rx_queue_2_bytes: 0
rx_queue_2_drops: 0
rx_queue_2_csum_err: 0
rx_queue_2_alloc_failed: 0
rx_queue_3_packets: 0
rx_queue_3_bytes: 0
rx_queue_3_drops: 0
rx_queue_3_csum_err: 0
rx_queue_3_alloc_failed: 0
rx_queue_4_packets: 0
rx_queue_4_bytes: 0
rx_queue_4_drops: 0
rx_queue_4_csum_err: 0
rx_queue_4_alloc_failed: 0
rx_queue_5_packets: 0
rx_queue_5_bytes: 0
rx_queue_5_drops: 0
rx_queue_5_csum_err: 0
rx_queue_5_alloc_failed: 0
rx_queue_6_packets: 0
rx_queue_6_bytes: 0
rx_queue_6_drops: 0
rx_queue_6_csum_err: 0
rx_queue_6_alloc_failed: 0
rx_queue_7_packets: 0
rx_queue_7_bytes: 0
rx_queue_7_drops: 0
rx_queue_7_csum_err: 0
rx_queue_7_alloc_failed: 0

Disabling VMDq

To disable VMDq:

To return the igb driver to default (non-VMDq) mode, erase the optional VMDq load parameters:

#esxcfg-module -s "" igb

Reboot the ESX host.

Detectable by VMware SkylineTM

Actions

Copy

link to clipboard
copied!

Print

Language:
English

Attachments

Additional Resources

KB
•
End Of General Availability of the Free vSphere Hypervisor (ESXi 7.x and 8.x) (2107518)

KB
•
QLogic および Emulex ホスト バス アダプタでの詳細なログを有効化または無効化する (1005576)

Results 1-2 of 2

Ask The Community

Get answers quickly from VMware experts in the community

Post Subject

CONTINUE IN COMMUNITIES

Related Products:

VMware vSphere ESXi

VMware ESX Server

Related Versions:

VMware vSphere ESXi 5.1

VMware vSphere ESXi 5.0

VMware ESXi 4.1.x Installable

VMware ESXi 4.1.x Embedded

VMware ESXi 4.0.x Installable

VMware ESXi 4.0.x Embedded

VMware ESX 4.1.x

VMware ESX 4.0.x

Take Our Survey

Support Assistant