Loading

×Sorry to interrupt
Uncaught ReferenceError: addCTracking is not defined
throws at https://kb.vmware.com/resource/KM_TealiumTrackingJS?v=0.5:100:1

Refresh

Cookie Preference Center

General Information on cookies

When you visit our website, we use cookies to ensure that we give you the best experience. This information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies by clicking on the different category headings to find out more and change your settings. However, blocking some types of cookies may impact your experience on the site and the services we are able to offer. Further information can be found in our

Cookie Policy.

Strictly Necessary

Strictly Necessary

Always Active

Strictly Necessary

Strictly necessary cookies are always enabled since they are essential for our website to function. They enable core functionality such as security, network management, and website accessibility. You can set your browser to block or alert you about these cookies, but this may affect how the website functions. For more information please visit www.aboutcookies.org or www.allaboutcookies.org.

Cookie Details‎

Performance

Performance

Performance

Performance cookies are used to analyze the user experience to improve our website by collecting and reporting information on how you use it. They allow us to know which pages are the most and least popular, see how visitors move around the site, optimize our website and make it easier to navigate.

Cookie Details‎

Please confirm your settings by refreshing the page.

Functional

Functional

Functional

Functional cookies help us keep track of your past browsing choices so we can improve usability and customize your experience. These cookies enable the website to remember your preferred settings, language preferences, location and other customizable elements such as font or text size. If you do not allow these cookies, then some or all of these services may not function properly.

Cookie Details‎

Advertising

Advertising

Advertising

Advertising cookies are used to send you relevant advertising and promotional information. They may be set through our site by third parties to build a profile of your interests and show you relevant advertisements on other sites. These cookies do not directly store personal information, but their function is based on uniquely identifying your browser and internet device.

Cookie Details‎

Advertising Cookies

Consent Leg.Interest

Select All Vendors

Select All Vendors

Select All Hosts

Select All

Clear Filters

Information storage and access

Apply

Confirm My Choices Reject All Allow All

My Subscriptions

Loading

Knowledge Base

NSX Manager is using 100% of CPU when pushing a large number of firewall rules or rules to a large number of hosts (55818)

Last Updated: 9/29/2023Categories: InformationalTotal Views: 6122Language:
Chinese (Simplified)JapaneseEnglish

Subscribe to get an email
newsletter with new updates

Please provide article feedback:

Symptoms

NSX Manager shows it is using 100% of CPU.

A large number of firewall rules have been recently added to the environment.

A large number of hosts are in the environment and configured to receive firewall rules.

NSX Manager is unresponsive to Web Interface or API commands.

ESXi to NSX Manager communication channel appears down for several hosts.

In the vsfwd.log file on a host showing communication channel down, you see entries similar to:

Re-read credentials to broker <IP Address>:5671: Logging in: Input/output error
2018-04-18T16:00:04UTC rmqClient Closing, No Ack received for Client netClient index 7

In the vsm.log file on the affected hosts, you see entries similar to:

2018-04-18 12:34:50.894 MDT ERROR HeartbeatManagerHeartbeatTimer HeartbeatManager$HeartbeatTask:297 - Client has not responded to the heartbeat for longer than the alert threshold. Peer name = 'com.vmware.vshield.userworld', client token = 'host-71', client id = '<UUID>', last heartbeat response = '4', last published heartbeat = '74'

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Purpose

This article provides information on lowering CPU usage and ensuring firewall rules are getting published to all applicable hosts.

Cause

This issue occurs because the NSX Manager's communication channel to the ESXi hosts is down or unavailable. This leads to NSX Manager repeatedly trying to reconnect to the ESXi hosts and synchronize the firewall rules.

Resolution

This issue is resolved in:

VMware NSX for vSphere 6.2.8, available at VMware Downloads.

VMware NSX for vSphere 6.3.6, available at VMware Downloads.

Workaround

To work around this issue if you do not want to upgrade:

Stop the vsfwd services on all the hosts which should clear out pending queues by running this command:

/etc/init.d/vShield-Stateful-Firewall stop

Restart the NSX Manager and wait for a few minutes for the services to be in a ready state in the User Interface (UI) before proceeding to the next step.

Note: There are no hosts syncs during this time as vsfwd is down.

Start the vsfwd service on a few hosts (5-8 hosts) at a time by running this command:

/etc/init.d/vShield-Stateful-Firewall start

Note: This spikes the NSX Manager CPU for a few mins (~10).

Once spike is done, restart the next batch of vsfwds.

Detectable by VMware SkylineTM

Actions

Copy

link to clipboard
copied!

Print

Language:
Chinese (Simplified)JapaneseEnglish

Attachments

Additional Resources

KB
•
End Of General Availability of the Free vSphere Hypervisor (ESXi 7.x and 8.x) (2107518)

KB
•
有关 Horizon 7 扩展服务分支（ESB）的信息 (52845)

Results 1-2 of 2

Ask The Community

Get answers quickly from VMware experts in the community

Post Subject

CONTINUE IN COMMUNITIES

Related Products:

VMware NSX for vSphere

Related Versions:

VMware NSX for vSphere 6.3.x

VMware NSX for vSphere 6.2.x

Take Our Survey

Support Assistant