Loading

×Sorry to interrupt
Uncaught ReferenceError: addCTracking is not defined
throws at https://kb.vmware.com/resource/KM_TealiumTrackingJS?v=0.5:100:1

Refresh

Cookie Preference Center

General Information on cookies

When you visit our website, we use cookies to ensure that we give you the best experience. This information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies by clicking on the different category headings to find out more and change your settings. However, blocking some types of cookies may impact your experience on the site and the services we are able to offer. Further information can be found in our

Cookie Policy.

Strictly Necessary

Strictly Necessary

Always Active

Strictly Necessary

Strictly necessary cookies are always enabled since they are essential for our website to function. They enable core functionality such as security, network management, and website accessibility. You can set your browser to block or alert you about these cookies, but this may affect how the website functions. For more information please visit www.aboutcookies.org or www.allaboutcookies.org.

Cookie Details‎

Performance

Performance

Performance

Performance cookies are used to analyze the user experience to improve our website by collecting and reporting information on how you use it. They allow us to know which pages are the most and least popular, see how visitors move around the site, optimize our website and make it easier to navigate.

Cookie Details‎

Please confirm your settings by refreshing the page.

Functional

Functional

Functional

Functional cookies help us keep track of your past browsing choices so we can improve usability and customize your experience. These cookies enable the website to remember your preferred settings, language preferences, location and other customizable elements such as font or text size. If you do not allow these cookies, then some or all of these services may not function properly.

Cookie Details‎

Advertising

Advertising

Advertising

Advertising cookies are used to send you relevant advertising and promotional information. They may be set through our site by third parties to build a profile of your interests and show you relevant advertisements on other sites. These cookies do not directly store personal information, but their function is based on uniquely identifying your browser and internet device.

Cookie Details‎

Advertising Cookies

Consent Leg.Interest

Select All Vendors

Select All Vendors

Select All Hosts

Select All

Clear Filters

Information storage and access

Apply

Confirm My Choices Reject All Allow All

My Subscriptions

Loading

Knowledge Base

NSX Edge fails to connect to the broker after upgrade (52051)

Last Updated: 2/3/2018Categories: TroubleshootingTotal Views: 4489Language:
English

Subscribe to get an email
newsletter with new updates

Please provide article feedback:

Symptoms

NSX Edge fails to connect to broker after upgrade.

In the /var/log/rabbitmq/[email-masked] file, when the NSX Edge is trying to connect to broker, you see entries similar to:

=ERROR REPORT==== 16-Nov-2017::02:42:11 ===
Error on AMQP connection <0.24620.25> (50.6.0.39:16053 -> 10.114.12.246:5671, user: 'vse_50394af8-387a-809f-46c2-97026b5196fe', state: opening):
access to vhost 'vshield' refused for user 'vse_50394af8-387a-809f-46c2-97026b5196fe'

In the VSFWD logs on the ESXi host, you see entries similar to:

2017-11-16T13:18:02Z vsfwd: [INFO] Connected to 23DB6AF8:36985.
2017-11-16T13:18:02Z vsfwd: [INFO] Edge VMCI: 36 <--> Broker NET: 38:Port 29477
2017-11-16T13:18:02Z vsfwd: [ERROR] Queuing as Unable to Write 197 Bytes for Edge Client Type VMCI fd 38 error ret -1 errno 107
2017-11-16T13:18:02Z vsfwd: [INFO] Edge (VMCI:NET) Socket 36 hung up

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Cause

This issue occurs while upgrading the NSX Manager (and hence the RMQ broker), permission for the NSX Edge user created on the broker is getting deleted.

Resolution

To resolve this issue, after the NSX Manager upgrade, if some of the Edges are not able to connect to the broker, do the following:

If hosts are not able to connect, resync them through REST API:

POST https:NSXMGR_IP/api/2.0/nwfabric/configure?action=synchronize

<nwFabricFeatureConfig>
<featureId>com.vmware.vshield.vsm.messagingInfra</featureId>
<resourceConfig>
<resourceId>host-15</resourceId>
</resourceConfig>
</nwFabricFeatureConfig>

If Edges are not able to connect to the broker, confirm if users are available on the broker:

a. Run this command on the NSX Manager to see how many Edge users are there on the broker:

rabbitmqctl list_users |grep -i vse_

vse_50394d8b-72cb-baac-7ffb-2a08f1bfe068
vse_5039f49a-dfb3-9320-22aa-21001545ae30

b. Compare this list from all the deployed Edge and see if it has users for all the Edges. If a user is missing from the broker then redeploy the Edge.

Assuming all the users are present in step 2, check if permissions are set on the broker for each Edge:

a.  Run this command on the NSX Manager to see how many Edges have permission set on the broker:

rabbitmqctl list_permissions -p vshield|grep -i vse_

vse_503986f7-b6a5-4a1c-7d7a-55c3113e9593 ^vse.*|^amq\.gen.*|^amq\.default|^vsm.* ^vse.*|^amq\.gen.*|^amq\.default ^vse.*|^amq\.gen.*|^amq\.default|^vsm.*
vse_5039c932-f1bb-5d0b-fd53-4e12ea419d58 ^vse.*|^amq\.gen.*|^amq\.default|^vsm.* ^vse.*|^amq\.gen.*|^amq\.default ^vse.*|^amq\.gen.*|^amq\.default|^vsm.*

b. Compare this list and see if permission for any of the Edge is missing. Run this command to create permission for the Edge user whose permission is missing on the broker:

curl -H "Content-Type: application/json" -k -u vsm:KwXoVpTX64 -X PUT -d '{"configure":"^vse.*|^amq\.gen.*|^amq\.default|^vsm.*", "write":"^vse.*|^amq\.gen.*|^amq\.default", "read":"^vse.*|^amq\.gen.*|^amq\.default|^vsm.*"}' https://localhost:15671/api/permissions/vshield/vse_502cdddf-cfcf-4b20-b20e-541b8e8bfec5

Where:

vsm:KwXoVpTX64 - username:password for admin account on broker. you can get this by running this command:

/home/secureall/secureall/sem/WEB-INF/classes/GetVsmRabbitPassword.sh

Detectable by VMware SkylineTM

Actions

Copy

link to clipboard
copied!

Print

Language:
English

Attachments

Additional Resources

KB
•
End Of General Availability of the Free vSphere Hypervisor (ESXi 7.x and 8.x) (2107518)

Result 1 of 1

Ask The Community

Get answers quickly from VMware experts in the community

Post Subject

CONTINUE IN COMMUNITIES

Related Products:

VMware NSX for vSphere

Related Versions:

VMware NSX for vSphere 6.3.x

Take Our Survey

Support Assistant