Loading

×Sorry to interrupt
This page has an error. You might just need to refresh it.
[NoErrorObjectAvailable] Uncaught ReferenceError: addCTracking is not defined
Failing descriptor: {markup://c:KM_ArticleThemeLayout}

Refresh

Cookie Preference Center

General Information on cookies

When you visit our website, we use cookies to ensure that we give you the best experience. This information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies by clicking on the different category headings to find out more and change your settings. However, blocking some types of cookies may impact your experience on the site and the services we are able to offer. Further information can be found in our

Cookie Policy.

Strictly Necessary

Strictly Necessary

Always Active

Strictly Necessary

Strictly necessary cookies are always enabled since they are essential for our website to function. They enable core functionality such as security, network management, and website accessibility. You can set your browser to block or alert you about these cookies, but this may affect how the website functions. For more information please visit www.aboutcookies.org or www.allaboutcookies.org.

Cookie Details‎

Performance

Performance

Performance

Performance cookies are used to analyze the user experience to improve our website by collecting and reporting information on how you use it. They allow us to know which pages are the most and least popular, see how visitors move around the site, optimize our website and make it easier to navigate.

Cookie Details‎

Please confirm your settings by refreshing the page.

Functional

Functional

Functional

Functional cookies help us keep track of your past browsing choices so we can improve usability and customize your experience. These cookies enable the website to remember your preferred settings, language preferences, location and other customizable elements such as font or text size. If you do not allow these cookies, then some or all of these services may not function properly.

Cookie Details‎

Advertising

Advertising

Advertising

Advertising cookies are used to send you relevant advertising and promotional information. They may be set through our site by third parties to build a profile of your interests and show you relevant advertisements on other sites. These cookies do not directly store personal information, but their function is based on uniquely identifying your browser and internet device.

Cookie Details‎

Advertising Cookies

Consent Leg.Interest

Select All Vendors

Select All Vendors

Select All Hosts

Select All

Clear Filters

Information storage and access

Apply

Confirm My Choices Reject All Allow All

My Subscriptions

Loading

Knowledge Base

East-West traffic between workloads behind different T1 is impacted, when NAT is configured on T0 (71363)

Last Updated: 9/25/2019Categories: TroubleshootingTotal Views: 16311Language:
Chinese (Simplified)Portuguese (Brazil)JapaneseSpanishEnglishGerman

1 like this article

Subscribe to get an email
newsletter with new updates

1 people found this helpful Please provide article feedback:

Symptoms

In a Multi-tenant (T1s) environment with the following conditions:

Workloads behind T1s have SNAT rules configured on the T0 for S-N communication

A T1 has a Service Router (SR) component configured

Tenant T1s connected to the same T0 router, and the SR component of a Tenant T1 is active on the same Edge node where the T0 SR is active.

In the T0 NAT configuration, any DNAT rule is configured

Below is an example from a PKS topology. Same issue will be seen in a Non-PKS topology as well.

Topology Information:

- Bosh IP: 172.16.80.2

- Bosh IP 172.16.80.2 is SNAT’ed at the T0 to 192.168.80.2

- K8s Master VM IP: 172.16.90.2

- K8s Master VM IP 172.16.90.2 is SNAT’ed at the T0 to 192.168.90.2

- There’s a DNAT rule configured at the T0 to translate 192.168.80.2 to 172.16.80.2 for North-South communication

Note:

This DNAT rule is configured for illustration purpose in this use case. This issue would occur even if a DNAT rule is configured for a completely different workflow

East-West traffic between workloads behind different T1 is impacted, when communication happens over their Private IPs (i.e. NON-NATed IPs). In the above example when 172.16.80.2 tries to communicate 172.16.90.2, the communication is impacted.

Note:

East-West traffic between workloads behind different T1s works as expected when workloads communicate over their Public IPs (NAT’ed IPs). In the above example when 172.16.80.2 tries to communicate 192.168.90.2, the communication will work as expected.

North-South traffic outside NSX-T works as expected. In the above example, when 172.16.80.2 sends a packet to an external workloads, north of T0 Logical Router, we will see the SNAT occur and communication will work as expected.

Cause

The SNAT rule starts taking effect in 2.4.2 between the T0 and T1 causing the traffic to be SNATed twice, once while traffic is egress to the destination and once again when traffic returning back from the destination. This leads to the workload dropping the traffic. The following is the packet walk, for the above example:

Packet Walk:

Request:

Packet leaves Opsmgr with source IP 172.16.80.2 and Destination 172.16.90.2

Traffic arrives at the T1 DR interface on the Source host and is routed to the T1 SR which resides on Edge Node edge01. Source is 172.16.80.2 and Destination is 172.16.90.2

From the edge node, the traffic is routed from the T1 SR to the T0 DR on edge01. Source IP is 172.16.80.2 and Destination is 172.16.90.2

Starting 2.4.2, Firewall is automatically enabled on LinkedPorts. Also, SNAT is applied as traffic egresses the T0 DR interface to the Destination T1. This causes the source IP to be SNATed as it leaves the T0 DR interface towards the Destination T1. At this point, the Source IP is SNATed to 192.168.80.2 and Destination is 172.16.90.2

Packet is sent to the destination host via Geneve Encapsulated packet

Response:

Destination sends a packet with Source IP 172.16.90.2 and Destination 192.168.80.2

Traffic is routed via the T1 DR interface on the source host to T0 DR on the source host. Since there’s no T1 SR on this T1 router, the traffic will directly be forwarded from T1 DR to T0 DR on this host. Source IP of this packet is 172.16.90.2 and Destination 192.168.80.2

Since Firewall is enabled by default on Linked Ports, the traffic will be forwarded from the T0 DR on this host to the T0 SR on edge node edge01. At this point, the source IP of the packet is 172.16.90.2 to 192.168.80.2

From the T0, the packet will be forwarded to the Source T1 DR interface. Here SNAT and Reverse NAT will occur, and the source IP will now be changed to 192.168.90.2 and Destination to 172.16.80.2

Packet will be sent to the Destination host via a Geneve Encapsulated packet to destination 172.16.80.2 with Source IP of 192.168.90.2

The destination VM will drop the packet since  the response is received from Different IP  ”192.168.90.2” instead  of “172.16.90.2”

Resolution

This issue is resolved in VMware NSX-T Data Center 2.5, available at VMware Downloads.

Workaround

Workaround:

If there are no services (like NAT, Firewall, etc.) on the T1 SR, it is safe to detach it from the edge cluster. This is a two-step process as illustrated below

Choose T1 Router → Services →  Edge Firewall → Click on Disable Firewall

Click on Overview → Click Edit (Next to Summary) → Click the ‘x’ next to Edge Cluster to detach the edge cluster from T1

If you are not able to perform this workaround or have any additional questions, file a support request with VMware Support and quote this Knowledge Base article ID (71363)  in the problem description. For more information, see How to Submit a Support Request.

Detectable by VMware SkylineTM

Actions

Copy

link to clipboard
copied!

Print

Language:
Chinese (Simplified)Portuguese (Brazil)JapaneseSpanishEnglishGerman

Attachments

Additional Resources

KB
•
vSAN Health Service - 容量利用率 - 读取缓存预留 (71173)

KB
•
core.jsvc.xxx ファイルが生成されて SPS サービスがクラッシュし、vCSA 6.7 U2 の /storage/core がいっぱいになる (71170)

KB
•
在 ESXi 6.7 上运行 Storage vMotion 期间，与卷的连接中断 (74514)

KB
•
Se perdió la conectividad con el volumen durante la ejecución de Storage VMotion en ESXi 6,7 (74514)

Results 1-4 of 4

Ask The Community

Get answers quickly from VMware experts in the community

Post Subject

CONTINUE IN COMMUNITIES

Related Products:

Related Versions:

VMware NSX-T Data Center

Take Our Survey

Support Assistant