Loading

×Sorry to interrupt
This page has an error. You might just need to refresh it.
[NoErrorObjectAvailable] Uncaught ReferenceError: addCTracking is not defined
Failing descriptor: {markup://c:KM_ArticleThemeLayout}

Refresh

Cookie Preference Center

General Information on cookies

When you visit our website, we use cookies to ensure that we give you the best experience. This information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies by clicking on the different category headings to find out more and change your settings. However, blocking some types of cookies may impact your experience on the site and the services we are able to offer. Further information can be found in our

Cookie Policy.

Strictly Necessary

Strictly Necessary

Always Active

Strictly Necessary

Strictly necessary cookies are always enabled since they are essential for our website to function. They enable core functionality such as security, network management, and website accessibility. You can set your browser to block or alert you about these cookies, but this may affect how the website functions. For more information please visit www.aboutcookies.org or www.allaboutcookies.org.

Cookie Details‎

Performance

Performance

Performance

Performance cookies are used to analyze the user experience to improve our website by collecting and reporting information on how you use it. They allow us to know which pages are the most and least popular, see how visitors move around the site, optimize our website and make it easier to navigate.

Cookie Details‎

Please confirm your settings by refreshing the page.

Functional

Functional

Functional

Functional cookies help us keep track of your past browsing choices so we can improve usability and customize your experience. These cookies enable the website to remember your preferred settings, language preferences, location and other customizable elements such as font or text size. If you do not allow these cookies, then some or all of these services may not function properly.

Cookie Details‎

Advertising

Advertising

Advertising

Advertising cookies are used to send you relevant advertising and promotional information. They may be set through our site by third parties to build a profile of your interests and show you relevant advertisements on other sites. These cookies do not directly store personal information, but their function is based on uniquely identifying your browser and internet device.

Cookie Details‎

Advertising Cookies

Consent Leg.Interest

Select All Vendors

Select All Vendors

Select All Hosts

Select All

Clear Filters

Information storage and access

Apply

Confirm My Choices Reject All Allow All

My Subscriptions

Loading

Knowledge Base

How to create a Stretched cluster on a NSX-T Workload Domain in VMware Cloud Foundation 3.8 (71136)

Last Updated: 7/28/2021Categories: InformationalTotal Views: 11498Language:
English

1 like this article

Subscribe to get an email
newsletter with new updates

1 people found this helpful Please provide article feedback:

Symptoms

The article below describes how to deploy NSX-T based Workload domain with stretch cluster where the management, vMotion, vSAN, and vTEP networks are Layer 2 stretched between both availability zones. While this deployment model is supported it is not the recommended and preferred deployment model.

The workflow should only use this Layer 2 stretch deployment model after extensively investigating and ruling out the standard deployment model where the management, vMotion, VSAN, and vTEP networks separate Layer 2 networks per site

Purpose

Deploying a VSAN stretched Cluster on a NSX-T Workload Domain is not a functionality currently available in VMware Cloud Foundation.
Below are the prerequisites required to successfully complete this workflow.

Enough VSAN and ESXi licenses : VSAN and ESXi licenses ( CPUs) must be adequate

Naming conventions:

Host group : {Cluster-Name}_primary-az-hostgroup , {Cluster-Name}_secondary-az-hostgroup

VM group: {Cluster-Name}_primary-az-vmgroup

Preferred site: {Cluster-Name}_primary-az-faultdomain

Secondary Site: {Cluster-Name}_secondary-az-faultdomain

Number of ESXi hosts on preferred site/primary-az-hostgroup should be equal to Number of hosts on secondary site/secondary-az-hostgroup.

Impact / Risks

Note: This workflow is not intended for VMware Cloud Foundation on VxRail deployments. To stretch VCF on VxRail deployments please refer to the KB How to create a Stretched cluster on a NSX-T Workload Domain on VMware Cloud Foundation on VxRail 3.8 environment

Resolution

Deploy and configure Witness host:

Once bringup is completed, access the vCenter Server associated to Management Domain and create following artifacts

Create a new cluster.

Create a DVS switch with a portgroup "MGMT-PG" and tag the portgroup with the Default Management VLAN.

Create a portgroup "VSAN-PG" on the VDS and tag it with the Witness VLAN. This portgroup will be used for VSAN traffic for the witness host.

Add an ESXi Host, which is not part of any cluster to the new cluster.

Associate host to to the DVS

Deploy the VSAN WITNESS OVA on the new cluster. When in the deployment wizard on the "Select Networks" section, associate "VSAN-PG" to Witness Network and "MGMT-PG" for Management Network and provide appropriate root passwords.

Once the Witness VM is deployed, access web console of the Witness VM and configure Management network and DNS appropriately and enable SSH.

Add the Witness VM as a host to the "Workload vCenter" as a standard host at the Datacenter level, it should not be part of any cluster.

Once Witness is added as host, access the Host in the vCenter Web Client and navigate to Configure ->Networking -> VMKernel Adapter. Configure the IP address details for vmk1 associated to Witness Switch.

Provide IP routing rules for the VSAN networks for the hosts in AZ1 and AZ2.

Create NSX-T enabled domain/cluster

Using VCF SDDC Manager UI, create a NSX-T Workload Domain or create a new cluster in an existing NSX-T Workload Domain.

Note: Ensure that the cluster has sufficient ESXi hosts such that VSAN Stretch cluster operation can be performed.

Expand NSX-T enabled cluster

NSX-T enabled cluster is expanded using the "Add Host" functionality in SDDC Manager. Use the steps below to perform expansion of NSX-T cluster with AZ2 ESXi hosts.

Commission the AZ2 ESXi hosts using the same network pool.

In the SDDC Manager UI, navigate to the NSX-T Workload Domain and select the cluster to expand. Click Actions and select Add Host and choose AZ2 hosts and complete finish cluster expansion.

Create fault domains and other stretch cluster related configurations

Complete the below steps to enable a stretched VSAN Cluster.

Access vCenter Server Web Client associated with the desired cluster.

Navigate to the vSAN cluster.

Click the Configure tab.

Under vSAN, click Fault Domains.

Click the Stretched Cluster Configure button to open the stretched cluster configuration wizard.

Select the hosts or the fault domain that you want to assign to the secondary domain and click >>.The hosts that are listed under the Preferred fault domain are in the preferred site.

Click Next.

Select a witness host that is not a member of the vSAN stretched cluster and click Next.

Claim the storage devices on the witness host and click Next.

Select one flash device for the cache tier, and one or more devices for the capacity tier.

On the Ready to complete page, review the configuration and click Finish.

Navigate to Configure -> Services and complete the below configurations for the cluster.

In vSphere Availability, Set Admission Control, the Host failures cluster tolerates  needs to be set to 50% of the cluster. If the stretched cluster is a total of  8 hosts this value will be 4.

In vSphere Availability -> Advanced Options, set VSAN gateway as isolated address as following:  das.isolationaddress1=<VSAN-GATEWAY-IP-for-preferred-site-hosts>

Navigate to VM/Host Groups and click Add to create two host groups, one for the preferred site and one for the secondary site.

In VM/Host Groups create two VM groups, one to hold the VMs on the preferred site and one to hold the VMs on the secondary site.

In VM/Host Rules the HA settings should respect VM-Host affinity rules during failover.

Navigate to Policies and Profiles and edit the vSAN profile associated with the cluster. In the vSAN panel, under the Availability tab, set the following:

Site disaster tolerance to Dual Site Mirroring (stretched cluster)

Failure to tolerate is set to 1 failure - Raid-1  (Mirroring).

Download setup patch vcf-stretch-cluster-patch.zip attached to this article.

Using a file transfer utility, copy the file to the /tmp/ directory on the SDDC Manager VM.

SSH to the SDDC Manager using the vcf user and change to root:

su -

Unzip the file:

unzip /tmp/vcf-stretch-cluster-patch

Change directory to the extracted folder:

cd /tmp/vcf-stretch-cluster-patch

Execute the setup.sh script to update the SOS package.

After completing the stretch cluster operation, execute the below command to enable the stretch cluster flag for in the SDDC Manager inventory.The below command will enable the stretch flag and will block the Add Host functionality.

/opt/vmware/sddc-support/sos ---enable-stretch-cluster-flag  {cluster-name} --domain-name {Workload-Domain-name}

Day2 operations on a Stretch Cluster

Adding host to the stretched cluster.

Removal of an ESXi host.

Upgrading the stretched cluster through LCM.

ESXi Hosts can be added through the Add Host workflow in the SDDC Manager UI.

The stretched cluster flag, firstly, needs to be disabled to allow the Add Host functionality to be enabled in the SDDC Manager UI. Please run the below command on the SDDC Manager using root:

/opt/vmware/sddc-support/sos --disable-stretch-cluster-flag {cluster-name} --domain-name {Workload-Domain-name}

Once a host is added to a stretched cluster, follow the below instructions to make the host part of the fault domains.

Once host is added to the stretched cluster, in the vCenter Web Client the cluster will have a critical alarm for  "VSAN Health Alarm "Unexpected number of fault domains". This is generated as the ESXi host that was added is not part of any of the fault domains.

Select Cluster -> Configuration -> Fault Domains.

Select the ESXi host, which is not part of any of the fault domain, click on the "Move to fault domain" icon and select the fault domain the ESXi host should be part of.

Re-enabled the stretched cluster flag:

/opt/vmware/sddc-support/sos ---enable-stretch-cluster-flag  {cluster-name} --domain-name {Workload-Domain-name}

Upgrading Stretch Cluster

ESXi hosts that are part of the SDDC Manager inventory, except VSAN Witness Host, will be upgraded through SDDC Manager's LCM functionality.

NSX-T upgrade will also be handled by through SDDC Manager's LCM functionality.

Upgrade the vSAN Witness Host

The Witness host upgrade is performed outside of the SDDC Manager as the witness host is not part of the inventory. The upgrade of witness host need to be performed using the vCenter Update Manager.

Below are the steps to be performed for upgrade of the witness.

Login to vCenter Web Client, where witness host is added.

Navigate to Home -> Update Manager

Select the ESXi Images tab and click IMPORT to upload the ESXi image.

Select the Baselines tab and click NEW -> Baseline. Provide a name for the baseline and ensure Content is Upgrade. In Select Image, use the uploaded ESXi Image from Step 3. Click OK.

Navigate to Hosts and Clusters, select the Witness host and select the Updates tab.

Attach the baseline to the host and click on Remediate. After this task is successful, the host will have been upgraded to the desired version.

Remove of an ESXi host from the Stretched Cluster.

Should a host be required to be removed from the cluster due to any reason, the Remove Host functionality in the SDDC Manager UI should be used to properly hand its removal from vCenter and the cluster.

Detectable by VMware SkylineTM

Actions

Copy

link to clipboard
copied!

Print

Language:
English

Attachments

vcf-stretch-cluster-patch

Additional Resources

KB
•
End Of General Availability of the Free vSphere Hypervisor (ESXi 7.x and 8.x) (2107518)

KB
•
升级到 vRealize Network Insight 3.9 和 4.0 所需的升级前修补程序 (71014)

KB
•
Network Insight（SaaS のみ）コレクタ仮想マシンが証明書の例外によりサービスへの接続に失敗する (70834)

Results 1-3 of 3

Ask The Community

Get answers quickly from VMware experts in the community

Post Subject

CONTINUE IN COMMUNITIES

Related Products:

VMware Cloud Foundation

Related Versions:

VMware Cloud Foundation 3.8.x

Take Our Survey

Support Assistant