Loading

×Sorry to interrupt
Uncaught ReferenceError: addCTracking is not defined
throws at https://kb.vmware.com/resource/KM_TealiumTrackingJS?v=0.5:100:1

Refresh

Cookie Preference Center

General Information on cookies

When you visit our website, we use cookies to ensure that we give you the best experience. This information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies by clicking on the different category headings to find out more and change your settings. However, blocking some types of cookies may impact your experience on the site and the services we are able to offer. Further information can be found in our

Cookie Policy.

Strictly Necessary

Strictly Necessary

Always Active

Strictly Necessary

Strictly necessary cookies are always enabled since they are essential for our website to function. They enable core functionality such as security, network management, and website accessibility. You can set your browser to block or alert you about these cookies, but this may affect how the website functions. For more information please visit www.aboutcookies.org or www.allaboutcookies.org.

Cookie Details‎

Performance

Performance

Performance

Performance cookies are used to analyze the user experience to improve our website by collecting and reporting information on how you use it. They allow us to know which pages are the most and least popular, see how visitors move around the site, optimize our website and make it easier to navigate.

Cookie Details‎

Please confirm your settings by refreshing the page.

Functional

Functional

Functional

Functional cookies help us keep track of your past browsing choices so we can improve usability and customize your experience. These cookies enable the website to remember your preferred settings, language preferences, location and other customizable elements such as font or text size. If you do not allow these cookies, then some or all of these services may not function properly.

Cookie Details‎

Advertising

Advertising

Advertising

Advertising cookies are used to send you relevant advertising and promotional information. They may be set through our site by third parties to build a profile of your interests and show you relevant advertisements on other sites. These cookies do not directly store personal information, but their function is based on uniquely identifying your browser and internet device.

Cookie Details‎

Advertising Cookies

Consent Leg.Interest

Select All Vendors

Select All Vendors

Select All Hosts

Select All

Clear Filters

Information storage and access

Apply

Confirm My Choices Reject All Allow All

My Subscriptions

Loading

Knowledge Base

How to replace network switches in VMware Cloud Foundation 2.2 and greater (51853)

Last Updated: 10/16/2020Categories: How toTotal Views: 4328Language:
English

Subscribe to get an email
newsletter with new updates

Please provide article feedback:

Purpose

The instructions for replacing VMware Cloud Foundation 2.2 network switches included in Administering VMware Cloud Foundation - VMware Cloud Foundation 2.2 are incorrect. This article will provide the correct instructions.

Resolution

Replacing a management switch:

Login to the SDDC Manager virtual machine as the root user.

Ensure that there is a backup of the management switch configuration under the /var/tmp folder.

There should be multiple folders present, with names similar to backup-<date>, backup-bringup-<date> and scheduled-backup-<date>.

In each folder there should exist a folder named switch.

In the switch folder, there should be a .tgz file for the management switch.

Identify the most recent .tgz file for the management switch (192.168.3.254). If one does not exist and the management switch to be replaced is still accessible, run /opt/vmware/sddc-support/sos --backup to generate one.

Use a file transfer utility to copy the .tgz file off of the SDDC Manager Controller virtual machine.

Use the /home/vrack/bin/lookup-passwords command to find the current password for the cumulus user and the switch ID on the management switch. Output similar to the following will be returned:

MANAGEMENT: quanta lb9
identifiers: 192.168.3.254, R1S0
workload: hardware
username: cumulus
password: EvoSddc!2016
type: SSH

Note: In this example, the password is EvoSddc!2016 and the switch ID is R1S0. Make a note of both of these values.

Unplug the current management switch . Make a note of all the current connections to the TOR/Spine switches and hosts in the rack.

Note: Several critical alerts will likely be generated in SDDC Manager at this point.

Plugin the new Management switch and ensure that it is in Onie mode by issuing the cl-img-select -i and reboot commands.

Image the new management switch with the VMware Imaging Appliance (VIA). Follow the steps in the VIA User's Guide under, Image New Management Switch.

Once imaging of the management switch is complete, use a file transfer utility to copy the configuration backup noted in Step 2 to the /home/cumulus folder on the management switch.

Log in to the management switch as the cumulus user with the password "EvoSddc!2016".

Switch to the root user account by issuing the sudo su - command.

Issue commands similar to the following to restore the configuration backup:

cd /
tar zxvf /home/cumulus/cumulus-R1S0-192.168.3.254.tgz
service networking restart

Change the cumulus user's password from EvoSddc!2016 to the password noted in Step 3 with the passwd cumulus command.

Note: Complete Step 12 only if the Management, Datacenter, VSAN, VMotion, or VXLAN networks are using VLANs in the 3000-3999 range, or if the VMware Cloud Foundation version is 2.3 or higher. Otherwise, skip to step 13.

On the management switch, as the root user, issue the following commands to update the reserved VLAN range:

sed -i.orig -e 's/#resv_vlan_range/resv_vlan_range/' -e 's/3300-3999/<START>-<END>/' /etc/cumulus/switchd.conf

Note: Replace the <START> with the beginning ID of the VLAN range to be reserved and <END> with the end ID of the VLAN range to be reserved

service switchd restart
service networking restart

Log in to the SDDC Manager Controller virtual machine as the root user. Switch to the vrack user by issuing the su - vrack command.

Note: If the SDDC Manager Controller virtual machine is not accessible on the network, wait a few minutes and try again as the previous step may require additional time to complete background tasks.

​Issue the following command to populate the /home/vrack/.ssh/known_hosts file with the new ssh key from the management switch:

ssh -oHostKeyAlgorithms='ssh-rsa' [email-masked]

Note: Enter the cumulus user password when prompted.
Note: Type exit to log out of the management switch and return to the SDDC Manager Controller virtual machine.

Rotate the management switch SSH key using the commands noted below (run as the vrack user):

curl -H "Content-Type: application/json" -X PUT --data "{\"keyGenAlgorithm\":\"RSA\", \"keyLength\":2048}" http://localhost:8080/hms-local/api/1.0/hms/napi/switches/<switch-id>/sshkeys/rotate

Note: Replace <switch-id> with the name value noted in the Step 3 (R1S0 for this example).
Note: The new ssh key value will be the output of the above command. Make a note of this new value.

Open the .ssh/known_hosts file with a text editor.

Locate the entry that begins with 192.168.3.254.

Replace the ssh key value after ssh-rsa with the value returned in Step 15.

Save and close the file.

Verify the management switch configuration information via the following command:

curl -X GET http://localhost:8080/hms-local/api/1.0/hms/switches/<switch-id>

Note: Replace <switch-id> with the name value noted in the Step 3 (R1S0 for this example).
Note: Output similar to the following will be returned:

{"fruId":"1862331373","componentIdentifier":{"description":null,"manufacturer":"quanta","product":"lb9","partNumber":"1LB9BZZ0STR","manufacturingDate":"2014/12/23","serialNumber":"QTFCA65100034"},"location":null,"rackId":"93dfd957-b561-489c-ae63-103cf79c19ed","switchId":"R1S0","switchPorts":["eth0","swp1","swp2","swp3","swp4","swp5","swp6","swp7","swp8","swp9","swp10","swp11","swp12","swp13","swp14","swp15","swp16","swp17","swp18","swp19","swp20","swp21","swp22","swp23","swp24","swp25","swp26","swp27","swp28","swp29","swp30","swp31","swp32","swp33","swp34","swp35","swp36","swp37","swp38","swp39","swp40","swp41","swp42","swp43","swp44","swp45","swp46","swp47","swp48","swp49","swp50","swp51","swp52"],"ipAddress":"192.168.3.254","mangementPort":null,"managementMacAddress":"2c:60:0c:45:89:b6","operational_status":"true","osName":"Cumulus Linux","osVersion":"2.5.8","firmwareName":"ONIE","firmwareVersion":"2014.05.01-b23b0ab","adminStatus":null,"role":"MANAGEMENT","validationStatus":null,"powered":true,"discoverable":true}

Log in to the SDDC Manager UI.

Navigate to the Status page and click the View Details link in the Alerts pane.

Click the Edit link at the top of the page.

Select any alerts related to the management switch.

Click the Clear Selected Alerts link at the top of the page.

Navigate to the Dashboard page and then click the View Details link next to Physical Resources.

Click the Rack and then click on the management switch. Verify that the switch is healthy and the configuration is accurate.

Replacing a Cisco Top of Rack (TOR) Switch:

Login to the SDDC Manager virtual machine as the root user.

Ensure that there is a backup of the TOR switch configuration under the /var/tmp folder.

There should be multiple folders present, with names similar to backup-<date>, backup-bringup-<date> and scheduled-backup-<date>.

In each folder there should exist a folder named switch.

In the switch folder, there should be a .gz file for each TOR and Rack-Interconnect switch

Identify the most recent .gz file for the Cisco TOR switch (<switch-id>-<IP address>-cisco-running-config-<date>.gz). If one does not exist and the TOR switch to be replaced is still accessible, run /opt/vmware/sddc-support/sos --backup to generate one.

Use the /home/vrack/bin/lookup-passwords command to find the current password, IP Address and the switch ID on the TOR switch. Output similar to the following will be returned:

TOR: Cisco Systems, Inc. N9K-C9372PX-E
identifiers: 192.168.0.112, R1S1
workload: hardware
username: admin
password: EvoSddc!2016
type: SSH

Note: In this example, the password is EvoSddc!2016, the IP address is 192.168.0.112 and the switch ID is R1S1. Make a note of all of these values.

Unplug the TOR switch . Make a note of all the current connections to the TOR/Rack-Interconnect/management switches and hosts in the rack.

Note: Several critical alerts will likely be generated in SDDC Manager at this point.

Plugin the new TOR switch. Clear any existing configuration by issue the write erase and reload commands.

Exit out of POAP (Power of Auto Provisioning mode). Follow the instruction on the switch console screen:

Abort Auto Provisioning and continue with normal setup ?(yes/no)[n]:yes

When prompted, set a password for the admin user. Please assign an easy to remember password and make a note of the password.

Configure the IP address noted in Step 3 on the mgmt 0 interface, using a /21 subnet mask. Configure VRF to the mgmt 0 interface.

configure Terminal
interface mgmt 0
ip address 192.168.0.112/21
vrf member management
no shut
end

Note: Replace 192.168.0.112 with the appropriate IP address.

Verify network connectivity on the new TOR switch with the ping <other TOR switch IP address> vrf management command.

Copy the configuration backup noted in Step 2 to the TOR switch:

copy scp: bootflash: vrf management
Enter source filename: /var/tmp/<backup folder>/<name of TOR switch .gz file noted in Step 2>
Enter hostname for the scp server: 192.168.100.40
Enter username: root
[email-masked] password: <SDDC Manager Controller VM root password>

Note: Output similar to the following will be displayed when the copy starts:

<name of TOR switch .gz file noted in Step 2>    100% 1891     1.9KB/s   00:00
Copy complete, now saving to disk (please wait)

Decompress and Install the restored configuration in the TOR switch's running configuration:

gunzip bootflash:/// <name of TOR switch .gz file noted in Step 2>
copy <extracted file from previous step> startup-config
copy startup-config running-config

Regenerate the VMCA signed certificate for this IP Address and configure the switch with the certificate.

Download the attached fru_switch_2.3.zip file. Extract the contents.

Use a file transfer utility to copy the extracted fru_switch_2.3.py file to the /tmp/ folder on the SDDC Manager Controller virtual machine.

SSH to the SDDC Manager Controller virtual machine as the root user.

Execute the following command to run the fru_switch_2.3.py script:

python /tmp/fru_switch_2.3.py

Note: Enter the replacement TOR switch's IP address when prompted.

Run the following commands from the SDDC Manager Controller VM to verify that a 200 OK response is returned from the Hardware Monitoring Service (HMS):

curl -X GET http://localhost:8448/api/1.0/hms/switches/<switch-id>
curl -X GET http://localhost:8080/hms-local/api/1.0/hms/switches/<switch-id>

Note: Replace <switch-id> with the name value noted in the Step 3 (R1S1 for this example).

Replacing an Arista Top of Rack (TOR) Switch:

Login to the SDDC Manager virtual machine as the root user.

Ensure that there is a backup of the TOR switch configuration under the /var/tmp folder.

There should be multiple folders present, with names similar to backup-<date>, backup-bringup-<date> and scheduled-backup-<date>.

In each folder there should exist a folder named switch.

In the switch folder, there should be a .gz file for each TOR and Rack-Interconnect switch

Identify the most recent .gz file for the Arista TOR switch (<switch-id>-<IP address>-arista-running-config-<date>.gz). If one does not exist and the TOR switch to be replaced is still accessible, run /opt/vmware/sddc-support/sos --backup to generate one.

Use the /home/vrack/bin/lookup-passwords command to find the current password, IP Address and the switch ID on the TOR switch. Output similar to the following will be returned:

TOR: Arista Networks DCS-7280SE-72-F
identifiers: 192.168.0.108, R1S1
workload: hardware
username: admin
password: EvoSddc!2016
type: SSH

Note: In this example, the password is EvoSddc!2016, the IP address is 192.168.0.108 and the switch ID is R1S1. Make a note of all of these values.

Unplug the TOR switch . Make a note of all the current connections to the TOR/Rack-Interconnect/management switches and hosts in the rack.

Note: Several critical alerts will likely be generated in SDDC Manager at this point.

Plugin the new TOR switch. Clear any existing configuration by issue the write erase and reload commands.

Exit out of POAP (Power of Auto Provisioning mode) by issuing the zerotouch cancel command.

Configure the IP address noted in Step 3 on the management 1 interface, using a /22 subnet mask.

configure Terminal
interface management 1
ip address 192.168.0.108/22

Note: Replace 192.168.0.108 with the appropriate IP address.

Verify network connectivity on the new TOR switch with the ping <other TOR switch IP address> command.

Copy the configuration backup noted in Step 2 to the TOR switch:

bash
cd /mnt/flash
scp [email-masked]:/var/tmp/<backup folder>/<name of TOR switch .gz file noted in Step 2> .
exit

Note: Enter the management switch cumulus user's password when prompted.
Note: Output similar to the following will be displayed when the copy starts:

<name of TOR switch .gz file noted in Step 2>    100% 1835     1.8KB/s   00:00

Verify that the new file exists with the dir flash command.

Decompress and Install the restored configuration in the TOR switch's running configuration:

bash
cd /mnt/flash
gunzip <name of TOR switch .gz file noted in Step 2>
exit
copy <extracted file from previous step> startup-config
copy startup-config running-config

Note: The password will now be updated to what was noted in Step 3.

Regenerate the VMCA signed certificate for this IP Address and configure the switch with the certificate.

Download the attached fru_switch_2.3.zip file. Extract the contents.

Use a file transfer utility to copy the extracted fru_switch_2.3.py file to the /tmp folder on the SDDC Manager Controller virtual machine.

SSH to the SDDC Manager Controller virtual machine as the root user.

Execute the following command to run the fru_switch_2.3.py script:

python /tmp/fru_switch_2.3.py

Note: Enter the replacement TOR switch's IP address when prompted.

Run the following commands from the SDDC Manager Controller VM to verify that a 200 OK response is returned from the Hardware Monitoring Service (HMS):

curl -X GET http://localhost:8448/api/1.0/hms/switches/<switch-id>

Note: Replace <switch-id> with the name value noted in the Step 3 (R1S1 for this example).

Replacing a Cisco Rack-Interconnect Switch:

Login to the SDDC Manager virtual machine as the root user.

Ensure that there is a backup of the Rack-Interconnect switch configuration under the /var/tmp folder.

There should be multiple folders present, with names similar to backup-<date>, backup-bringup-<date> and scheduled-backup-<date>.

In each folder there should exist a folder named switch.

In the switch folder, there should be a .gz file for each TOR and Rack-Interconnect switch.

Identify the most recent .gz file for the Cisco Rack-Interconnect switch (<switch-id>-<IP address>-cisco-running-config-<date>.gz). If one does not exist and the Rack-Interconnect switch to be replaced is still accessible, run /opt/vmware/sddc-support/sos --backup to generate one.

Copy the Cicso Rack-Interconnect Switch backup file noted in Step 2 to the Management Switch:

scp <backup filename> [email-masked]:/tmp

Note: Enter the Management Switch's cumulus user password when prompted.

Use the /home/vrack/bin/lookup-passwords command to find the current password, IP Address and the switch ID on the Rack-Interconnect switch. Output similar to the following will be returned:

SPINE: Cisco Systems, Inc. N9K-C9332PQ
identifiers: 192.168.0.30, R1S3
workload: hardware
username: admin
password: EvoSddc!2016
type: SSH

Note: In this example, the password is EvoSddc!2016, the IP address is 192.168.0.30 and the switch ID is R1S3. Make a note of all of these values.

Unplug the Rack-Interconnect switch . Make a note of all the current connections to the TOR/Rack-Interconnect/management switches and hosts in the rack.

Note: Several critical alerts will likely be generated in SDDC Manager at this point.

Plugin the new Rack-Interconnect switch. Clear any existing configuration by issue the write erase and reload commands.

Exit out of POAP (Power of Auto Provisioning mode). Follow the instruction on the switch console screen:

Abort Auto Provisioning and continue with normal setup ?(yes/no)[n]:yes

When prompted, set a password for the admin user. Please assign an easy to remember password and make a note of the password.

Configure the IP address noted in Step 3 on the mgmt 0 interface, using a /21 subnet mask. Configure VRF to the mgmt 0 interface.

configure Terminal
interface mgmt 0
ip address 192.168.0.30/21
vrf member management
no shut
end

Note: Replace 192.168.0.30 with the appropriate IP address.

Verify network connectivity on the new Rack-Interconnect switch with the ping <other Rack-Interconnect switch IP address> vrf management command.

Copy the configuration backup on the Management Switch (noted in Step 3) to the Rack-Interconnect switch:

copy scp: bootflash: vrf management
Enter source filename: /tmp/<name of Rack-Interconnect switch .gz file noted in Step 3>
Enter hostname for the scp server: 192.168.3.254
Enter username: cumulus
[email-masked] password: <Management Switch's cumulus user password>

Note: Output similar to the following will be displayed when the copy starts:

<name of Rack-Interconnect switch .gz file noted in Step 3>    100% 1891     1.9KB/s   00:00
Copy complete, now saving to disk (please wait)

Decompress and Install the restored configuration in the TOR switch's running configuration:

gunzip bootflash:/// <name of Rack-Interconnect switch .gz file noted in Step 3>
copy <extracted file from previous step> startup-config
copy startup-config running-config

Reboot the Rack-Interconnect switch with the reload command.

Note: Wait until a system ready message is displayed before proceeding.

Verify the configuration on the Rack-Interconnect switch.

Note: The following is an example of from a dual rack installation.

Spine ports 1,2,3,4 should be up, and have "duplex full" setting. Verify this with the show run interface e1/<1-4> commands. Output similar to the following should be returned

interface Ethernet1/1

description Rack-1 Link
switchport
switchport mode trunk
speed 40000
no negotiate auto
duplex full
mtu 9216
channel-group 1
no shutdown

interface Ethernet1/2

description Rack-1 Link
switchport
switchport mode trunk
speed 40000
no negotiate auto
mtu 9216
channel-group 1
no shutdown

interface Ethernet1/3

description Rack-2 Link
switchport
switchport mode trunk
speed 40000
no negotiate auto
mtu 9216
channel-group 2
no shutdown

interface Ethernet1/4

description Rack-2 Link
switchport
switchport mode trunk
speed 40000
no negotiate auto
duplex full
mtu 9216
channel-group 2
no shutdown

If ports 3 and 4 are not in the channel-group 2 port channel group (per output above), the following commands can be used to place them in the correct port channel group:

int e1/3-4
channel-group 2 force

Regenerate the VMCA signed certificate for this IP Address and configure the switch with the certificate.

Download the attached fru_switch_2.3.zip file. Extract the contents.

Use a file transfer utility to copy the extracted fru_switch_2.3.py file to the /tmp folder on the SDDC Manager Controller virtual machine.

SSH to the SDDC Manager Controller virtual machine as the root user.

Execute the following command to run the fru_switch_2.3.py script:

python /tmp/fru_switch_2.3.py

Note: Enter the replacement TOR switch's IP address when prompted.

Run the following commands from the SDDC Manager Controller VM to verify that a 200 OK response is returned from the Hardware Monitoring Service (HMS):

curl -X GET http://localhost:8448/api/1.0/hms/switches/<switch-id>

Note: Replace <switch-id> with the name value noted in the Step 4 (R1S3 for this example).

Replacing an Arista Rack-Interconnect Switch:

Login to the SDDC Manager virtual machine as the root user.

Ensure that there is a backup of the Rack-Interconnect switch configuration under the /var/tmp folder.

There should be multiple folders present, with names similar to backup-<date>, backup-bringup-<date> and scheduled-backup-<date>.

In each folder there should exist a folder named switch.

In the switch folder, there should be a .gz file for each TOR and Rack-Interconnect switch

Identify the most recent .gz file for the Arista TOR switch (<switch-id>-<IP address>-arista-running-config-<date>.gz). If one does not exist and the Rack-Interconnect switch to be replaced is still accessible, run /opt/vmware/sddc-support/sos --backup to generate one.

Copy the Arista Rack-Interconnect Switch backup file noted in Step 2 to the Management Switch:

scp <backup filename> [email-masked]:/tmp

Note: Enter the Management Switch's cumulus user password when prompted.

Use the /home/vrack/bin/lookup-passwords command to find the current password, IP Address and the switch ID on the Rack-Interconnect switch. Output similar to the following will be returned:

SPINE: Arista Networks DCS-7050QX
identifiers: 192.168.0.30, R1S3
workload: hardware
username: admin
password: EvoSddc!2016
type: SSH

Note: In this example, the password is EvoSddc!2016, the IP address is 192.168.0.30 and the switch ID is R1S3. Make a note of all of these values.

Unplug the Rack-Interconnect switch . Make a note of all the current connections to the TOR/Rack-Interconnect/management switches and hosts in the rack.

Note: Several critical alerts will likely be generated in SDDC Manager at this point.

Plugin the new Rack-Interconnect switch. Clear any existing configuration by issue the write erase and reload commands.

Exit out of POAP (Power of Auto Provisioning mode) by issuing the zerotouch cancel command.

Configure the IP address noted in Step 4 on the management 1 interface, using a /22 subnet mask.

configure Terminal
interface management 1
ip address 192.168.0.30/22

Note: Replace 192.168.0.30 with the appropriate IP address.

Verify network connectivity on the new Rack-Interconnect switch with the ping <other Rack-Interconnect switch IP address> command.

Copy the configuration backup on the Management Switch (noted in Step 3) to the Rack-Interconnect switch:

bash
cd /mnt/flash
scp [email-masked]:/tmp/<name of Rack-Interconnect switch .gz file noted in Step 3> .

Note: Enter the management switch cumulus user's password when prompted.

exit

Note: Output similar to the following will be displayed when the copy starts:

<name of Rack-Interconnect switch .gz file noted in Step 3>    100% 1835     1.8KB/s   00:00

Verify that the new file exists with the dir flash command.

Decompress and Install the restored configuration in the TOR switch's running configuration:

bash
cd /mnt/flash
gunzip <name of TOR switch .gz file noted in Step 3>
exit
copy <extracted file from previous step> startup-config
copy startup-config running-config

Note: The password will now be updated to what was noted in Step 4.

Regenerate the VMCA signed certificate for this IP Address and configure the switch with the certificate.

Download the attached fru_switch_2.3.zip file. Extract the contents.

Use a file transfer utility to copy the extracted fru_switch_2.3.py file to the /tmp folder on the SDDC Manager Controller virtual machine.

SSH to the SDDC Manager Controller virtual machine as the root user.

Execute the following command to run the fru_switch_2.3.py script:

python /tmp/fru_switch_2.3.py

Note: Enter the replacement TOR switch's IP address when prompted.

Run the following commands from the SDDC Manager Controller VM to verify that a 200 OK response is returned from the Hardware Monitoring Service (HMS):

curl -X GET http://localhost:8448/api/1.0/hms/switches/<switch-id>

Note: Replace <switch-id> with the name value noted in the Step 4 (R1S3 for this example).

Related Information

To be alerted when this document is updated, click the Subscribe to Article link in the Actions box.

Detectable by VMware SkylineTM

Actions

Copy

link to clipboard
copied!

Print

Language:
English

Attachments

fru_switch_2.3.zip

Additional Resources

No results for undefined

Ask The Community

Get answers quickly from VMware experts in the community

Post Subject

CONTINUE IN COMMUNITIES

Related Products:

VMware Cloud Foundation

Related Versions:

VMware Cloud Foundation 2.2.x

Take Our Survey

Support Assistant