Loading

×Sorry to interrupt
Uncaught ReferenceError: addCTracking is not defined
throws at https://kb.vmware.com/resource/KM_TealiumTrackingJS?v=0.5:100:1

Refresh

Cookie Preference Center

General Information on cookies

When you visit our website, we use cookies to ensure that we give you the best experience. This information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies by clicking on the different category headings to find out more and change your settings. However, blocking some types of cookies may impact your experience on the site and the services we are able to offer. Further information can be found in our

Cookie Policy.

Strictly Necessary

Strictly Necessary

Always Active

Strictly Necessary

Strictly necessary cookies are always enabled since they are essential for our website to function. They enable core functionality such as security, network management, and website accessibility. You can set your browser to block or alert you about these cookies, but this may affect how the website functions. For more information please visit www.aboutcookies.org or www.allaboutcookies.org.

Cookie Details‎

Performance

Performance

Performance

Performance cookies are used to analyze the user experience to improve our website by collecting and reporting information on how you use it. They allow us to know which pages are the most and least popular, see how visitors move around the site, optimize our website and make it easier to navigate.

Cookie Details‎

Please confirm your settings by refreshing the page.

Functional

Functional

Functional

Functional cookies help us keep track of your past browsing choices so we can improve usability and customize your experience. These cookies enable the website to remember your preferred settings, language preferences, location and other customizable elements such as font or text size. If you do not allow these cookies, then some or all of these services may not function properly.

Cookie Details‎

Advertising

Advertising

Advertising

Advertising cookies are used to send you relevant advertising and promotional information. They may be set through our site by third parties to build a profile of your interests and show you relevant advertisements on other sites. These cookies do not directly store personal information, but their function is based on uniquely identifying your browser and internet device.

Cookie Details‎

Advertising Cookies

Consent Leg.Interest

Select All Vendors

Select All Vendors

Select All Hosts

Select All

Clear Filters

Information storage and access

Apply

Confirm My Choices Reject All Allow All

My Subscriptions

Loading

Knowledge Base

High VM Stun time during snapshot deletion or SVmotion failure on ESXi 6.7U2 or later (77030)

Last Updated: 2/22/2024Categories: TroubleshootingTotal Views: 28354Language:
Chinese (Simplified)JapaneseEnglish

10 like this article

Subscribe to get an email
newsletter with new updates

10 people found this helpful Please provide article feedback:

Symptoms

Environment is running ESXi 6.7U2 and later

Storage is using > 1 MB unmap granularity

Datastores are using VMFS6.

Snapshot Deletion Operation

In the virtual machines vmware.log file, you see log entries similar  to:

2019-05-08T15:22:16.507Z| vcpu-0| I125: DISKLIB-CTK   : Forcing flush of change info for "/vmfs/volumes/5c5b4879-4163c971-b0e6-246e963d8d88/VM/VM-000001-ctk.vmdk".
2019-05-08T15:22:16.518Z| vcpu-0| I125: DISKLIB-CTK   : ChangeTracker_EndCombine()
2019-05-08T15:25:41.336Z| vcpu-0| I125: DISKLIB-CTK   : Unlinked /vmfs/volumes/5c5b4879-4163c971-b0e6-246e963d8d88 /VM/VM-ctk.vmdk, tmp file: /vmfs/volumes/5c5b4879-4163c971-b0e6-246e963d8d88/VM/VM-ctk.vmdk-tmp
019-05-08T15:25:41.556Z| vcpu-0| I125: Checkpoint_Unstun: vm stopped for 205052348 us

In the vmkernel.log file, you see log entries similar to:

2019-05-08T15:22:20.528Z cpu54:2192401)DLX: 4319: vol 'Datastore', lock at 10182656: [Req mode 1]>Checking liveness:
2019-05-08T15:22:20.528Z cpu54:2192401)[type 10c00002 offset 10182656 v 133804, hb offset 3674112 gen 255, mode 1, owner 5cca60c4-0579f659-c9e4-246e963d8fd0 mtime 7306729 num 0 gblnum 0 gblgen 0 gblbrk 0]
2019-05-08T15:22:24.529Z cpu32:2192401)DLX: 4319: vol 'datastore', lock at 10182656: [Req mode 1]>Checking liveness:
2019-05-08T15:22:24.529Z cpu32:2192401)[type 10c00002 offset 10182656 v 133806, hb offset 3674112 gen 255, mode 1, owner 5cca60c4-0579f659-c9e4-246e963d8fd0 mtime 7306869 num 0 gblnum 0 gblgen 0 gblbrk 0]
2019-05-08T15:22:28.529Z cpu32:2192401)DLX: 4968: vol 'datastore', lock at 10182656: [Req mode: 1] >Not free:
2019-05-08T15:22:28.529Z cpu32:2192401)[type 10c00002 offset 10182656 v 133808, hb offset 3674112 gen 255, mode 1, owner 5cca60c4-0579f659-c9e4-246e963d8fd0 mtime 7306910 num 0 gblnum 0 gblgen 0 gblbrk 0] alloc owner 3473408
2019-05-08T15:22:28.529Z cpu32:2192401)Res3: 2325: Rank violation threshold reached: cid 0xc1d0000c, resType 1, cnum 75 vol datastore

SVmotion Failure

In the vmkernel.log file, you see log entries similar to:

2019-11-05T14:32:06.281Z| vmx| I125: [msg.configdb.open] An error occurred while opening configuration file "/vmfs/volumes/5a9e9fcb-f0e68c87-2cbe-000e1eea48b0/vm/vm": Failed to lock the file.
2019-11-05T14:32:06.281Z| vmx| I125: ----------------------------------------
2019-11-05T14:32:06.281Z| vmx| W115: Migrate: Failed to write out config file

2019-11-05T14:31:57.877Z cpu28:2382713)DLX: 4319: vol 'datastore', lock at 59031552: [Req mode 1] Checking liveness:
2019-11-05T14:31:57.877Z cpu28:2382713)[type 10c00001 offset 59031552 v 2986, hb offset 3407872
gen 32505, mode 1, owner 5d8a2dab-ee894bff-6122-000e1eea48b0 mtime 14405428

Note: If you are on  an ESXi version lower than 6.7U2 or have a backend storage array that supports <= 1 mb unmap granularity you are NOT impacted with this issue.

Cause

Automatic Unmap manager in VMFS6  locking and unlocking these resource clusters repeatedly that cause this issue.

Resolution

This issue is resolved in

VMware vSphere ESXi 6.7 Patch release ESXi670-202004002 available at Customer Connect Patch Downloads

VMware vSphere 7.0b, available at VMware Downloads

Workaround

To workaround this issue, disable the automatic unmap processing on all the hosts sharing the volume.

Note: You can continue to use manual unmap using esxcli to reclaim space after disabling automatic unmap.

Run this command from one of the hosts sharing the volume.

esxcli storage vmfs reclaim config set --volume-label VolName --reclaim-priority=none

Unmount and remount the volume on all hosts accessing the volume for reclaim priority change to take effect.

esxcli storage filesystem unmount -l VolName
esxcli storage filesystem mount -l VolName

If you are unable to unmount the volume, automatic unmap will need to be toggled on/off on all the hosts accessing the volume.

Alternative step "In case unable to unmount the volume""
Run this command to toggle reclaim priority low/none on all the hosts accessing the volume for the reclaim priority change to take effect.

esxcli storage vmfs reclaim config set --volume-label VolName --reclaim-priority=low
esxcli storage vmfs reclaim config set --volume-label VolName --reclaim-priority=none

List the volumes with automatic unmap processing enabled with this command:

vsish -e ls /vmkModules/vmfs3/auto_unmap/volumes/

Notes:

None of the volumes with backend storage array having > 1 mb unmap granularity should appear in the volumes listed out by the above command.

If even after the Alternate Step 2  any of the hosts still lists such a volume you should fall back to the preferred Step 2 which is to remount the volume in question.

Related Information

Refer VMware KB for How to Download ESXi, vCenter Server Patches in Customer Connect

Detectable by VMware SkylineTM

Actions

Copy

link to clipboard
copied!

Print

Language:
Chinese (Simplified)JapaneseEnglish

Attachments

Additional Resources

KB
•
End Of General Availability of the Free vSphere Hypervisor (ESXi 7.x and 8.x) (2107518)

KB
•
仮想マシンのネットワークアダプタを再接続するときに「デバイス「0」の構成が無効です」というエラーが発生する (77134)

Results 1-2 of 2

Ask The Community

Get answers quickly from VMware experts in the community

Post Subject

CONTINUE IN COMMUNITIES

Related Products:

VMware vCenter Server

Related Versions:

Take Our Survey

Support Assistant