Loading

×Sorry to interrupt
CSS Error

Refresh

Cookie Preference Center

General Information on cookies

When you visit our website, we use cookies to ensure that we give you the best experience. This information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies by clicking on the different category headings to find out more and change your settings. However, blocking some types of cookies may impact your experience on the site and the services we are able to offer. Further information can be found in our

Cookie Policy.

Strictly Necessary

Strictly Necessary

Always Active

Strictly Necessary

Strictly necessary cookies are always enabled since they are essential for our website to function. They enable core functionality such as security, network management, and website accessibility. You can set your browser to block or alert you about these cookies, but this may affect how the website functions. For more information please visit www.aboutcookies.org or www.allaboutcookies.org.

Cookie Details‎

Performance

Performance

Performance

Performance cookies are used to analyze the user experience to improve our website by collecting and reporting information on how you use it. They allow us to know which pages are the most and least popular, see how visitors move around the site, optimize our website and make it easier to navigate.

Cookie Details‎

Please confirm your settings by refreshing the page.

Functional

Functional

Functional

Functional cookies help us keep track of your past browsing choices so we can improve usability and customize your experience. These cookies enable the website to remember your preferred settings, language preferences, location and other customizable elements such as font or text size. If you do not allow these cookies, then some or all of these services may not function properly.

Cookie Details‎

Advertising

Advertising

Advertising

Advertising cookies are used to send you relevant advertising and promotional information. They may be set through our site by third parties to build a profile of your interests and show you relevant advertisements on other sites. These cookies do not directly store personal information, but their function is based on uniquely identifying your browser and internet device.

Cookie Details‎

Advertising Cookies

Consent Leg.Interest

Select All Vendors

Select All Vendors

Select All Hosts

Select All

Clear Filters

Information storage and access

Apply

Confirm My Choices Reject All Allow All

My Subscriptions

Loading

Knowledge Base

Frequently Asked Questions for vStorage APIs for Array Integration (1021976)

Last Updated: 2/27/2024Categories: InformationalTotal Views: 381367Language:
Chinese (Simplified)JapaneseEnglish

16 like this article

Subscribe to get an email
newsletter with new updates

16 people found this helpful Please provide article feedback:

Purpose

This article provides answers to frequently asked questions about vStorage APIs for Array Integration (VAAI).

Resolution

vStorage APIs for Array Integration is a feature introduced in ESXi/ESX 4.1 that provides hardware acceleration functionality. It enables your host to offload specific virtual machine and storage management operations to compliant storage hardware. With the storage hardware assistance, your host performs these operations faster and consumes less CPU, memory, and storage fabric bandwidth.

For more information about vStorage APIs for Array Integration and when it is used, see the Storage Hardware Acceleration section in the Configuration Guide for your version of ESXi/ESX.

What does VAAI require?

VAAI requires:

ESXi/ESX 4.1 or later

Standard, Enterprise, Enterprise Plus or Remote Office Branch Office licensing for ESXi/ESX hosts

Storage arrays that support VAAI storage-based hardware acceleration

Notes:

ESXi/ESX 4.1 does not support hardware acceleration with NAS storage devices.

Support for NAS storage devices was introduced in ESXi 5.x.

How do I know if my storage array supports VAAI?

To determine if your storage array supports VAAI, see the Hardware Compatibility List or consult your storage vendor. To enable the hardware acceleration on the storage array, check with your storage vendor. Some storage arrays require explicit activation of hardware acceleration support.

Note:

In VMware HCL version 2, there are options under the Features Category and Features fields where arrays can be searched against to verify if they are VAAI-certified.

What are the fundamental operations used by VAAI and what are their corresponding Advanced Settings?

VAAI uses these fundamental operations:

Atomic Test & Set (ATS), which is used during creation and locking of files on the VMFS volume

Clone Blocks/Full Copy/XCOPY, which is used to copy or migrate data within the same physical array

Zero Blocks/Write Same, which is used to zero-out disk regions

Thin Provisioning in ESXi 5.x and later hosts, which allows the ESXi host to tell the array when the space previously occupied by a virtual machine (whether it is deleted or migrated to another datastore) can be reclaimed on thin provisioned LUNs.

Block Delete in ESXi 5.x and later hosts, which allows for space to be reclaimed using the SCSI UNMAP feature. For more information on Block Delete/SCSI UNMAP, see Using vmkfstools to reclaim VMFS deleted blocks on thin-provisioned LUNs (2014849).

The fundamental operations are controlled by these advanced settings:

Advanced Parameter name

Description

HardwareAcceleratedLocking

Atomic Test & Set (ATS), which is used during creation of files on the VMFS volume

HardwareAcceleratedMove

Clone Blocks/Full Copy/XCOPY, which is used to copy data

HardwareAcceleratedInit

Zero Blocks/Write Same, which is used to zero-out disk regions

Note: For Thin Provisioning, enabling/disabling occurs on the array and not on the ESXi host.

What improvements have been made to VAAI for ESXi 5.x?

To encourage customers to use larger and fewer datastores, support for Thin Provisioning VAAI primitive has been added in ESXi 5.0.

In ESXi 5.x, support for NAS Hardware Acceleration is included with support for these primitives:

Full File Clone – Like the Full Copy VAAI primitive provided for block arrays, this Full File Clone primitive enables virtual disks to be cloned by the NAS device.

Native Snapshot Support – Allows creation of virtual machine snapshots to be offloaded to the array.

Extended Statistics – Enables visibility to space usage on NAS datastores and is useful for Thin Provisioning.

Reserve Space – Enables creation of thick virtual disk files on NAS.

Note: Previously, the only supported VMDK type that could be created on NAS was thin.

Support for Block Delete to reclaim space use on thin provisioned LUNs on support arrays is also included in ESXi 5.x. For more information on Block Delete/SCSI UNMAP, see Using vmkfstools to reclaim VMFS deleted blocks on thin-provisioned LUNs (2014849).

How do I know if VAAI is enabled?

To determine if VAAI is enabled using the vSphere Client:

In the vSphere Client inventory panel, click the host.

Click the Configuration tab, then click Advanced Settings under Software.

Check that these options are set to 1 (enabled):

DataMover.HardwareAcceleratedMove
DataMover.HardwareAcceleratedInit
VMFS3.HardwareAcceleratedLocking

Note: These options are enabled by default.

On ESXi/ESX 4.x hosts, to determine if VAAI is enabled using the service console in ESX or the vCLI in ESXi, use the esxcfg-advcfg command to check if the options are set to 1 (enabled):

# esxcfg-advcfg -g /DataMover/HardwareAcceleratedMove
# esxcfg-advcfg -g /DataMover/HardwareAcceleratedInit
# esxcfg-advcfg -g /VMFS3/HardwareAcceleratedLocking

On ESXi 5.x hosts, to determine if VAAI is enabled, run these commands and check if Int Value is set to 1 (enabled):

# esxcli system settings advanced list -o /DataMover/HardwareAcceleratedMove
# esxcli system settings advanced list -o /DataMover/HardwareAcceleratedInit
# esxcli system settings advanced list -o /VMFS3/HardwareAcceleratedLocking

You see output similar to:

Path: /VMFS3/HardwareAcceleratedLocking
Type: integer
Int Value: 1 <-- set to 1 if enabled
Default Int Value: 1
Min Value: 0
Max Value: 1
String Value:
Default String Value:
Valid Characters:
Description: Enable hardware accelerated VMFS locking (requires compliant hardware)

Note: For Thin Provisioning, enabling/disabling occurs on the array and not on the ESXi host.

When enabling or disabling VAAI settings, is a reboot required?

No, a host reboot is not required. The changes made may or may not be effective immediately depending on use case. For details please see below table:.

Use Case

Effective

ESXi Host reboot Needed

Volume mount/unmount Needed

Remark

Disabling Hardware Accelerated Move (XCOPY) in ESXi

Immediate

No

No

Disabling Hardware Accelerated Init (WRITESAME) in ESXi

Immediate

No

No

Disabling ATS Usage at ESXi host level - For upgraded VMFS datastores a.k.a Non ATS-Only VMFS datastores

Immediate

No

No

Disabling ATS Usage at ESXi host level - for ATS-Only VMFS datastores

No

No

Yes

Not effective unless the datastore is unmounted and mounted back.
If you reboot the ESXi host to achieve the mount/unmount you will potentially end up in a mixture i.e. ATS+SCSI locking till all the host are rebooted which is not recommended

Disabling ATS usage per VMFS datastore - For upgraded VMFS datastores a.k.a Non ATS-Only VMFS datastores

NA

NA

NA

There is no option/need to disable ATS on such datastore.

Disabling ATS usage per VMFS datastore - For ATS-Only VMFS datastores

No

No

Yes

Datastore must be unmounted before executing command to disable ATS usage.

Disabling ATS Usage only for VMFS heartbeats

Immediate

No

No

What happens if I have VAAI enabled on the host but some of my disk arrays do not support it?

When storage devices do not support or provide only partial support for the host operations, the host reverts to its native methods to perform the unsupported operations.

Are there any caveats that I should be aware of in ESXi 5.x?

Use of the Atomic Test-and-Set (ATS) primitive depends on the type of filesystem:

On VAAI Hardware

New VMFS-5

Upgraded VMFS-5

VMFS-3

Single-extent datastore

ATS only [1]

ATS, but fall back to SCSI-2 reservations

ATS, but fall back to SCSI-2 reservations

Multi-extent datastore

Only allow spanning on ATS hardware [2]

ATS, except when locks on non-head

ATS, except when locks on non-head

Notes:

If a new VMFS-5 is created on a non-ATS storage device, SCSI-2 reservations will be used.

When creating a multi-extent datastore where ATS is used, the vCenter Server will filter out non-ATS devices, so that only devices that support the ATS primitive can be used.

Are there any caveats that I should be aware of in ESXi/ESX 4.x?

There are a few reasons why a hardware-accelerated operation may fail.

For any primitive that the array does not implement, the array returns an error, which triggers the ESX data mover to attempt the operation using software data movement. In the case of ATS, it reverts to using SCSI Reservations.

VAAI hardware offload cannot be used when:

The source and destination VMFS volumes have different block sizes

The source file type is RDM and the destination file type is non-RDM (regular file)

The source VMDK type is eagerzeroedthick and the destination VMDK type is thin

The source or destination VMDK is any kind of sparse or hosted format

Cloning a virtual machine that has snapshots because this process involves consolidating the snapshots into the virtual disks of the target virtual machine.

The logical address and/or transfer length in the requested operation is not aligned to the minimum alignment required by the storage device (all datastores created with the vSphere Client are aligned automatically)

The VMFS datastore has multiple LUNs/extents spread across different arrays

Notes:

When using enhanced vMotion (move both VM and datastore simultaneously), the source and destination hosts must both see the source and destination datastores, or the file transfer defaults to NFC over the the management network, instead of VAAI.

Hardware cloning between arrays (even if within the same VMFS datastore) does not work.

For more information on supportability with Horizon View, see View Composer API for Array Integration (VCAI) support in VMware Horizon View (2061611).

What do I need to know about the Hardware Acceleration Support Status?

If you go to Host > Configuration > Storage, you can see the Hardware Acceleration Status in the panel on the right side.

For each storage device and datastore, the vSphere Client displays the hardware acceleration support status in the Hardware Acceleration column of the Devices view and the Datastores view.

The status values are Unknown, Supported, and Not Supported. The initial value is Unknown. The status changes to Supported after the host successfully performs the offload basic operations. If the offload operation fails, the status changes to Not Supported.

To determine if your storage device supports VAAI, test the Full Copy VAAI primitive:

Using the vSphere Client, browse the datastore and locate a virtual disk (VMDK) of at least 4 MB that is not in use.

Copy the virtual disk to a new file.

Check the Hardware Acceleration status to verify that it changes from Unknown to either Supported or Not Supported.

Note: VAAI primitives can also be tested by creating a virtual machine with at least one new virtual disk, or cloning a virtual machine.

Can I check the VAAI status from the command line?

On ESXi 5.x

To check the VAAI status, run this command:

# esxcli storage core device vaai status get

You see output similar to:

mpx.vmhba1:C0:T1:L0
VAAI Plugin Name:
ATS Status: unsupported
Clone Status: unsupported
Zero Status: unsupported
Delete Status: unsupported

mpx.vmhba1:C0:T0:L0
VAAI Plugin Name:
ATS Status: unsupported
Clone Status: unsupported
Zero Status: unsupported
Delete Status: unsupported

On ESXi/ESX 4.x

To check the VAAI status, run this command:

# esxcfg-scsidevs -l | egrep "Display Name:|VAAI Status:"

You see output similar to:

Display Name: Local VMware Disk (mpx.vmhba2:C0:T0:L0)
VAAI Status: unsupported
Display Name: Local USB CD-ROM (mpx.vmhba32:C0:T0:L0)
VAAI Status: unknown
Display Name: Local TEAC CD-ROM (mpx.vmhba3:C0:T0:L0)
VAAI Status: unknown
Display Name: DGC Fibre Channel Disk (naa.6006016030801c00f8506073d0d7de11)
VAAI Status: unknown
Display Name: DGC Fibre Channel Disk (naa.6006016030801c00f9506073d0d7de11)
VAAI Status: unknown

Are changes to VAAI settings logged?

Changes to VAAI advanced settings (for example, enabling or disabling VAAI) are recorded in the VMkernel logs at /var/log/vmkernel (for ESXi 5.0 vmkernel.log) or /var/log/messages.

After enabling VAAI using the advanced settings, the VMkernel log contains entries similar to:

cpuN:1234)Config: 297: "HardwareAcceleratedMove" = 1, Old Value: 0, (Status: 0x0)
cpuN:1234)Config: 297: "HardwareAcceleratedInit" = 1, Old Value: 0, (Status: 0x0)
cpuN:1234)Config: 297: "HardwareAcceleratedLocking" = 1, Old Value: 0, (Status: 0x0)

After disabling VAAI using the advanced settings, the VMkernel log contains entries similar to:

cpuN:1234)Config: 297: "HardwareAcceleratedMove" = 0, Old Value: 1, (Status: 0x0)
cpuN:1234)Config: 297: "HardwareAcceleratedInit" = 0, Old Value: 1, (Status: 0x0)
cpuN:1234)Config: 297: "HardwareAcceleratedLocking" = 0, Old Value: 1, (Status: 0x0)

VAAI For NAS Storage

In ESXi 5.x, NAS Hardware Acceleration was introduced with support for these primitives:

· Full File Clone – Like the Full Copy VAAI primitive provided for block arrays, this Full File Clone primitive enables virtual disks to be cloned by the NAS device.

· Native Snapshot Support – Allows creation of virtual machine snapshots to be offloaded to the array.

· Extended Statistics – Enables visibility to space usage on NAS datastores and is useful for Thin Provisioning.

· Reserve Space – Enables creation of thick virtual disk files on NAS.

With NAS storage devices, the hardware acceleration integration is implemented through vendor-specific NAS plug-ins. These plug-ins are typically created by vendors and are distributed as VIB packages through a website. No claim rules are required for the NAS plug-ins to function.

Installing the NAS Plugin

1. Place the host into maintenance mode

2. Set the host acceptance level:

# esxcli software acceptance set–-level=value

The host acceptance level must be the same or less restrictive than the acceptance level of any VIB you want to add to the host. The value can be one of the following:

· VMwareCertified

· VMwareAccepted

· PartnerSupported

· CommunitySupported

3. Install the VIB package:

# esxcli software vib install –-viburl=URL

The URL specifies the URL to the VIB package to install. http:, https:, ftp:, and file: are supported.

4. Restart the host for the installation to take effect

Verify Hardware Acceleration Status for NAS

You can verify the NAS hardware acceleration status using the following command:

# esxcli storage nfs list

The Hardware Acceleration column in the output shows the status

Disabling VAAI when using NAS storage

To disable NAS VAAI functionality the plugin must be removed using the below commands.

1. Log into ESXi using SSH/DCUI and execute command:

# esxcli software vib remove –-vibname=name
For Example: # esxcli software vib remove –-vibname=NetAppNasPlugin

2. Verify that the plugin has been removed using command:

esxcli software vib list

3. Reboot the ESXi host to take effect.

Note: The advanced configuration options for Block VAAI mentioned above (DataMover.HardwareAcceleratedMove, DataMover.HardwareAcceleratedInit, VMFS3.HardwareAcceleratedLocking) do not affect NAS VAAI functionality

Related Information

For more information, see the vSphere Storage Guide for your release of vSphere:

vSphere 5.5 Storage Guide

vSphere 5.1 Storage Guide

vSphere 5.0 Storage Guide

vSphere 4.1 SAN Configuration Guide

This is a list of commonly used SCSI opcodes related to VAAI operations:

0x93

WRITE SAME(16)

0x41

WRITE SAME(10)

0x42

UNMAP

0x89

SCSI COMPARE and WRITE - ATS

0x83

EXTENDED COPY

Occasionally, failures during VAAI operations would be logged in the vmkernel.log file under /var/log as shown in this example:

Cmd(0x41248092e240) 0x42, CmdSN 0x13bb23 to dev "naa.xxxxxxxx" failed H:0x0 D:0x2

How to reclaim VMFS deleted blocks on thin-provisioned LUNs
vStorage APIs for Array Integration 常见问题
esxtop コマンドを使用して、 VAAI の監視および状態確認をする方法

Detectable by VMware SkylineTM

Actions

Copy

link to clipboard
copied!

Print

Language:
Chinese (Simplified)JapaneseEnglish

Attachments

Additional Resources

KB
•
End Of General Availability of the Free vSphere Hypervisor (ESXi 7.x and 8.x) (2107518)

KB
•
Download and Install VMware Fusion (2014097)

Results 1-2 of 2

Ask The Community

Get answers quickly from VMware experts in the community

Post Subject

CONTINUE IN COMMUNITIES

Related Products:

VMware vSphere ESXi

VMware ESX Server

Related Versions:

VMware vSphere ESXi 6.0

VMware vSphere ESXi 5.5

VMware vSphere ESXi 5.1

VMware vSphere ESXi 5.0

VMware ESXi 4.1.x Installable

VMware ESXi 4.1.x Embedded

VMware ESX 4.1.x

Take Our Survey

Support Assistant