Loading

×Sorry to interrupt
Uncaught ReferenceError: addCTracking is not defined
throws at https://kb.vmware.com/resource/KM_TealiumTrackingJS?v=0.5:100:1

Refresh

Cookie Preference Center

General Information on cookies

When you visit our website, we use cookies to ensure that we give you the best experience. This information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies by clicking on the different category headings to find out more and change your settings. However, blocking some types of cookies may impact your experience on the site and the services we are able to offer. Further information can be found in our

Cookie Policy.

Strictly Necessary

Strictly Necessary

Always Active

Strictly Necessary

Strictly necessary cookies are always enabled since they are essential for our website to function. They enable core functionality such as security, network management, and website accessibility. You can set your browser to block or alert you about these cookies, but this may affect how the website functions. For more information please visit www.aboutcookies.org or www.allaboutcookies.org.

Cookie Details‎

Performance

Performance

Performance

Performance cookies are used to analyze the user experience to improve our website by collecting and reporting information on how you use it. They allow us to know which pages are the most and least popular, see how visitors move around the site, optimize our website and make it easier to navigate.

Cookie Details‎

Please confirm your settings by refreshing the page.

Functional

Functional

Functional

Functional cookies help us keep track of your past browsing choices so we can improve usability and customize your experience. These cookies enable the website to remember your preferred settings, language preferences, location and other customizable elements such as font or text size. If you do not allow these cookies, then some or all of these services may not function properly.

Cookie Details‎

Advertising

Advertising

Advertising

Advertising cookies are used to send you relevant advertising and promotional information. They may be set through our site by third parties to build a profile of your interests and show you relevant advertisements on other sites. These cookies do not directly store personal information, but their function is based on uniquely identifying your browser and internet device.

Cookie Details‎

Advertising Cookies

Consent Leg.Interest

Select All Vendors

Select All Vendors

Select All Hosts

Select All

Clear Filters

Information storage and access

Apply

Confirm My Choices Reject All Allow All

My Subscriptions

Loading

Knowledge Base

VMware response to ‘L1 Terminal Fault - VMM’ (L1TF - VMM) Speculative-Execution vulnerability in Intel processors for vSphere: CVE-2018-3646 (55806)

Last Updated: 2/27/2024Categories: SecurityTotal Views: 177549Language:
Chinese (Simplified)JapaneseEnglish

232 like this article

Subscribe to get an email
newsletter with new updates

232 people found this helpful Please provide article feedback:

Learn how VMware Skyline Advisor and Skyline Health Diagnostics work together to provide proactive intelligence and self-service log analysis.

Use Skyline Advisor to avoid issues before they occur across VCF, vSphere, vSAN, NSX, vROps & Horizon. Skyline Findings are based on top KBs, VMSAs/CVEs and design best practices.

Use Skyline Health Diagnostics to root cause vSphere & vSAN logs when an issue occurs. Skyline Health Diagnostic Findings are based on top KBs and VMSA/CVEs. You can use the Skyline Health Diagnostics tool in online or offline mode.

If you require additional help from technical support, please use Skyline Advisor to initiate a Log Assist to reduce time to upload logs. TSEs can also initiate logs to reduce effort.

Symptoms

This article documents the Hypervisor-Specific Mitigations required to address CVE-2018-3646 (L1 Terminal Fault - VMM) in vSphere.

The Update History section of this article will be revised if there is a significant change. Click Subscribe to Article in the Actions box to be alerted when new information is added to this document and sign up at our Security-Announce mailing list to receive new and updated VMware Security Advisories.

Introduction to CVE-2018-3646

Intel has disclosed details on a new class of CPU speculative-execution vulnerabilities known collectively as “L1 Terminal Fault” that can occur on past and current Intel processors (from at least 2009 – 2018) [See Table 1 for supported vSphere processors that are affected].

Like Meltdown, Rogue System Register Read, and "Lazy FP state restore", the “L1 Terminal Fault” vulnerability can occur when affected Intel microprocessors speculate beyond an unpermitted data access. By continuing the speculation in these cases, the affected Intel microprocessors expose a new side-channel for attack. (Note, however, that architectural correctness is still provided as the speculative operations will be later nullified at instruction retirement.)

CVE-2018-3646 is one of these Intel microprocessor vulnerabilities and impacts hypervisors. It may allow a malicious VM running on a given CPU core to effectively infer contents of the hypervisor's or another VM's privileged information residing at the same time in the same core's L1 Data cache. Because current Intel processors share the physically-addressed L1 Data Cache across both logical processors of a Hyperthreading (HT) enabled core, indiscriminate simultaneous scheduling of software threads on both logical processors creates the potential for further information leakage. CVE-2018-3646 has two currently known attack vectors which will be referred to here as "Sequential-Context" and "Concurrent-Context.” Both attack vectors must be addressed to mitigate CVE-2018-3646.

Attack Vector Summary

Sequential-context attack vector: a malicious VM can potentially infer recently accessed L1 data of a previous context (hypervisor thread or other VM thread) on either logical processor of a processor core.

Concurrent-context attack vector: a malicious VM can potentially infer recently accessed L1 data of a concurrently executing context (hypervisor thread or other VM thread) on the other logical processor of the hyperthreading-enabled processor core.

Mitigation Summary

Mitigation of the Sequential-Context attack vector is achieved by vSphere updates and patches. This mitigation is enabled by default and does not impose a significant performance impact. Please see the resolution section for details.

Mitigation of the Concurrent-context attack vector requires enablement of a new feature known as the ESXi Side-Channel-Aware Scheduler. The initial version of this feature will only schedule the hypervisor and VMs on one logical processor of an Intel Hyperthreading-enabled core. This feature may impose a non-trivial performance impact and is not enabled by default. Please see resolution section for details.

Important: Disabling Intel Hyperthreading in firmware/BIOS (or by using VMkernel.Boot.Hyperthreading) after applying vSphere updates and patches is not recommended and precludes potential vSphere scheduler enhancements and mitigations that will allow the use of both logical processors. Mitigation should be done by enabling the ESXi Side Channel Aware Scheduler (see below).

Unlike explicit disabling of Intel Hyperthreading in firmware/BIOS (or by using VMKernel.Boot.Hyperthreading), the side channel aware scheduler enablement will be ignored on AMD processors and newer Intel processors that are not vulnerable to L1TF-VMM. [See Table 1 for supported vSphere processors that are affected].

Resolution

The mitigation process for CVE-2018-3646 is divided into three phases:

Update Phase: Apply vSphere Updates and Patches

The Sequential-context attack vector is mitigated by a vSphere update to the product versions listed in VMware Security Advisory VMSA-2018-0020. This mitigation is dependent on Intel microcode updates (provided in separate ESXi patches for most Intel hardware platforms) which are also documented in VMSA-2018-0020. This mitigation is enabled by default and does not impose a significant performance impact.

Note: As displayed in the workflow above, vCenter Server should be updated prior to applying ESXi patches. Notification messages were added in the aforementioned updates and patches to explain that the ESXi Side-Channel-Aware Scheduler must be enabled to mitigate the Concurrent-context attack vector of CVE-2018-3646. If ESXi is updated prior to vCenter you may receive cryptic notification messages relating to this. After vCenter has been updated, the notifications will be shown correctly.

Planning Phase: Assess Your Environment

The Concurrent-context attack vector is mitigated through enablement of the ESXi Side-Channel-Aware Scheduler which is included in the updates and patches listed in VMSA-2018-0020. This scheduler is not enabled by default. Enablement of this scheduler may impose a non-trivial performance impact on applications running in a vSphere environment. The goal of the Planning Phase is to understand if your current environment has sufficient CPU capacity to enable the scheduler without operational impact.

The following list summarizes potential problem areas after enabling the ESXi Side-Channel-Aware Scheduler:

VMs configured with vCPUs greater than the physical cores available on the ESXi host

VMs configured with custom affinity or NUMA settings

VMs with latency-sensitive configuration

ESXi hosts with Average CPU Usage greater than 70%

Hosts with custom CPU resource management options enabled

HA Clusters where a rolling upgrade will increase Average CPU Usage above 100%

Important: The above list is meant to be a brief overview of potential problem areas related to enablement of the ESXi Side-Channel-Aware Scheduler. The VMware Performance Team has provided an in-depth guide as well as performance data in KB55767. It is strongly suggested to thoroughly review this document prior to enablement of the scheduler.

Note: It may be necessary to acquire additional hardware, or rebalance existing workloads, before enablement of the ESXi Side-Channel-Aware Scheduler. Organizations can choose not to enable the ESXi Side-Channel-Aware Scheduler after performing a risk assessment and accepting the risk posed by the Concurrent-context attack vector. This is NOT RECOMMENDED and VMware cannot make this decision on behalf of an organization.

Scheduler-Enablement Phase:

Enable the ESXi Side-Channel-Aware Scheduler in ESXi 5.5, 6.0, 6.5, and 6.7 (prior to 6.7u2) and 7.0.

After addressing the potential problem areas described above during the Planning Phase, the ESXi Side-Channel-Aware Scheduler must be enabled to mitigate the Concurrent-context attack vector of CVE-2018-3646. The scheduler can be enabled on an individual ESXi host via the advanced configuration option hyperthreadingMitigation.

Notes:

Enabling this option will result in the vSphere UI reporting only a single logical processor per physical core; halving the number of logical processors if Hyperthreading was previously enabled. In addition Hyperthreading may be reported as 'Disabled' in various configuration tabs.

The current ESXi Side-Channel-Aware scheduler also addresses CVE-2018-5407.

Enabling the ESXi Side-Channel-Aware Scheduler using the vSphere Web Client or vSphere Client

Connect to the vCenter Server using either the vSphere Web or vSphere Client.

Select an ESXi host in the inventory.

Click the Manage (5.5/6.0) or Configure (6.5/6.7/7.0) tab.

Click the Settings sub-tab.

Under the System heading, click Advanced System Settings.

Click in the Filter box and search VMkernel.Boot.hyperthreadingMitigation

Select the setting by name and click the Edit pencil icon.

Change the configuration option to true (default: false).

Click OK.

Reboot the ESXi host for the configuration change to go into effect.

Enabling the ESXi Side-Channel-Aware Scheduler using ESXi Embedded Host Client

Connect to the ESXi host by opening a web browser to https://HOSTNAME.

Click the Manage tab.

Click the Advanced settings sub-tab.

Click in the Filter box and search VMkernel.Boot.hyperthreadingMitigation

Select the setting by name and click the Edit pencil icon.

Change the configuration option to true (default: false).

Click Save.

Reboot the ESXi host for the configuration change to go into effect.

Enable ESXi Side-Channel-Aware Scheduler setting using ESXCLI

SSH to an ESXi host or open a console where the remote ESXCLI is installed. For more information, see the http://www.vmware.com/support/developer/vcli/.

Check the current runtime value of the HTAware Mitigation Setting by running esxcli system settings kernel list -o hyperthreadingMitigation

To enable HT Aware Mitigation, run this command:

esxcli system settings kernel set -s hyperthreadingMitigation -v TRUE

Reboot the ESXi host for the configuration change to go into effect.

Enable the ESXi Side-Channel-Aware Scheduler (SCAv1) or the ESXi Side-Channel-Aware Scheduler v2 (SCAv2) in ESXi 6.7u2 (13006603) or later

Note: ESXi 6.7u2 (13006603) and future release lines of ESXi include the ESXi Side-Channel-Aware Scheduler v2. Prior release lines such as 6.5, 6.0, and 5.5 cannot accommodate this new scheduler.

VMware has published a white paper entitled Performance of vSphere 6.7 Scheduling Options which provides a more detailed look into the performance differences between SCAv1 and SCAv2. Please review this document before continuing.

Enabling the ESXi Side-Channel-Aware Scheduler (SCAv1) using the vSphere Web Client or vSphere Client

Connect to the vCenter Server using either the vSphere Web or vSphere Client.

Select an ESXi host in the inventory.

Click the Configure tab.

Under the System heading, click Advanced System Settings.

Click Edit.

Click in the Filter box and search VMkernel.Boot.hyperthreadingMitigation.

Select the setting by name.

Change the configuration option to true (default: false).

Click in the Filter box and search VMkernel.Boot.hyperthreadingMitigationIntraVM.

Change the configuration option to true (default: true).

Click OK.

Reboot the ESXi host for the configuration change to go into effect.

Enabling the ESXi Side-Channel-Aware Scheduler (SCAv1) using ESXi Embedded Host Client

Connect to the ESXi host by opening a web browser to https://HOSTNAME.

Click Manage under host navigator.

Click the Advanced settings Tab.

Use the search box to find VMkernel.Boot.hyperthreadingMitigation

Select the VMkernel.Boot.hyperthreadingMitigation setting and click the Edit Option.

Change the configuration option to true (default: false).

Click Save.

Use the search box to find VMkernel.Boot.hyperthreadingMitigationIntraVM.

Select the VMkernel.Boot.hyperthreadingMitigationIntraVM setting and click the Edit Option.

Change the configuration option to true (default: true).

Click Save.

Reboot the ESXi host for the configuration change to go into effect.

Enable ESXi Side-Channel-Aware Scheduler (SCAv1) using ESXCLI

SSH to an ESXi host or open a console where the remote ESXCLI is installed. For more information, see the https://www.vmware.com/support/developer/vcli/.

Check the current runtime values by running esxcli system settings kernel list -o hyperthreadingMitigation and esxcli system settings kernel list -o hyperthreadingMitigationIntraVM.

To enable the ESXi Side-Channel-Aware Scheduler Version 1 run these commands:

esxcli system settings kernel set -s hyperthreadingMitigation -v TRUE

esxcli system settings kernel set -s hyperthreadingMitigationIntraVM -v TRUE

Reboot the ESXi host for the configuration change to go into effect.

Enabling the ESXi Side-Channel-Aware Scheduler Version 2 (SCAv2) using the vSphere Web Client or vSphere Client

Connect to the vCenter Server using either the vSphere Web or vSphere Client.

Select an ESXi host in the inventory.

Click the Configure tab.

Under the System heading, click Advanced System Settings.

Click Edit.

Click in the Filter box and search VMkernel.Boot.hyperthreadingMitigation.

Select the setting by name.

Change the configuration option to true (default: false).

Click in the Filter box and search VMkernel.Boot.hyperthreadingMitigationIntraVM.

Change the configuration option to false (default: true).

Click OK.

Reboot the ESXi host for the configuration change to go into effect.

Enabling the ESXi Side-Channel-Aware Scheduler Version 2 (SCAv2) using ESXi Embedded Host Client

Connect to the ESXi host by opening a web browser to https://HOSTNAME.

Click Manage under host navigator.

Click the Advanced settings Tab.

Use the search box to find VMkernel.Boot.hyperthreadingMitigation.

Select the VMkernel.Boot.hyperthreadingMitigation setting and click the Edit Option.

Change the configuration option to true (default: false).

Click Save.

Use the search box to find VMkernel.Boot.hyperthreadingMitigationIntraVM.

Select the VMkernel.Boot.hyperthreadingMitigationIntraVM setting and click the Edit Option.

Change the configuration option to false (default: true).

Click Save.

Reboot the ESXi host for the configuration change to go into effect.

Enable ESXi Side-Channel-Aware Scheduler Version 2 (SCAv2) using ESXCLI

SSH to an ESXi host or open a console where the remote ESXCLI is installed. For more information, see the https://www.vmware.com/support/developer/vcli/.

Check the current runtime values by running esxcli system settings kernel list -o hyperthreadingMitigation and esxcli system settings kernel list -o hyperthreadingMitigationIntraVM

To enable the ESXi Side-Channel-Aware Scheduler Version 1 run these commands:

esxcli system settings kernel set -s hyperthreadingMitigation -v TRUE

esxcli system settings kernel set -s hyperthreadingMitigationIntraVM -v FALSE

Reboot the ESXi host for the configuration change to go into effect.

ESXi 6.7u2 (and later) Scheduler Configuration Summary

hyperthreadingMitigation

hyperthreadingMitigationIntraVM

Scheduler Enabled

FALSE

TRUE or FALSE

Default scheduler (unmitigated)

TRUE

TRUE

SCAv1

TRUE

FALSE

SCAv2

HTAware Mitigation Tool

VMware has provided a tool to assist in performing both the Planning Phase and the Scheduler-Enablement Phase at scale. This tool has been updated to include SCAv2 support and can be found in KB56931 along with detailed instructions on its usage, capabilities, and limitations.

Table 1: Affected Intel Processors Supported by ESXi

Intel Code Name

FMS

Intel Brand Names

Nehalem-EP

0x106a5

Intel Xeon 35xx Series;
Intel Xeon 55xx Series

Lynnfield

0x106e5

Intel Xeon 34xx Lynnfield Series

Clarkdale

0x20652

Intel i3/i5 Clarkdale Series;
Intel Xeon 34xx Clarkdale Series

Arrandale

0x20655

Intel Core i7-620LE Processor

Sandy Bridge DT

0x206a7

Intel Xeon E3-1100 Series;
Intel Xeon E3-1200 Series;
Intel i7-2655-LE Series;  Intel i3-2100 Series

Westmere EP

0x206c2

Intel Xeon 56xx Series;
Intel Xeon 36xx Series

Sandy Bridge EP

0x206d7

Intel Pentium 1400 Series;
Intel Xeon E5-1400 Series;
Intel Xeon E5-1600 Series;
Intel Xeon E5-2400 Series;
Intel Xeon E5-2600 Series;
Intel Xeon E5-4600 Series

Nehalem EX

0x206e6

Intel Xeon 65xx Series;
Intel Xeon 75xx Series

Westmere EX

0x206f2

Intel Xeon E7-8800 Series;
Intel Xeon E7-4800 Series;
Intel Xeon E7-2800 Series

Ivy Bridge DT

0x306a9

Intel i3-3200 Series; Intel i7-3500-LE/UE, Intel i7-3600-QE,
Intel Xeon E3-1200-v2 Series;
Intel Xeon E3-1100-C-v2 Series;
Intel Pentium B925C

Haswell DT

0x306c3

Intel Xeon E3-1200-v3 Series

Ivy Bridge EP

0x306e4

Intel Xeon E5-4600-v2 Series;
Intel Xeon E5-2400-v2 Series;
Intel Xeon E5-2600-v2 Series;
Intel Xeon E5-1400-v2 Series;
Intel Xeon E5-2600-v2 Series

Ivy Bridge EX

0x306e7

Intel Xeon E7-8800/4800/2800-v2 Series

Haswell EP

0x306f2

Intel Xeon E5-2400-v3 Series;
Intel Xeon E5-1400-v3 Series;
Intel Xeon E5-1600-v3 Series;
Intel Xeon E5-2600-v3 Series;
Intel Xeon E5-4600-v3 Series

Haswell EX

0x306f4

Intel Xeon E7-8800/4800-v3 Series

Broadwell H

0x40671

Intel Core i7-5700EQ;
Intel Xeon E3-1200-v4 Series

Avoton

0x406d8

Intel Atom C2300 Series;
Intel Atom C2500 Series;
Intel Atom C2700 Series

Broadwell EP/EX

0x406f1

Intel Xeon E7-8800/4800-v4 Series;
Intel Xeon E5-4600-v4 Series;
Intel Xeon E5-2600-v4 Series;
Intel Xeon E5-1600-v4 Series

Skylake SP

0x50654

Intel Xeon Platinum 8100 (Skylake-SP) Series;
Intel Xeon Gold 6100/5100 (Skylake-SP) Series
Intel Xeon Silver 4100, Bronze 3100 (Skylake-SP) Series

Broadwell DE

0x50662

Intel Xeon D-1500 Series

Broadwell DE

0x50663

Intel Xeon D-1500 Series

Broadwell DE

0x50664

Intel Xeon D-1500 Series

Broadwell NS

0x50665

Intel Xeon D-1500 Series

Skylake H/S

0x506e3

Intel Xeon E3-1500-v5 Series;
Intel Xeon E3-1200-v5 Series

Kaby Lake H/S/X

0x906e9

Intel Xeon E3-1200-v6

Related Information

VMware Skyline Health Diagnostics for vSphere - FAQ

Know more about L1TF (L1 Terminal Fault) here

Virtual machines appear as invalid or orphaned in vCenter Server

Detectable by VMware SkylineTM

Actions

Copy

link to clipboard
copied!

Print

Language:
Chinese (Simplified)JapaneseEnglish

Attachments

Additional Resources

No results for undefined

Ask The Community

Get answers quickly from VMware experts in the community

Post Subject

CONTINUE IN COMMUNITIES

Related Products:

Datacenter

Related Versions:

Take Our Survey

Support Assistant