Loading

×Sorry to interrupt
CSS Error

Refresh

Cookie Preference Center

General Information on cookies

When you visit our website, we use cookies to ensure that we give you the best experience. This information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies by clicking on the different category headings to find out more and change your settings. However, blocking some types of cookies may impact your experience on the site and the services we are able to offer. Further information can be found in our

Cookie Policy.

Strictly Necessary

Strictly Necessary

Always Active

Strictly Necessary

Strictly necessary cookies are always enabled since they are essential for our website to function. They enable core functionality such as security, network management, and website accessibility. You can set your browser to block or alert you about these cookies, but this may affect how the website functions. For more information please visit www.aboutcookies.org or www.allaboutcookies.org.

Cookie Details‎

Performance

Performance

Performance

Performance cookies are used to analyze the user experience to improve our website by collecting and reporting information on how you use it. They allow us to know which pages are the most and least popular, see how visitors move around the site, optimize our website and make it easier to navigate.

Cookie Details‎

Please confirm your settings by refreshing the page.

Functional

Functional

Functional

Functional cookies help us keep track of your past browsing choices so we can improve usability and customize your experience. These cookies enable the website to remember your preferred settings, language preferences, location and other customizable elements such as font or text size. If you do not allow these cookies, then some or all of these services may not function properly.

Cookie Details‎

Advertising

Advertising

Advertising

Advertising cookies are used to send you relevant advertising and promotional information. They may be set through our site by third parties to build a profile of your interests and show you relevant advertisements on other sites. These cookies do not directly store personal information, but their function is based on uniquely identifying your browser and internet device.

Cookie Details‎

Advertising Cookies

Consent Leg.Interest

Select All Vendors

Select All Vendors

Select All Hosts

Select All

Clear Filters

Information storage and access

Apply

Confirm My Choices Reject All Allow All

Skip to main content

My Subscriptions

Loading

Knowledge Base

Cannot power on a virtual machine due to COW memory heap exhaustion (1009086)

Last Updated: 5/31/2017Categories: TroubleshootingTotal Views: 9126Language:
English

1 like this article

Subscribe to get an email
newsletter with new updates

1 people found this helpful Please provide article feedback:

Details

Cannot power on virtual machine due to COW memory heap exhaustion.

In the /var/log/hostd.log file, you see entries similar to:

2015-11-02T10:56:09.776Z [2DE80B70 verbose 'Vmsvc.vm:/vmfs/volumes/52fb3113-e61f6181-dd44-0025b5f2007d/Wpsapp01/Wpsapp01.vmx'] Handling message _vmx4: Heap cow already at its maximum size of 201331560. Cannot expand.

2015-11-02T10:56:09.979Z [2C580B70 verbose 'Vmsvc.vm:/vmfs/volumes/52fb3113-e61f6181-dd44-0025b5f2007d/Wpsapp01/Wpsapp01.vmx'] Power On message: Heap cow already at its maximum size of 201331560. Cannot expand.

In the /var/log/vmkwarning.log file, you see entries similar to:

WARNING: Heap: 1445: Heap cow could not be grown to accommodate the memory request
WARNING: Heap: 1522: Heap_Align(cow, 68096/68096 bytes, 4 align) failed. caller: 0xa0155e
WARNING: Cow: 576: No memory available! Called from 0xa0155e
WARNING: Cow: 1223: Creation of the device with device lib failed Out of memory

Cloning a virtual disk using vmkfstools -i fails.

In the /var/log/vmkernel.log file, you see entries similar to:

WARNING: Heap: 1435: Heap cow already at its maximumSize. Cannot expand.
WARNING: Heap: 1645: Heap_Align(cow, 1048576/1048576 bytes, 8 align) failed. caller: 0x418032712212
WARNING: Cow: 530: No memory available! Called from 0x418032712212 Cow: 1714: Failed on handle 12 (33833182) of 14 with Out of memory

Note: This log excerpt is an example. Date, time, and environmental variables may vary depending on your environment.

Solution

If you use snapshots on virtual machines running on an ESX host, each snapshot delta disk is a COW (Copy On Write) disk. For each one in use by running virtual machines, their data structures take up ESX kernel memory. This allocation is known as the COW heap. This memory is used to store cached metadata, pointing to where in a VMDK or in a chain of VMDK files disk data to be accessed resides.

In ESX 3.5, the default COW memory heap is 32MB. As more snapshots are used by running virtual machines, particularly by virtual machines with large base disks, this COW memory heap may run low. When exhausted, virtual machines no longer power on.

The following ESX versions can have their COW heap manually increased:

ESX 3.5.0 Update 1 with the patch ESX350-200803202-UG

ESX 3.5.0 Update 2 or higher

ESX 4.0 or higher

Notes:

You do not automatically have to increase the COW memory heap allocation to its configurable maximum. You can double it to begin with, and see if this resolves the issue, unless you feel the ESX host has a lot of memory to spare.

The ESX host must be rebooted for the changes to take effect.

To increase the COW Memory Heap allocation through VI Client:

Open vCenter/VirtualCenter, and click on the relevant host.

Click the Configurations tab.

Under the Software section click Advanced Settings, and in the dialog that appears, click COW.

Change the value of COW.MaxHeapSizeMB.

To increase the COW Memory Heap Allocation through the Service Console in ESX 3.5:

Ensure that you are logged in as the root user.

Run this command to get the current value:

esxcfg-advcfg -g /COW/MaxHeapSizeMB

Value of MaxHeapSizeMB is 32MB.

Run this command to set the new value:

esxcfg-advcfg -s 64 /COW/MaxHeapSizeMB

Value of MaxHeapSizeMB is 64MB.

Note: ESX 3.5 default COW Memory Heap allocation is 32MB and the maximum is 192 MB.

To increase the COW Heap Memory Allocation through the Service Console in ESX 4.0:

Ensure that you are logged in as the root user.

Run these command to get the current value:

esxcfg-advcfg -g /COW/COWMaxHeapSizeMB

Value of MaxHeapSizeMB is 192 MB.

Run this command to set the new value:

esxcfg-advcfg -s 256 /COW/COWMaxHeapSizeMB

Value of MaxHeapSizeMB is 256 MB.

Note: The maximum COW Memory Heap allocation is 256MB. For more information on COW Heap Value, see Setting COW heap values on Lab Manager (1009144).

For additional information, see VMware Community Out of COW space.

See also, Copy on Write (COW) Heap limitations for ESXi 4.1, and 5.x(2036522).

Detectable by VMware SkylineTM

Actions

Copy

link to clipboard
copied!

Print

Language:
English

Attachments

Additional Resources

KB
•
End Of General Availability of the Free vSphere Hypervisor (ESXi 7.x and 8.x) (2107518)

Result 1 of 1

Ask The Community

Get answers quickly from VMware experts in the community

Post Subject

CONTINUE IN COMMUNITIES

Related Products:

VMware vSphere ESXi

VMware ESX Server

Related Versions:

VMware vSphere ESXi 5.5

VMware vSphere ESXi 5.1

VMware vSphere ESXi 5.0

VMware ESXi 4.0.x Installable

VMware ESXi 4.0.x Embedded

VMware ESXi 3.5.x Installable

VMware ESXi 3.5.x Embedded

VMware ESX Server 3.5.x

VMware ESX Server 3.0.x

VMware ESX 4.0.x

view all

Take Our Survey

Was this article helpful?

YES

NO

Copyright © 2005-2024 Broadcom. All Rights Reserved. The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries.

Terms of Use

Your California Privacy Rights

Privacy

Accessibility

Cookie Settings

Support Assistant