Loading

×Sorry to interrupt
Uncaught ReferenceError: addCTracking is not defined
throws at https://kb.vmware.com/resource/KM_TealiumTrackingJS?v=0.5:100:1

Refresh

Cookie Preference Center

General Information on cookies

When you visit our website, we use cookies to ensure that we give you the best experience. This information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies by clicking on the different category headings to find out more and change your settings. However, blocking some types of cookies may impact your experience on the site and the services we are able to offer. Further information can be found in our

Cookie Policy.

Strictly Necessary

Strictly Necessary

Always Active

Strictly Necessary

Strictly necessary cookies are always enabled since they are essential for our website to function. They enable core functionality such as security, network management, and website accessibility. You can set your browser to block or alert you about these cookies, but this may affect how the website functions. For more information please visit www.aboutcookies.org or www.allaboutcookies.org.

Cookie Details‎

Performance

Performance

Performance

Performance cookies are used to analyze the user experience to improve our website by collecting and reporting information on how you use it. They allow us to know which pages are the most and least popular, see how visitors move around the site, optimize our website and make it easier to navigate.

Cookie Details‎

Please confirm your settings by refreshing the page.

Functional

Functional

Functional

Functional cookies help us keep track of your past browsing choices so we can improve usability and customize your experience. These cookies enable the website to remember your preferred settings, language preferences, location and other customizable elements such as font or text size. If you do not allow these cookies, then some or all of these services may not function properly.

Cookie Details‎

Advertising

Advertising

Advertising

Advertising cookies are used to send you relevant advertising and promotional information. They may be set through our site by third parties to build a profile of your interests and show you relevant advertisements on other sites. These cookies do not directly store personal information, but their function is based on uniquely identifying your browser and internet device.

Cookie Details‎

Advertising Cookies

Consent Leg.Interest

Select All Vendors

Select All Vendors

Select All Hosts

Select All

Clear Filters

Information storage and access

Apply

Confirm My Choices Reject All Allow All

My Subscriptions

Loading

Knowledge Base

VMware PKS cluster creation fails with an error: "failed to start all system specs after 1200 with exit code" (67233)

Last Updated: 3/5/2019Categories: InformationalTotal Views: 3134Language:
English

Subscribe to get an email
newsletter with new updates

Please provide article feedback:

Symptoms

PKS cluster creation failed with kube-dns errors.

In the bosh task xx --debug logs, you see the entries similar to:

deployment.extensions/kube-dns created
Waiting for deployment \"kube-dns\" rollout to finish: 0 of 1 updated replicas are available...
failed to start all system specs after 1200 with exit code 1\n","stderr":"error: deployment \"kube-dns\" exceeded its progress deadline\n","logs":{"blobstore_id":"7bf95a81-a6ce-4149-5111-4e9677ea405b","sha1":"e43a300328d23ab2395c4a36e75ee237b8f2de00"}}

Cause

While creating or upgrading clusters on PKS, BOSH runs apply-addons errand which deploys a bunch of add-ons and kube-dns. The kube-dns job rolls out a kube-dns deployment object at Kubernetes level. The kube-dns deployment fails if there is an error or misconfiguration in the underlying overlay networking solution.

Resolution

To resolve this issue, verify the below options. Please note that these are the possible but not limited options,  which cause the cluster creation failure. You need to check the logs and troubleshoot further.

Validate that there are no conflicting IP ranges present under IP Pools, when you go to Inventory from NSX-T Manger

Ensure that you have not used underscore in --external-hostname parameter when you run pks create cluster command.

Validate the NSX-T components health status from NSX-T Manager and confirm all the components are in healthy state.

Validate that all components are set for the NSX-T required configuration MTU size of 1600, including the DVS switches and physical network.

Verifiy that hyperbus interface exists on ESXi hosts by running the command:
esxcfg-vmknic -l

Verify the hyperbus agent health status on the ESXi host by following:

Connect to ESXi as root and run nsxcli to switch to nsxcli.

Run get hyperbus connection info to check the hyperbus health status.

Verify that you have selected the Enable Post Deploy Scripts option under Bosh tile, which will run some scripts to deploy K8S add-ons.

Log in to K8S master & worker nodes and confirm that all services are running:

Run bosh vms to get the deployment and node vm details.

Run bosh ssh <vm-name> -d <deployment-name> to connect to master/worker node.

Run sudo -i to switch to sudo mode.

Run monit summary to check the status of all services.

Review the /var/vcap/sys/log/ncp/ncp.stdout.log on master node to see if there are any network connectivity issues.

Review the below logs on worker nodes :

kubelet.*.log available under /var/vcap/sys/log/kubelet.

nsx-node-agent.*.log available under /var/vcap/sys/log/nsx-node-agent.

Related Information

For more information, refer the below Pivotal articles:
How to troubleshoot kube-dns rollout failure when apply addons errand fails to start all system specs after 1200s
PKS-cluster-creation-fails-with-underscore
PKS-cluster-creation-in-NSX-T-fails-conflicting-cidr
PKS-cluster-creation-in-NSX-T-fails-as-all-available-IPs-in-external-SNAT-IP-pools-are-exhausted

Detectable by VMware SkylineTM

Actions

Copy

link to clipboard
copied!

Print

Language:
English

Attachments

Additional Resources

KB
•
End Of General Availability of the Free vSphere Hypervisor (ESXi 7.x and 8.x) (2107518)

Result 1 of 1

Ask The Community

Get answers quickly from VMware experts in the community

Post Subject

CONTINUE IN COMMUNITIES

Related Products:

VMware PKS

Related Versions:

VMware PKS 1.x

Take Our Survey

Support Assistant