Welcome back to vSphere 8 in the real world, and now we're going to discuss high availability.
So HA is basically a way of ensuring business continuity. It's making sure that we can
continue to function in case of a failure. Of course, that depends on how drastic our failure
is. So what we're talking about here is like a host failure. We have a certain host that's failed,
and we want to make sure that we can continue running when this happens.
HA is not DRS. They're two different services, and DRS is a way of balancing the workload
between our hosts. And so we see that a host is getting overloaded, its utilization is getting
to be too high, and we're going to vMotion a VM from one host to another. Now, a pure HA in
its simplest form means that a host failed, and so I need to start that VM on another host.
With DRS, we don't have any downtime. With a very traditional HA configuration,
there's going to be downtime. Hopefully, we can measure it in seconds or maybe like a minute or
so. What will happen is that a host can fail, and then within seconds, the other host in the
cluster realized that this host is down. We cannot communicate with this host. We know what
VMs are on that host, and so now we need to restart those VMs and all the other hosts in our cluster.
This is determined in advance. The hosts have already decided that these VMs are on this host.
If this host goes down, then the VMs start up here. If this host goes down, the VMs start up
here. The reason why it's important to note that is some people have been hesitant in the past.
I don't think that's so much the case anymore because people realize how this works now.
In the past, though, they did not want to have a virtualized vCenter server because if the host
running vCenter were to go down, then how is HA going to run? It's because it's already been
determined. vCenter does not determine if a host is up or down. vCenter does not tell the other
host in the cluster if a host is up or down. They decide this amongst themselves. They figure it
out amongst themselves, and there's different ways they can do that. It used to be back when
we had very simple HA in the older version of ESXi. Basically, there's a heartbeat. If these
other hosts could see that host HA heartbeat, it's online. If they can't, they can't. What
sometimes happens is whatever the network HA was on, that might have an outage, but all the other
networks may be fine. The storage is running. The VMs are running. Everything's fine on this host.
Everything's running well, but the HA heartbeat for that host went down. The other host would say,
oh, that host is down. We need to start its VMs up, but they're already running, and they're
already running on the host they're supposed to be running on, and everything's fine.
Now we also have other mechanisms in place besides that HA heartbeat. What that is,
is we can have a datastore that HA is going to write to. All the hosts can see that datastore,
and all the hosts are writing to that datastore, and they're saying, I'm checking in, I'm checking
in, I'm checking in. What will happen is if the host that's running those VMs,
if HA heartbeat goes down, it has other checks it can do, and it can say right in the datastore,
hey, I'm still here. I'm still running my VMs. My VMs are fine. Everything's good. The other host
will say, okay, we'll leave you alone. We'll let you run your VMs. Or the host can say, you know
what? I'm having a problem, and I'm going to shut down my VMs, and it can write to that datastore.
I'm shutting down my VMs, and the VMs go down, and then the other hosts know, okay, that host is
down. It's having some issues. It has shut down its VMs, and now we're going to start up the VMs
someplace else. We also have a form of HA that's a preemptive or proactive HA. If a host starts
reporting problems, and often in event logs, the host is like, wow, problem, problem, problem,
problem. All of a sudden, I'm getting errors on my power supply. I'm getting errors for my CPU.
I'm getting errors in RAM. Something significant has happened, and this host is no longer considered
to be trustworthy, stable, reliable, and we can preemptively HA, but in that case, we work with
DRS to move the VMs off of that host, right? So, we'll re-motion them off in that scenario,
but if something happens and the host just goes down, that means it's not writing to that
datastore anymore. There's no HA heartbeats, and nothing's happening to let us know that that host
is in any way still active and running those VMs. Then the other hosts in the environment have
already determined if host A goes down, VM1 goes here, VM2 goes there. If host B goes down, then
VM10 goes here, and VM11 goes there, that kind of thing. It's already determined. So, if the host
running vCenter goes down, the other host will know that host is down. vCenter is one of the
VMs on there. We're going to start that VM up like with any other VM, get it up and going again,
all right? So, you don't need to worry about using a virtual appliance for vCenter and HA,
because vCenter is not involved in figuring if an HA event has occurred or not. It's not involved
in starting the VMs up someplace else. That was a concern for people for a while in the earlier days,
like the 3, 5, and 4, and things like that. But even then, vCenter was not involved,
because people didn't realize that, right? So, they always wanted a physical vCenter server.
They wanted a physical Windows server running vCenter for reasons like that. And they're like,
what if we don't know where it's at, which it didn't matter where it was at. We knew what host
it was on, that kind of thing. Now, vCenter is only a virtual appliance. People understand better
how HA works and things like that, and it's only available as a virtual appliance, okay?
So, vCenter is a way of ensuring business continuity. If a host were to fail,
we can get the VMs up and running very quickly on other hosts. We don't have to wait
for an admin to be notified, and for them to get to a computer and log into the computer and say,
oh, lookit, this server's down. What VMs are down? Oh, I think these ones are down. Let's
start them someplace else. It's automated completely, so VMs are down, hopefully,
only for minutes, not for half an hour, hour, two hours, however long it may be, all right?
So, it keeps downtime to a minimum, and maybe with this preemptive HA to not at all,
no downtime at all. If a server just all of a sudden blows up and catches fire out of nowhere,
there's going to be downtime, right? But again, that downtime should be minimal,
should be minutes while the other hosts turn up those VMs, okay? So, that's what HA's all about.
That's what, you know, is meant by that business continuity, okay? So, high availability,
you know, hardware backups, software backups, interoperability, administration control. So,
admission control, what that means is that if I set up my HA cluster, and I'll show you how to do
that, if I show up my HA cluster, I can say, I want to be able to survive one host failing,
or I won't be able to survive two hosts failing, I won't be able to survive three hosts failing,
right? If I have a large cluster of 30 hosts in there, maybe I want to be able to survive if
three hosts were to fail on me, right? That's kind of a lot, but in a big cluster, that's,
you know, a reasonable number. So, what will happen is, as hosts power on in that cluster,
the, you know, things like reservations and things like that are all taken into consideration, and,
you know, CPU size, what's being guaranteed, and, you know, what's necessary to run the VMs,
is all looked at, and it says, okay, if a host were to fail, can I still run all the VMs? I power
on another one, another one, another one, and as I'm powering on VMs, it's calculating every time
I power on a VM, if a host were to fail, can I power on all the VMs? Or, if my number is set to
two, if two hosts were to fail, can I restart all those VMs someplace else? And eventually,
if you keep powering on VMs, eventually you'll get to a point where HA can't guarantee if a host
fails, or two hosts fail, whatever your limit is set to, it can be a percentage as well. If,
you know, 5% of my hosts were to fail, can I start up all the VMs that were on those hosts
that failed? And if I can't, then what happens? You can tell HA that you want to still power on
the VMs anyway, but you want to be warned, or just, nope, hard limit, we're not going to allow
VMs to power on that are going to exceed my HA threshold, meaning if, you know, those two hosts
failed, I can't start up all their VMs again, so I'm not going to power on any more VMs. It's going
to put a hard limit on that, so that you will be able to start everything up, and you might be
heavily utilized on those hosts, but I can power everything up. So, the downside is, you don't want
to set it to 50%, especially if you have a large cluster, and if you only have two hosts, then 50%,
one host, same thing, right? But if I have 30 hosts, and I set it to 50% is my threshold,
that means I can utilize 15. The VMs can be on all 30, but really, I'm only running enough workloads,
really, to accommodate, you know, the need 15 hosts. I'm wasting half of my resources then,
right? I have way too much, probably, in reserves, so that's something to consider, okay?
So, data protection. So, one of the things about business continuity, obviously, is you want to
make sure your VMs are backed up and some things like that. High availability is a piece of data
business continuity. Backing up your data is a piece of business continuity. Disaster recovery.
Now, people sometimes think, well, that's kind of the same thing, right? And it's really not.
DR and backup are two different things. Backup is meant for me to be able to restore an entire VM
back into my environment, and if there's corruption on that VM, go back in time. I need to have
versions of VMs, versions of files that I can go back to. That's the point of backup. Backup is,
oh, look, this got corrupted, and it looks like, you know, it got corrupted a week ago. I want to
restore that from before it got corrupted and start, you know, from that point, you can start
going forward, whereas DR doesn't necessarily give you all that version history, right?
Version history is backup. DR is every time a change is made to a VM, I'm going to replicate
that change. I'm going to replicate that change. I'm going to replicate that change. Some DR
solutions will allow you to, you know, start a VM in your DR environment at a certain point in time.
It'll keep, like, so many seconds of, you know, versions of VMs, but that's not really what
applications are about. When you look at a DR solution, you're not seeing versions. You're
seeing how many seconds out of date is this copy of the VM from the production one that's running,
and if you can keep that, you know, within, like, 30 seconds or so, that is fantastic. If you can
get below 30 seconds, that's phenomenal. It's great. It means you have huge bandwidth between
them and things like that. So, what a DR solution is going to do, disaster recovery,
is this production environment failed, right? And it's down. The building caught fire,
and I need to go to my DR environment, which is probably going to be a different building,
maybe in a different region, different state, and I need to power everything up, and now run
all of everything that was here over here, and a backup solution will restore a VM for you
exactly how it was. So, my address was 192.168.5.20 on that VM. Backup is going to restore
it, and I'll have the same IP address. A DR solution might go in and say, well, over there,
you were 192.168.5.20, but over here, we have a different IP scheme. So, over here, I'm going to
IP this one at 172.16.5.20 or something like that. We have a different IP scheme over here,
and so part of DR is going to be bringing up those VMs and re-IPing them in this new environment.
That's a DR solution. So, Veeam is an example of a data protection of backup solution, right?
There's lots of them out there. This is an example of one. Zerto is not a backup solution. Zerto is
a DR solution. So, when you look at Zerto, you don't see that you have, you know, 10 different
copies, and you have, you know, one copy of this VM, one backup of this VM, you know, for the last
day, and one from two weeks ago, and one from six months ago. I see that this is just in synchronized
from the production to my Zerto environment, and we're within 45 seconds, right? And that number
might go down. It might go up, but, you know, it'll tell you how in sync it is from production.
That's disaster recovery. That's something like what Zerto would do versus Veeam. That would just
make a copy, and then make another copy, and keep old copies, and say, okay, now I've got so many
copies and start deleting older ones. That's what backup is. That's data protection. So, they're
very different things. People sometimes will kind of talk about them as the same, and they're
definitely not. They're very different functions, different purposes, different use cases, all right?
So, ensuring business continuity with high availability,
vMotion is possible with a, you know, with a predictive or preemptive HA solution.
Replication, so replicating, you know, VMs from one place to another. So, maybe our SAN does
replication for us. Zerto also does replication for you, things like that, and high availability
solutions, right? High availability. A host failed, and I want to start that VM up on other hosts in
that environment. So, HA isn't really a DR solution. So, I'm not saying this entire cluster
failed, the entire data center running these VMs failed. I'm saying a host failed in a cluster,
and all, there's plenty of other resources that are very healthy in a healthy environment
to start that VM. I'm going to do a disaster recovery if all the hosts in my cluster failed,
and maybe that happened because there was like a power outage or something significant, and,
you know, it's going to run for so long that the batteries can't keep up. Now, I'm going to DR it.
I'm going to fail it over to this other environment over here that still has power, okay?
Data protection is also a beast for data protection, for instance, replication and things like that.
You're probably going to use some kind of third-party solution for that.
So, an HA cluster. So, you need to have at least two hosts. Really, you want more than two. One of
the things that's important that VMware really is great about evangelizing is when you're dealing
with your environment, you want whatever you, you know, whatever resources you need, plus,
plus, in case of a failure, plus one, right? So, they don't really just say you want to have
N plus one. So, you like, you know, you need three hosts plus one more for a disaster.
They want, you want to have N plus one, and they'll say, you know, plus two or N plus one,
plus one, and why would you want that? Because maybe I'm patching a host, and while I'm patching
a host, that host is in maintenance mode. That host is no longer available. You still want your
N plus one capability, even if a host is in maintenance mode and being patched. So, if
another host that isn't in that, you know, the one being patched were to have a failure,
a process error to fail or something like that, you want the VMs on that host to be able to be
started, right? So, that, we're that N plus one, plus one, because one is for maintenance mode,
for, you know, patching and updating, plus you still have your failure. They really talk about
that a lot, evangelizing that a lot when we talk about vSAN, because that's the actual data,
right? I mean, if you lose multiple copies of VM, it's just gone forever. The zeros are just,
you know, in the air and gone forever. But it's not only true for vSAN. Again,
they talk about that a lot with vSAN, and it's, you know, for very good reason.
But they also talk about that in other environments. So, if you're, you know,
what you need is an N plus one or an N plus three, then you want actually N plus four. Or, you know,
if you have an N plus one, you want, you actually want N plus two, because you want whatever your
company's standard is for recovery. You know, if that happens to be an N plus two,
you, well, if I'm patching, I have a host down for patching, I no longer have N plus two. So,
now I need to add another one and actually be an N plus three, if my company standard is N plus two,
right? So, that's just something to consider, right? So, let's say you need to have at least
two hosts. You really, you want three, four, five hosts kind of thing. Small environments,
I see it all the time where they only have two or three hosts, and if they lose two,
meaning one's patching and one fails, they're going to not have all their VMs running for a
little while. And the idea is there that eventually that host is going to be done patching within 20,
30 minutes probably, comes back up, hopefully it's healthy, and then we can start up all the
other VMs. So, that's something that you, with your business, have to take in consideration and decide
what your acceptable risk factor is for things like that, right? So, we need to have at least
two hosts because, again, if one fails, the other one needs to run the VMs. We can't over-allocate
resources in scenarios like that. So, one host has two VMs running. The other host will have
no VMs on it because, again, we perform a failover. VMs running application that clients
are accessing. So, what happens in this scenario is that the VMs are running, the clients are
accessing the VMs, and there's resources being used on that. So, how can you protect that process
on that VM if the host is showing signs that it may fail soon? Well, that is when predictive HA
is going to come into effect and it can just move things over for us. So, then we don't see downtime.
If we see that we're going to have hosts having problems and we can predict it,
we can then do a preemptive HA, right? That's one possible solution.
So, here we're looking at, we have our ESXi host 2, we have EFX host 1 with a VM on it.
They're in an HA cluster. Host 1 were to fail, the VM is going to, and this is really important,
the VM restarts on the other host. Sometimes, you know, I've had people ask, so, if I have HA,
then a VM motions the VM to the other host. In traditional, very basic HA, no, because the host
failed. The host failed, the host caught on fire, it's no longer serving resources,
and it was unexpected. That VM isn't running to vMotion over. It's going to be restarted
on that new host, okay? So, was there downtime? Yes, there was downtime. Was the downtime measured
in hours? No, it was probably measured in single-digit minutes, you know, three,
four, five minutes, maybe at the most, maybe less than that, depending upon
how quickly the HA cluster, you know, realized the downtime and how large was the VM that needed
to be booted up and services to start and things like that. But it can be a very short amount
of downtime. So, active, passive, and witness nodes. So, with HA, really, you need three nodes
to make it work. So, how can I do it with two? I can have my active passes, and I can have what's
called a witness node. And that witness node could actually be an ESXi host, for example,
in another cluster, right? They can both talk and communicate to it. And so, because otherwise,
they'd only have the two. And let's just say neither one are down, but there is a breaking
communication between them. Well, they both think they're running, right? So, they need a witness
to say, do you see me? Yes, I see you. The other one, do you see me? Do you see me? And when it
doesn't respond, oh, it's me that's down. It's my network that's down. That's why I can't see
the other one. And again, we do have things like data stores and things like that as well.
If I had a larger cluster, let's say I had three nodes in my cluster, and one would lose
communication, the other two will ask, do you see me? And they both say, yeah, I see you.
Okay, we know we're good. Can we see him? Nope, he's the one that's down. We need to look at his
VMs and get his VMs up and running. Or the one that's isolated will ask, can you see me? Can
you see me? No response. It's me. All right. And then what do you do? That's an option you can
configure, and when you configure HA, is do I shut down my VMs, or do I leave them running?
This is a host isolation incident. And probably most of the time, you're going to say, shut down
my VMs. This is a host isolation incident. Shut down my VMs so the other host can start them up.
It's possible that the problem is just on the HA network, and the other networks are running fine,
the VMs are running fine. But it's also very possible that they're not, and how do you know?
So to be safe, shut down the VMs, let them start someplace else. You will have a little bit of
downtime. You're probably going to get some help. That's tickets really fast. But before the tickets
can even be addressed, the host should have those VMs restarted, okay? So a witness node is just
simply a node that isn't part of the HA cluster that will, you know, basically identify who it
can see, who it can't see if there's some kind of host isolation response, okay?
So ActiveNode runs with a vCenter server instance, right? Has a public-facing IP address,
links to the management interface that the hosts are running on, used to reenact the passive and
the witness nodes, okay? That's an ActiveNode. PassiveNode starts as a clone of the ActiveNode,
right? Updates and synchronizes with the ActiveNode, so it exchanges information in there.
ActiveNode fails, so the passive node can replace and become the ActiveNode.
And then a witness node starts as a clone with the ActiveNode and prevents a split-brain. It
doesn't do any of the other work other than it's just witness, CFIC, you guys are there, all right?
So here we're just looking at, you know, we have our HA interfaces, management interfaces. So for
the HA interface, they all can see and talk to each other. And then we have a management interface
that is there for, you know, managing, you know, communications, connecting to vCenter,
and things like that. So primary host, right, is our manager. And it can, you know, do things like
monitor the state of the VMs, that kind of stuff, determines who's who in the cluster,
and helps communicate, like, what's going to happen in case of an incident response,
that kind of stuff. The secondary nodes are just running the VMs, right? They're running the VMs,
they're monitoring the state of their host, what's going on, and they'll report any updates
if anything changes to the primary. Admission control, we kind of talked about that.
So you want to make sure if you're running HA that you have enough resources available
so that if there is a failure that you can start all the VMs. So you will say you won't allow,
you can say that you won't allow more VMs to start, then the cluster can run and meet all
the reservations and things like that in case of a failure of how many hosts? X number of hosts.
You can determine what happens there, right? And then what happens, you know, are you going to
start the VMs or not start them? And then what happens when there is a failure, right? You can
control, you know, whether VMs are shut down and running and that kind of thing, right?
So you can say that you want to basically keep reserved a certain amount of resource
in your cluster for HA recovery, all right? So admission control often determined as a
percentage of a resource. So you could say in my cluster I have 30 hosts and I want to be able to
handle, you know, if 10 percent of my cluster were to fail, 10 percent of your cluster, that's three
hosts out of 30, right? You can also say that you want to be able to specify a number of hosts
failing. So I can say I want to be able to handle two hosts failing. You can say two hosts or you
can say three hosts or whatever number you want. And the reason why people sometimes use a
percentage is because they can then add and remove hosts which are resources from the cluster. And
the number of hosts that can fail is going to dynamically, you know, move with the number of
hosts in that cluster. If I were to say I want to be able to survive three and then over time I take
hosts out of the cluster, all of a sudden three might be way too many. But if I said 10 percent
and I take hosts out of the cluster, when I'm down to 20, now it's only two, right? So using
a percentage like that makes it easier to add and remove hosts. And if that's something that you do
in your environment often, then that's a useful configuration. If you have a very static
configuration, I have these 15 hosts, it's always going to be this 15 hosts. It would be like a
significant, you know, change to our environment. It's very unusual. If we added the 16th or took
away the 15th and only had 14, that kind of thing, then, you know, then maybe you might
want to say the number of hosts. But most people would go with a percentage. That's just common,
all right? So interoperability. So HA does work with other services like vSAN, for example.
It can work with DRS. So with DRS, if we have preemptive HA. With vSAN, what happens,
because vSAN keeps multiple copies of VMs on different hosts, right? So the host that's running
the VM always has a copy of that VM. And if I'm running on an N plus one, that means the host
running the VM has a copy of that VM on its, you know, disk on vSAN, and another host has a copy.
Well, if the host that's running this VM fails, now I only have one copy of that VM. So I might
start up that VM on the host that is running that VM, but I'm going to let vSAN know that we
need to make another copy. So HA is going to let vSAN know, okay, we're going to start up the VM
on the host with the other copy, but we're also going to make another copy. vSAN will say,
make another copy of that VM someplace else so we can get back to our N plus one, right?
If we're doing an N plus two, then I'll have two places to pick from to start up that VM,
and, you know, then I'll start up and create, again, another copy to replace the one that
failed. So it does work with vSAN in communicating, one, where copies of the VMs live, as well,
making sure we have copies of VMs being made. So HA clusters are great. They do reduce downtime.
You don't need to reconfigure a VM if a host fails. So if a host were to go down
and completely fail, I don't have to go to another host, browse the storage, find that
VM on my shared storage, and import the VMX file to get the VM to come in and power it on and find
out which ones are down, that kind of thing. It'll just automate that process and keeps downtime to
a minimum, again, minutes. So HA with DRS. So we can preemptively move VMs around.
What also happens, though, when there's an HA event, we don't look and see, you know,
what host has the most resources, where's the best place. It just starts VMs up on the first host
that's available, right, and then the next, and then the next kind of thing. So what can happen
is we might end up finding that we have, out of our, you know, 30 nodes in our cluster, two or
three hosts are really overworked now because an HA event happened and we started up a bunch of VMs.
Then what will happen is DRS will say, okay, you're done, you did your thing, all the VMs
are running, good job, now let's balance out the cluster. Let's move VMs around using DRS
to balance the cluster out better so if another host were to fail, one of those hosts, we don't
lose so many VMs on that host, and we get better resource utilization, better performance for our
VMs, all that kind of stuff, right? So that's basically how DRS works with HA.
So for an HA cluster, what do we need? A minimum of two hosts, only two hosts and a witness,
but a minimum of two hosts, you probably, you know, want more than that. Persistent IP addresses
on the host, so whatever VM kernel port is being used for this HA needs to have a static IP address,
right? We need a common management network so VM kernel port that all of them can communicate
on the same network, the same IP subnet. VMs to host compatibility across the board. So that means
is that we don't want to have VMs that are using hardware features that the other hosts in the
cluster don't have. If all of a sudden, you know, if one host has a special piece of hardware and
it's running VMs and it fails, then that dependency might stop that VM from running
and functioning properly on another host. We want to make sure we have hardware compatibility
for all the hosts that are in our cluster, okay? VMware tools. So VMware tools are important if
we're doing VM monitoring. HA can, we've been talking about this whole time about monitoring
the host, but also HA can monitor VMs. So if a VM were to stop running, let's say Blue Springs,
well then VMware tools heartbeat is going to stop too. And the host can say, oh, this VM
has failed. VMware tools aren't responding. I don't see any heartbeat on the VMware tools for
this VM. And then it'll start that VM up again, restart it on a different host. And then different
hosts is just important because what if whoever caused this problem was host specific? And the
reason why it failed was something was wrong is going on with this host. I don't want to start
up on the same host again, have the same problem again. So I started up on a different host just
to make sure it's not a host specific problem. So VMware tools are important for VM monitoring
with HA. Supporting IP protocols, okay? You need to have IP configured. So IP4, IP version 6,
depending on what we're using, we need to have supported versions of IP protocols that all of
our hosts can use. We don't want some hosts on 4 and some hosts on 6 and then they can't talk to
each other. VMPC needs ESXi 6 or later. So we just need to, basically we're talking about we have
vSphere 8, we're going to have them across the board, we're going to be just fine with all of
that. And then you need to have a cluster. You need to create a cluster, put the host in a
cluster, and then enable HA on that cluster. So configuring a cluster. So one of the things you
can do is host failure. How many hosts are we going to allow to have a failure on in my cluster?
It can be a percentage, it can be a number, right? Host isolation. What happens if there's a host
isolation incident, right? So do you want the host to shut down the VMs that are running on it or do
we want to keep running them? Most of the time, you're probably going to say shut down those VMs
to make sure they can run on other hosts that are healthy still, okay? Now, data stores,
if what can cause an HA response could be a data store issue, right? We could say a
permanent device loss, which is a PDL. So a PDL being a permanent device loss, this data store is
just completely gone. How are we going to respond to losing a data store on a host and we can't get
it back but the other hosts still see it, right? Or, and more likely, you know, an all-path-down
scenario is one host can't see but other ones can. And what would cause one host to not be able to
see a data store anymore but another host can still see that data store? Our HBA could have failed.
If we have, you know, special – not special, but, you know, 25-gig NICs that we use for our
iSCSI storage and they're dedicated just to our iSCSI storage, which is ideal, maybe our switch
didn't fail but maybe the iSCSI NIC, you know, failed if that is connected to our data store.
So an all-path-down event is simply just saying that we can't access that data store. All the
paths that we have are down. So maybe we have, like, one card with four ports and that entire
card failed versus, like, a port failing or a cable failing, right? And then VM monitoring. We
can also monitor VMs. So we want to enable VM monitoring and on our cluster or just monitor
the hosts, right? Admission control. So we can specify the number of hosts in a cluster we want
to allow to fail, right? So we have a failover capacity. It can be a percentage of resources.
So slot policy. It used to be that we have what's called slots, right? And basically, a slot is a
VM. How big is a slot? It used to be back in the, you know, four days and things like that, I think
even, like, in 5.5, 5 and 5.5, that a slot was a set size for the entire cluster. And how big was
that slot? Whatever our biggest VM was in our environment, that was our slot size. That was
the size of the VM. So I might have, in my environment, one VM that was a beast that had,
you know, 16 virtual CPUs and it had, you know, 100 gigs of RAM and whatever, you know, big VM,
and all the rest were really small. But to HA, they were all the size of my biggest VM. So when
I enabled admission control, I wasn't really getting the full, you know, size that I wanted to.
Now slot size on the newer versions of ESXi, like in 7 and in 8, slot size is actually just
basically it's dynamic and every VM has its own slot size. And so we can more accurately configure
that. We can also specify a slot size. We say we want all of our VMs to be measured as,
you know, this size CPU and this much RAM, that kind of stuff. Okay. We can have a dedicated
failover host. A dedicated failover host simply means that if a host fails, I want this host over
here to be running and set aside just for failovers. Okay. And I can disable failover
capacity, which means I'm going to let as many VMs run in this HA cluster as the cluster can fit.
And if a host were to fail, I'll start up as many VMs as I can. And when I can't start anymore
because I can't guarantee resources for reservations and things like that, I stop running,
you know, stop starting VMs. That is not obviously a recommended
path because you don't know which of your VMs aren't going to start and you might get some
high priority VMs that don't get started. Right. So you can do a percentage if you want to for
your failover capacity. Obviously, the more hosts that fail, the more utilized your hosts get,
your performance starts to degradate. Right. And are you going to allow performance degradation?
Are you going to over-allocate? If you don't have reservations, then all your VMs are going
to start. You'll be very over-allocated. Right. And that could be a problem.
So, selecting a data store for the heartbeat. So, as I mentioned earlier,
HA now can have a data store for the HA heartbeat for monitoring if, you know,
HA is functioning properly for the hosts or if a host is functioning properly that's in HA.
And so, we can specify what data store or data stores they're going to write to and all the
hosts need to be able to see that same data store and they'll write to it and they'll all monitor
and say, okay, yeah, I see this host is writing, this host is good, that host is good, and they
just monitor that information. Okay. You can also specify a pool of data stores. They all
are going to share the same ones for that HA cluster. Okay. Proactive HA is basically just
going to allow you to, if you see a host or the system sees a host that looks like it's not
performing very well, it's seeing power supply issues, you know, errors from the power supply,
error of CPU, error in RAM, things like that. You know, different components don't look healthy
anymore to the system. It's being logged. What do you want it to do? You can say you want to
quarantine that host, put it in makes mode, quarantine it, or you can say manually you
want VM server to make, you know, suggestions are migrating VMs. You can say you want to
automatically migrate all the VMs off that host and then we can say we want, like, for example,
quarantine the VMs or the host that's on there that's not doing well. So, because you don't,
what you don't want to have happen is you might, HA might see a host and say, oh, this one isn't
healthy. If I just moved the VMs off and didn't do anything else to it, DRS might start putting
them back. So, we're going to put that host in maintenance mode. That way DRS won't start
putting VMs back on it. All right. So, what are the, what can you do with a degraded host?
You can put it in quarantine mode, you can put it in maintenance mode, or you can have a mixture
of different ways that you can say that you want to do it based upon what is the problems you're
seeing. Some troubleshooting. So, common issues, deployment errors, right? So, what can cause some
issues with deploying this? Network latency. Network latency is going to be a big deal for
any of these things. You want to make sure that your networks are healthy, you're not dropping a
lot of packets, your latency is not super high, things like that. A degraded HA cluster. Are you
routing between nodes? And if you are, is it, you know, is routing configured correctly? Again,
network issues, not necessarily latency, but network issues are going to be a problem. Any
kind of network issues with HA are going to be a problem for you because HA requires that
communication between the host to know who's up and who's not, that kind of thing.
Active nodes start serving client requests, right? You want to check the health of your active node
and your passive node and your witness nodes, make sure your nodes are running, make sure your
network's healthy. So, why is my cluster considered degraded? Did a node fail? If you have, you know,
one of your nodes are down, then HA will report a degraded cluster. Also, make sure you have enough
bandwidth between your hosts. If they're all running, they all look, you know, fine,
do they all know that? There's enough bandwidth on the network for them to see each other and
know that they're all fine, they're all healthy. Database replication failed. Again, network
latency. Configuration file not copied from the active and passive nodes. So, check for down
hardware network isolation. Are the two hosts able to see each other and communicate that database
back and forth? Is there enough bandwidth to communicate it, you know, not too high of a
latency? Again, network, network, network, right? Why is my active node not serving client requests?
Is it isolated? Check and see if there's connectivity between them. Not connected to
the active or passive node. What you can try doing there, if everything else looks fine,
is destroy the active node, go into Bash yourself, destroy the active node and recreate it and see
if it joins back. If everything else looks good, everything's healthy, maybe there's some kind of
corruption or something. Just recreating that node can fix that. So, your active node still
functions, but it's not – so, not replaced. And here you'll see there's a command. You can
write the Bash shell to go in and resolve that. That is HA. That is some of the more advanced
features of HA with a vSphere environment. And let's go ahead and go to our demo environment,
and I'll show you how to basically enable HA on a cluster. So, here we are in our vSphere
environment, and I have my datacenter, and here's my production cluster. If I want to configure
HA on this cluster, I can just click on, you know, production here. I can go to my configuration,
go to high availability, and here I'm seeing HA is turned off, right? So, I can say I want to edit
that, and I can say I want to enable HA, and now I have a bunch of different options that we
discussed, right, in here. So, we're going to enable HA. So, host failure response. What do
we want to do? We want to restart the VMs on another host, or do we want to disable, you know,
the host response? Response for host isolation. So, meaning that the host thinks it's up, but,
you know, it doesn't see anybody else. So, that's a host isolation response. We can say power off
and restart the VMs. If we do that, I'll be able to restart them on a different host, right? Or do
you want to shut down and reset the VMs? The difference is power off is going to literally
just take them down, whereas a shutdown will try to shut them down gracefully. The reason why you
might power them down is if you're using, like, say, iSCSI, and you think that if there's an
isolation response, it can't then see the storage. So, just power them off. If you think you still
can see the storage, maybe you have fiber channel HBAs or separate network adapters for your iSCSI
storage, and so you think there's a really good chance that VMs are still running properly and
still seeing storage, then you can shut them down, right? So, VM monitoring. Do you want to
enable VM monitoring? Right now yours are disabled. We can say we want to enable VM monitoring,
right? So, we do enable VM and application monitoring, right? There's some different
options there. VM monitoring sensitivity, right? You can specify, like, you know,
how long before you consider it a failure, some things like that. Admission control.
Host failures in the cluster to tolerate. Here we can say, you know, we want to tolerate one.
We only have two in here, so we really can't say more than that. We can't tolerate two with only,
you know, two in the cluster. We can also say if you want to do a percentage of the cluster or
some things like that. We can specify data stores for our data store heartbeat. We only have the
really one shared data store, so that was the only one we could do. We could also say we wanted to
automatically select it from, you know, the host. So, as you can see, HA is actually really simple
to configure. The first thing is just check that little green button there. It'll turn it on,
and then we can go through and configure all the different options that we've talked about.
So, once I've said that, I can say okay, and it'll enable HA on my cluster. That is HA for vSphere 8.