Welcome back to vSphere 8 in the real world and now we're going to discuss VM migration.
So let's say either planned or unplanned, okay, so you're expecting some kind of maintenance or
maybe it's something unexpected and you need to, for one or more reasons, you need a VM to be on a
different host than what is currently on. Maybe the host that the VM is running on is having issues
and you need to bring down that host. Maybe like all of a sudden you're seeing like a hardware
failure occurring, you're getting errors on a certain device and you want to quickly get the
VM off before that. That would be like a preventative measure or maybe you just need a
patch. So this is totally scheduled. It's that certain day of the month that you guys patch your
EFX by host and so we need to evacuate all the VMs off that host. Maybe just through your
monitoring, you're looking at this and you're saying, well, I'm seeing that this particular
host is heavily utilized. We're looking at CPU utilization in the 80s. We're seeing memory
utilization in the 80s, 90s. We need to redistribute some of these VMs because this other host over
here, its CPU utilization is only in the 20s and its memory utilization is only in the 30s.
So we need to move some VMs off of this heavily utilized host to another one. But it's the middle
of the day. You don't want to stop your workload. You don't want to interrupt your users, right?
You need to patch your host, as we said, and you don't want to have to wait till the middle of the
night to do it when no one's around. You want to do it right now in the middle of the day while
everyone's working, while you're scheduled to be at work, and so you want to evacuate the host.
So all these different scenarios are reasons why you might want to live migrate the VMs from one
host to another. Live migrate is kind of a generic industry term for a vMotion. VMware's branded
version of what they call a live migration is a vMotion. And so what a vMotion is, is simply
we have a VM running on a host, and for one of many different possible reasons, we want to move
that VM from host one over to host two without bringing down the workload. And so we decide we're
going to vMotion that VM's workloads. We're going to move the compute from one host to another.
I'm going to right-click on the VM. I'm going to say I want to migrate, and I'm going to go
through a wizard, which asks me a bunch of questions like what host do I want to move it to,
and the priority I want to use to move it. We'll walk through that, and I want to move it to another
host. So then I choose the second host. VMware's going to run a check and say, yep, you are able
to do a vMotion from host one to host two. And you go through the wizard, and you say okay.
And what VMware starts doing, what the ESXi hosts start doing, is they will start copying the RAM
from one host to the other. But while you're doing that, that VM's using that RAM. It's making
changes. And so it keeps a map of all the pages in RAM, and as a page changes during this copy,
it marks the changes. It marks the pages that have changed. So once you've gone through
all the pages of all the RAMs copied from host one to host two, it then goes through and says,
okay, now what page has changed during this copy process? And it copies just those pages over.
And again, it's keeping track of what pages changed. And then once it gets all those synced
over, it can make another copy. And when the host feels that the number of changes that are left
are small enough that we can quiesce the VM, move those final pages, move the VM over,
and fire it up without there being an obvious interruption in services, then it quiesces the VM
so it does this quick pause, copies those pages, and then it also copies over the CPU registers
that the VM has registered on the CPU, and then it fires it on this host. Now, you might wonder,
well, how long does it quiesce for? Because if I'm running a database, that could mess up my
database. My applications might disconnect. If users are trying to open files, I'm going to get
a bunch of help desk tickets saying, hey, this file server's down. I can't get to the file. Oh,
wait, there it is. But in the meantime, they've made the phone calls. Your help desk phone is
blowing up. Emails are being sent. Any monitoring software you have might be firing off alarms
saying, this server's down, this server's down. Oh, wait, no, it's back now. Actually, it is so fast.
It does it so quickly that if you're pinging from a workstation to that server and you'll see ping,
ping, ping, sometimes you'll see all, if you're going from a Windows box that defaults four,
all four will come across. You won't miss any. If you're in a continuous ping,
you will see ping, ping, ping, all the way down. You won't miss a ping. Sometimes you'll miss one.
Sometimes you'll miss one. Oftentimes, I'd say less than half, but still pretty often,
you won't see any packets dropped. It will actually just be so fast and so seamless.
Because what also happens is the VM, when it lands on the new host,
the host is going to send out a packet from that VM to the network so that the switches know,
this MAC address is now over here on this interface. It was on that interface. It's now
on this interface. So VMware gets the number of pages that have changed down to a bare minimum.
It fires the VM up on the new host. It sends a packet out to the network from that VM so the
switches know where that VM now is. The VM doesn't know what happened. The hosts obviously know,
but they don't care. The users don't know what happened, right? The network knows something
changed. They don't know why it changed, but they know this MAC address was on this interface,
and now it's over here on this interface. Okay, great. No problem. And the application still
runs smoothly. The user's experience is fine. You won't be dropping packets. Nothing. It'll
be perfectly smooth. So it's a really awesome thing. It happens very, very fast. VMware has
perfected what's called a live migration. So zero downtime, live migrations, moves your workload
from one host to another. It will move between different switches, different port groups if you
ask it to. You can actually move between clusters. You can even migrate from one cloud to another.
So if you have a vSphere environment in your data center here, and you have another one that's maybe
like in a different city or state, or maybe it's in Azure, instead of being in your data center,
it's in Azure, you can move workloads up into Azure. All kinds of very cool things. Now,
some of those things require additional services, right? You need a cloud license,
and you need a license for metro clustering and things like that. But the VMs will move
very quickly. It's a very, very cool feature. All right. So here we're showing that you have
a VM running on host one, and you can live migrate it from one host to host two. All right.
DRS will do this for you automatically too. If you have a license for DRS and your DRS is set
to fully automated, that scenario that we gave where this host is heavily utilized, 80% CPU,
90% memory, or something like that, this guy over here, 20% CPU, 30% memory, real low versus real
high, DRS can look at things like that and say, let's balance this out. Let's try and get it a
little bit closer. It's not going to try and get it exactly because it doesn't want to keep moving
workloads constantly. But it will say, okay, we're getting pretty close. And it will actually,
on its own, move a VM from one host to another to try and balance the workload.
You can specify when you configure DRS how aggressive you want that to be.
It's a scale of one to five, and you can say how aggressive or not aggressive you want it to be.
But DRS will do that for you automatically. You'll automatically live migrate VMs from one host to
another. And maybe your host, your cluster has 30 hosts. And it will look at all of them and say,
okay, these guys are heavily utilized. These guys are not. And why would that happen? Why
would you have a cluster where some hosts are really heavily utilized and some are not?
Well, maybe when you started all these up, certain VMs weren't that busy. The databases
weren't that big. They didn't have that many users. Time has changed. Databases have grown.
More users are hitting them. Other applications aren't as busy. It might not even be over a
period of months and years. It might be just within a month. This week is end of month,
and the servers that run all the accounting databases are just getting hammered from all
the reporting. And so we need to spread those out more. And the other servers for another department,
maybe they slow down a certain time of the month, and they don't do as much.
So at the beginning of the month, all of a sudden, we see manufacturing taking off. All
these new orders that sales pushed at the very end of the month, they hit the beginning of the month.
And so those reports are going crazy, and accounting has slowed down a little bit.
So there's going to be different reasons why certain VMs at certain days of the week or
certain days of the month or different times of the year, certain ones are busier. DRS is going
to monitor it constantly, and it can make adjustments to that automatically. Or you
can just be looking at it, and you can say, this host, all of a sudden, is really busier. That
one's not so much. Maybe it's not something you're doing to try and balance workload,
but you're trying to evacuate a host. We want to patch the host. We want to update hardware on
the host, and we don't want to wait to shut VMs down. We want to do it during the day,
during production time. We trust this service. It works well for us, and we're going to go
ahead and take care of it right now. All right? So that's live migration there.
There's also a storage migration. So all those things we just discussed
about with moving to compute, you can also do with storage. So we have a VM running on a certain
data store that's on our stand, for example, and we might see that that LUN is getting really
heavily utilized. The IO is very high. The latency is getting really high, and we need to lower the
latency on that data store. And one way we're going to do that is we're going to move some
VMs to another data store that is less utilized, or maybe it has much, much faster disks, and so
the latency is just going to be much lower anyway. And so we can storage migrate all the files that
make up that VM. We can move them from one data store to another data store doing a storage
migration. And this is a little bit less magical, so to speak, because it's just moving files,
and we're going to tell the host that's running that VM, okay, your files are here, and the host
is going to be moving them over and say, okay, now your files are there. Right? It'll copy files over,
it'll monitor for changed blocks, it'll synchronize those changed blocks, again, quiesce the storage,
and then finish, and now you're living over there. Not a real big surprise that we can do that.
It actually is a newer feature than a regular compute vMotion. A compute vMotion has been
around for a lot longer than storage migrations, but more storage migrations have been around for
quite a while too. So you can also change storage, and we can also change compute and storage at
once. So if we have a license for both a vMotion and a storage vMotion, and let's say I have a
VM that's on local storage on a host. Well, you can't move the compute to a different host because
that host can't see that local storage on the first host. So what I can do is say I want to
move the compute and the storage from host one to local storage on host two. And so then it will
literally just copy all the storage files over, and it'll copy the compute files all over, and now
it lives over there on that new host. So we can do a compute vMotion, we can do a storage vMotion,
or we can choose to do both. So there we're seeing we're just moving the virtual disk, right?
So let's go ahead and jump into our demo environment, and I'll show you how to do a vMotion.
So here we have two ESXi hosts, and we have a VM in the cluster with these two hosts.
So if we go look at this cluster, I can see that this demo VM, which is a Red Hat Linux VM,
is on the host that has the address of .38. So what we can do in our demo is we'll go ahead and
we'll migrate this VM from 38 to 39. So I'm going to go ahead and choose my virtual machine.
I'm going to right-click and migrate, and just so you know, this first time we attempt it,
it's not going to be successful. I'm going to go ahead and click on to change my compute resource,
say next, and if I choose 38, it tells me it's compatible because it's already on 38,
so I'm really not going to move anything. It's fine living there. If I choose 39,
I'm going to get an error that there's incompatibility or there's compatibility
issues that need attention. If I look to see, well, what are those compatibility issues?
In this case, I have a no vMotion interface configured for these two hosts. Now,
other things you might see is that the VM is on storage that one host can see and one host can't
see. You know, things like that could also happen. So if there's a storage problem, it'll tell you
that, you know, host number 39 can't see the storage on host 38 or something like that.
So there's different things that can come up. But in this case, it's that we don't have a
vMotion interface configured. So I'm going to go ahead and close this. I'm going to cancel this.
And ideally, what you'd want to do is have a dedicated vMotion network. So in a perfect
world, we would go to our configuration, and under our virtual switches, I would have a
VMkernel port configured for a vMotion network. And this vMotion network would have a dedicated
VLAN. So maybe it'd be on VLAN, you know, 30 or whatever you've chosen, just for vMotion.
And a lot of times, these hosts are going to be all in the same physical location.
And so I have a dedicated VLAN. And the only thing this VLAN is used for is for vMotion,
and there's no routing happening. So for example, this host, 38, has an address of 192.168.51.1
for this vMotion network. And 39
has a vMkernel port of 192.168.51.2. So they're on the same LAN. They can be on the same VLAN.
So vMotion is not going to be happening through a router, not going to be routing it.
It's just going to go through a Layer 2 switch and connect. So then what I would normally do
is I'm going to go in and edit this network, this port group here. I'm going to say edit settings.
Actually, you know what? I'm sorry. This is the wrong place. I need to go to my vMkernel. My
mistake. So my vMkernel, vmk1, this is the one that is on my vMotion network. This is where I'm
going to go and edit. And I'm just going to just enable vMotion on that kernel. And I would do the
same thing on both of these hosts. I go to the other host, and this is also my vMotion network.
And I'm going to edit this and choose vMotion. Now, in this environment, because this is all
virtualized, the virtual switch that's underlying all this environment isn't passing that traffic
correctly. So this is kind of show you to make this work. What we're going to do is we're going
to use vmk1. Normally, we wouldn't choose vmk1 because vmk1 is my management NIC.
So you already have management configured on here. I would normally have a dedicated
vMkernel port on a dedicated network just for vMotion. But for this demo,
I'm going to go ahead and choose it here. And I'm going to go to this host.
Let's see what's selected here. Make sure that's not there. Go to vmk0, edit, enable vMotion here.
And I wanted to be a little bit extra, you know, sure. Another thing I might want to do
is check my firewall and see that I have vMotion for incoming and vMotion for outgoing configured.
And I do on this host. And for vMotion, it also is configured on this host. That is done by default.
You don't need to configure that. But that's something you might want to be if you're being
really, really thorough, you might want to check. So now that I've checked my firewall, I've got – the
big thing I need to configure is I need to make sure my vMkernel adapters that can communicate
with each other with that vM service selected. So I have vMotion enabled on this vMkernel adapter.
And this vMkernel adapter also has vMotion enabled. So now I can go to this virtual machine.
I'm going to select this. I'm going to say I want to migrate. Same thing. Again, I'm only
going to change the compute resources. Say next. So again, it's on 38. And so, of course, that works.
And if I choose 39, 39 is also successful. So I'm going to say next. And if I wanted to change
the network from that host one is using to something different on host two, I could go in
and choose a different vM network. Maybe I have one for sales configured or something like that.
I'm going to leave it on vM network. We're good there. I can specify the priorities,
a high priority or just normal. I'm going to say high priority. I always just leave it there.
Next. And now I'm going to go ahead and say I want to finish this.
And I'll see a task startup down here. All right. Is that 20%? Once it gets past 2021,
that's a good indication. Not a guarantee, but a good indication is going to go through. That
means that the kernels are talking. So right now what it's doing is copying all that memory over
and it's making a record, a note of any changes to any pages in RAM that are changing. Okay. It
did all that. It synchronized everything over, copied over to CP registers. So now it says it's
completed. So if I go look at 38 and at VMs, that VM's gone. And if I go to 39, I see it's there.
So all I needed to do in this situation is there are some prerequisites. Like, for example,
this VM, if I go to look at storage, this VM is on my shared storage database. If the VM was on
datastore 38, datastore 38 only connected to one host, whereas this datastore where the VM lives
is on both hosts. So I need shared storage, right? And I see there's that demo VM that both the hosts
can connect to. So I need it to be on shared storage. I need to have a shared network. It needs
to be a low-latency network. You can actually do a VMotion over a metro network, so over a wide-area
network. Your latency for round-trip traffic needs to be under 150 milliseconds. So 150 is kind of
high. It's actually a pretty – that's not hard to achieve. So if you have a pretty decent network,
you can do it under – as long as you have under 150-millisecond latency. So shared storage,
a network that both the hosts can communicate on that has VMotion enabled on that interface,
and you've got VMotion configured, and we can move hosts between – or VMs between different hosts.
In this example, we only have two. In the same cluster, there might be 30 hosts. And in those
30 hosts, I can move the VM to any of those hosts, as long as all the hosts in the cluster can see
the storage, and as long as the hosts all have a VMkernel interface configured that can talk to
the other hosts. And that's how you configure and do a VMotion.