Welcome back to vSphere 8 in the real world. Let's discuss vSAN.
So vSAN is a way of, again, VMware wants to software define the entire data center and this is their way of software defining storage.
It also allows us to use local storage in our hosts and treat it as shared storage or a storage area network.
So basically, we take the storage in our hosts and we're going to break those disks that are up in the host into what they call disk groups.
Now, if I only have maybe four disks in my host, I'm probably going to have one disk group.
But if I've got a larger number, if I've got 12 or 16 or 20 or 32, I'll have multiple disk groups.
So let's just say I had 16 disks in all my hosts, then I'll probably have four disk groups of four disks.
That's a pretty common breakout. Then I'll do the same thing on my next host, my next host, my next host, right?
And I want them all to be the same. I don't really want to take a bunch of hosts that are completely different in storage and try to create vSAN with them.
I don't want this host to have, you know, 16, you know, 6-terabyte drives and this guy over here has 8, you know, 2-terabyte drives and mix all that up.
Because what will happen is, what happens if this big guy over here fails, right?
All of a sudden, this one guy failing is the equivalent of three of these guys failing or something like that, right?
So it's really important that we, kind of when it first came out, the idea was, I'm going to take my existing servers and their existing storage and create vSAN.
But as vSAN has grown and matured, you really want to target hardware with the intention of vSAN.
It's not a secondary, oh, let's go ahead and also do this.
We're going to buy the hardware with the idea that we're going to deploy vSAN.
We want, you know, controllers on those servers that will support vSAN.
One, we want them on the VMware, you know, HDL.
But also, we want, for example, controllers that we can say, this is not a RAID controller, this is simply a pass-through.
I want my ESXi host to see all 16 drives in that host.
I don't want to go into my RAID controller card and say, I'm going to take these four and create, you know, a RAID 5 and these four and create a RAID 5.
Or a RAID 1, or whatever I'm doing.
I don't want to do that.
I want to say, no, the RAID controller is not doing RAID.
It's pass-through and send all the disks to the operating system so that ESXi can see all 16 drives.
Because that's how the vSAN is going to want to write virtual machines.
It's going to want to write virtual machines based upon the drive that's in that vSAN.
And then replicate that to other hosts.
Meaning, replicate those VMs, you know, whatever data is on those disks to other hosts.
So, it's not necessarily something you want to piece together with a bunch of odd-sized disks.
One thing that comes up often in discussion of vSAN is, of course, well, I'm going to have so many spinning disks and I'm going to have a cache disk, right?
Which means I'm going to have a flash disk.
That is absolutely true with pretty much all the different versions of vSAN, except vSER 8.
That's not required.
I can actually do an all-flash vSAN and not have a cache disk.
With older versions, like, you know, vSphere 7, even if all my disks were flash, I still needed a caching flash disk also.
That's not true with 8.
So, I can actually use that disk as an additional disk for what's called capacity.
There was capacity drives and then there was cache drives, and the cache drives had to be flash.
My capacity drives could be flash, they could be spinning disk, they could be, you know, SATA, they could be SAS, they could be, you know, a bunch of different things.
But that caching disk needed to be a flash disk.
With vSphere 8, that's not necessary.
I can do, if I choose to, an all-flash, all-capacity vSAN deployment.
For most of the conversations we'll have, we'll talk about capacity disks and cache, and we'll discuss, like, you know, if you have spinning disks and cache and things like that.
Just realize that there is that possibility to do all-flash without the cache, and it's a newer feature in 8, and it's something that you're going to maybe start seeing more and more of.
But if you're upgrading from an older version of vSAN, then unless you want to completely just wipe it out and start over again, you're going to have, even if you have all-flash, you're going to have a cache disk and you're going to have a capacity disk.
All right?
So, tied to hybrid, so hybrid means you have flash, you know, for your memory, you have magnetic drives, you can do all-flash.
So, managing vSAN, you can create storage policies and you can use your storage policies, like we learned in a previous lesson, to manage vSAN, as well as you would with a more traditional physical storage array that you bought from some enterprise storage vendor.
So, some important concepts.
One of the most important ones are disk groups.
So, as we mentioned, you are going to take the disks that are in your host and you're going to want to break them up into disk groups, because that is how you're going to look at things like, okay, we need to recover, or we need to do some patching.
We're going to need to be able to recover a disk group, or we're going to need to be able to, if we're going to do upgrading, we need to have enough capacity to take a disk group offline and not overwhelm the rest of the disk groups.
Well, if I create one large disk group with all 16 of my drives, then if I want to patch that host, I've got to patch that entire host, all the disk groups at one time, which means I've got to move all that data to another host.
So, if I only have three nodes, in order to take this guy down, the other two have to be able to handle all of its data and still function, not be overloaded and overwhelmed.
If I were to instead break those 16 drives up into groups of four and four disk groups, and I want to start upgrading and patching, I only have to have enough capacity to upgrade one disk group at a time.
So, I have to be able to move the data from those four drives to the other host, not all 16 drives.
So, for upgrading, for patching, things like that, disk groups are important.
Backup and recovery disk groups are important.
A lot of disk groups do have a downside.
The downside being that it does take more resources to manage, and there's going to be more overhead.
Within a disk group, you're going to have some fault tolerance in there, so you're going to have more overhead as far as what capacity you might have lost.
So, there is downsides.
There's a lot of overhead to additional disk groups, but it does make for the most flexible deployment.
It makes for the most recoverable deployment, the easiest to patch, the easiest to upgrade.
You don't want to put yourself into a situation where I can't patch, I can't upgrade, I can't do anything because my disk groups are so big that I can't offload them anywhere else in my infrastructure.
Okay?
So, disk groups are just literally physical grouping of physical disks, all right, or I should say loss of grouping of physical disks.
So, a data store, once you have your vSAN configured, then your vSAN is going to present to your ESXi host as that data store.
And we discussed data stores in previous lessons where I had, you know, a LUN on a SAN presented, and I took that LUN and I created a partition and I formatted it as vMFS, all right?
In this situation, it's going to present itself as vSAN, all right?
Consumed capacity.
So, we're going to talk about, when we discuss vSAN, what is its actually consumable capacity?
I might say, okay, I've got, just trying to make some easy math here, I've got, you know, 10 drives and they are a terabyte each in this box.
So, that's, you know, 10 terabytes.
And I've got three hosts that are exactly the same, so that means I've got 30 terabytes.
But how much space do I have to use?
What's my consumable space in this cluster?
Because, for one thing, I want to have an N plus one.
So, I won't be able to afford to lose a host and my system keeps going.
So, that means that whenever data gets written to one of these guys, I'm going to have to have a copy of it someplace else, right?
So, right there, I've already lost, you know, a third of the space of my storage array if I'm doing an N plus one and I have three hosts.
Now, if I have eight hosts, I only lost 12.5%, right?
I lost a much smaller number.
So, the more hosts you have, then the smaller your overhead is for that N plus one.
The less you have, the bigger percentage it becomes, obviously.
There's also going to be some other things you're going to want to take into consideration.
So, you're going to not only are you going to take into consideration your fault tolerance,
but you also need to have space available for things like, you know, again, upgrading and patching and moving things around.
So, there's going to be maintenance involved on the hosts themselves that they kind of need to do.
They need to have reserved space for that, and we'll discuss that in a little bit.
So, your consumable space, though, the thing to note, is not going to be necessarily the size of your disk times how many disks times how many hosts
because there's a lot of overhead involved.
So, just like if you have just a traditional server and you have 10 disks in there that are a terabyte each,
10 terabytes, well, if you do a RAID 0, you do.
But if you do a RAID 1, you're going to have half of that, right?
If you do a RAID 5, then you'll have 90% of that.
If you do a RAID 6, you'll have 80% of that.
So, you have overhead with pretty much any way you do storage if you want any kind of fault tolerance and redundancy in that.
And vSAN is the same.
vSAN is just going to consume disks a lot in different ways.
So, the vSAN is object-based storage, right?
So, there's, you know, block-based and there's object-based.
It's object-based storage.
This means it focuses on files.
When you are looking at it and you're managing it, that's why you always kind of see it, but it manages things at a file level.
And so, a file is going to live someplace, whereas just like blocks live in some place.
So, your VM home namespace.
So, you're going to be a folder created for every virtual machine.
And in that virtual machine, you're going to have your VMX file, which is going to be the file that actually defines what the VM is.
And that VMX file is going to say how many virtual CPUs it has, how much RAM it has, is the virtual CPUs, you know, multi-core, how many cores they have.
Where are your VMDK files?
In that same folder are your VMDK files.
So, the virtual machine has VMDKs, which are those virtual disks where all the data for the VM is stored.
And the VMX file says this is where the VMDK files are, right?
This has all the defining features.
You know, do you have a CD-ROM drive?
Is that CD-ROM drive connected to an ISO?
Do you have a sound card?
What kind of disk controller are you using?
You know, which one did you say you wanted to virtualize in this host or this virtual machine?
All that stuff is defined in the VMX.
Your log files are in that home directory.
Your VMDK files are in that home directory.
VMDK, as we just said, that's the file that basically stores all the data that would be in the virtual hard drive.
And then you have swap.
And so, when you turn on a VM, it creates swap, right?
And that swap is going to be created when you turn it on, and it gets deleted when you turn it off.
Now, I've seen it before where data starts to get really full.
And you have some bigger VMs.
They have lots of RAM.
And you go to power it on.
It says, I can't power on the VMs.
I can't create the swap.
There's not enough space to create your swap, and so it can't power on the VM.
So, make sure that when you're planning how much space do I need, you're not just saying, okay, well, I need five VMs,
and they each need a terabyte of space.
Each one needs five terabytes.
And so, I only allocate five terabytes.
First of all, there are log files.
There is the VMX file.
There's, you know, all these different things like that.
And, you know, besides the VMDK.
So, already you're out of space.
Then you go to create power on that first VM, and that first VM may not power on because you don't have enough space to create the
first VM swap file, right?
So, just be careful of that.
Make sure you're planning.
Whatever you're thinking you need for your VMDK files, you want to add, you know, maybe a 30% on top of that number to make
sure you have enough room for everything.
Those are requirements.
So, again, a caching disk, generally speaking, is considered required.
You can do an all flash, though, with that being said.
Traditionally, we can look at vSAN, we need a caching disk, and we need capacity disk, right?
So, for our caching disk, we need a flash disk that will be used just for capacity.
That is our hybrid kind of deployment where we have a flash disk for our caching disk, and then we have capacity drives.
And those can be spinning disks, they can be flash disks, they can be, you know, pretty much whatever you have in that box.
You want it to be, you know, of course, on the HCL and all that kind of stuff, right?
So, VM data storage or VM data in a hybrid configuration.
So, basically what happens is you create a VM, it's going to live on a disk, right, on a spinning disk.
And then you'll be replicated to other systems in that vSAN deployment, all right?
All flash, you have what's called a disk group, and basically that's going to equal a SSD drive or a flash drive.
So, for controllers, you're going to get at least one HBA, right?
Or it can be a RAID controller, but the RAID controller needs to be in pass-through mode.
So, that way the OS sees the disks themselves and doesn't see RAID groups, okay?
You don't want to start mixing different controller types and modes and this and that on a host or cross host.
You want to have the same controller in all the hosts performing the same way, all the disks in all the hosts configured the same way.
You don't want to start mixing things around because you want to be able to predict what's going to happen in case of a failure.
You want to predict what's going to happen when you upgrade a disk group, how is the rest of the cluster going to perform.
If it's really like a bunch of random sizes and random configurations, you can't predict very easily what's going to happen, all right?
So, for flash, for your host, if you have over 512 gigabytes of RAM in that system, which is pretty common.
I'm not saying everybody does or all of them do, but it's not an uncommon thing anymore, right?
And you have external memory that's less than or equal to 4 gigabytes.
So, that's kind of a weird configuration there.
Basically, you can do a boot from external USB.
It's really not suggested anymore.
VMware really kind of stopped supporting back in, I think it was like 7, update 2 or update 3,
where they did support booting up from either little SD cards or from USB drives and things like that.
They have stopped supporting that.
They do not want you using SD drives or USB drives and things like that.
The most common way that you see is a point now when you have local storage is that they will have a controller card in there
that has some little NVMe drives on it, and you'll boot from an NVMe drive.
You can get a small NVMe.
Maybe it's 256 gigs.
I've seen 512 a lot, which is a huge waste.
And then just basically just mirror those, right, on that controller card.
That card is basically just there for installing ESXi.
It's a card.
It has two NVMe drives.
They're mirrored to each other, and really all you're doing with that is installing ESXi and booting it,
which is why I say 512 is a waste.
That extra space that's not used on those NVMe drives can be used for ISOs.
It can be used for templates.
There's a purpose for it.
But normally you do put that on the SAN somewhere on just really inexpensive low-end storage if you have that in there.
But you do see it different ways.
And booting to USB and things like that is not really supported in more VMWare.
So SAN design.
So basically we're seeing here we have VMs.
Those VMs live on vSAN, right?
So the hosts are, you know, vSAN.
It's not like you can say vSAN is on this one or vSAN is on that one.
They both are running vSAN.
Data is being replicated.
In a practical environment, you probably have at least three, probably four, five, six, seven, eight hosts or more,
or it can be considerably more if you're running vSAN.
And so those VMs could live on multiple of those.
There will be a copy of it on the host that's running the VM.
You know one copy is there.
And the other ones can be spread around, right?
And they can be spread around based upon how the vSAN algorithm determines they need to be placed.
Those hosts are connected to hard disk drive magnetic storage probably.
That's a pretty common way of doing it.
But it could be all flash as well.
You have that SSD flash drive for your cache, right?
That is your disk group, right?
So every disk group is going to have your flash drive, and it's going to have your capacity drives, right?
Altogether, those disks and all the different disk groups make up your vSAN.
So those VMs living on vSAN really kind of are living on all the hosts in that vSAN cluster
or any of them, and that's like all of them at the same time, but it could be any of them.
So you can kind of look at it like being all of them.
If a host that was running a VM or had a VM's redundant disks on it somewhere were to fail,
vSAN is going to immediately try to make copies of that someplace else so we can keep that redundancy going.
So keep in mind when you're discussing your design, your vSAN, all right?
So you're going to decide things on like the size, capacity,
what kinds of things are we going to take into consideration when we think about capacity.
Remember, I'm not going to say I think I need 5 terabytes, so I'm going to buy 5 terabytes.
I think I need 5 terabytes of storage, so I'm going to buy 7, right?
And that actually would be a tiny, tiny, tiny vSAN, right?
If I needed, you know, in reality I need 50 terabytes of storage, I'm probably going to buy like 60 or 70 at least.
And that is what I'm thinking about.
I'm not going to grow.
If I'm thinking about future growth and capacity and things like that, then I might be looking at more like 100 or 120
because I know I need, you know, for the additional files.
I need it for swap.
I need, you know, additional capacity for powering on and powering off, you know, VMs
and then patching hosts, upgrading hosts, all that kind of stuff, right?
All those things mean additional disk space.
So when you think about capacity, those are the kinds of things you need to consider, right?
So you plan your expectations of consumption around the capacity.
The VMs consume calculation, which, again, that's going to be like it's disk, it's logs, it's swap.
All of that has to be calculated in.
Then, of course, whatever the data store, you know, capacity ends up being.
And then vSAN has its own overhead, right?
vSAN itself has tasks that it runs, and those tasks create overhead also.
So that's why they're saying you want to leave at least like 30% of your vSAN empty, you know,
when you're doing your design.
You want to consider all the different things you need and then add 30% to that.
So you expect to get what you need and reserve additional capacity.
So make space for the VMs, snapshots, swap files, all that kind of stuff.
And, again, maintenance, hosts going down, on and on and on top of that.
So what is your tolerance level going to be?
Are you going to do an N plus one?
So, you know, I need to have, you know, three hosts.
So I'm going to do three plus one more, that's four.
Am I going to do an N plus two, right?
To run all my workloads and all my capacity, I need, you know, three hosts.
I'm going to do an N plus two, that needs five hosts.
So plan for things like that as well.
Think about your tolerance.
Think about your VMs' capacity needs and things like that.
So PCIe or Flash, so for your caching, for your capacity,
what are you going to use for your caching disk?
You're going to have to use Flash for caching if you're doing caching.
For capacity, you could do Flash.
You could do, you know, you could do a spinning disk.
You could do, like, magnetic, you know, spinning disks, things like that.
But you know for your cache, it has to be Flash, right?
For your RAID disk, it can be SAS or Nearline SAS.
Your controller design, you might want multiple controllers in your design,
but if you do have multiple controllers, you want to be in the same model,
same configuration, all of that.
And if your controllers, they need to be in pass-through mode,
not any kind of RAID configuration, okay?
You're not in a RAID, you know, oh, I'm going to make this more redundant.
I'm going to create on my RAID controller,
a RAID 1 or something like that.
You don't want to do that.
You want to pass it through, let the vSAN see all the disks,
and then let it manage the redundancy and things like that.
So consideration for the host, how much memory.
You need VM, you know, memory for the VMs themselves, you know,
for us just to run them.
You need memory for the host.
vSAN takes a lot of additional overhead as well, right?
So you need to take into consideration the vSAN overhead,
as well as the virtual machine overhead,
as well as the ESXi host overhead, all those kinds of things, right?
Host networking.
So here it says, you know, you've got an adapter, of course,
isn't that because we're going to be exchanging information on IP.
One gigabyte minimum.
Really, in today's day and age, 10 gig for vSAN is kind of your minimum.
And what I mean by that is you want a dedicated network for your vSAN.
You want these hosts to be communicating on a dedicated network.
Can you do it on shared NICs?
Can you have VMs running on the same NICs that you are, you know,
replicating and managing vSAN on?
You can.
You should not.
Okay, you absolutely should not.
You should have dedicated NICs.
They should be at least 10 gig NICs in reality.
25 gig NICs are really, really common now.
It's not that uncommon to see, you know,
enterprise switches with several 25 gig interfaces, cards and servers,
25 gig.
I mean, it's not that uncommon in the enterprise to see, you know,
40, 50, 100 gig all the time, like in enterprise data centers.
Recently I've seen some demonstrations from Cisco where they're showing
their 200 gig interfaces.
For your vSAN deployments in most situations, you want 10 gig minimum,
probably at least, you know, 25 gig.
It's going to depend on your budget.
Twenty-five gig is more expensive.
It definitely is a cost difference there,
but it's not as drastic as it was just, you know,
three or four years ago even.
So consider really minimum probably, you know,
a 10 gig connection between your ESXi host dedicated just for vSAN.
And you want that to be redundant, too.
You don't want just one.
You want at least two.
So multiple disk groups, there are pros to multiple disk groups, right?
That is improved performance, you know, better failover capabilities.
Recovery times are lower.
Cons is it consumes more memory.
Controller increase, you know,
so you may have additional controllers for additional redundancy with your
disk groups, things like that.
You need more drive bays because you have more disks.
If you had fewer disk groups and bigger disks,
you would need fewer controllers.
You'd need fewer drive bays, fewer hot swap, you know,
trays, all that kind of stuff.
More disk groups probably means smaller, but more disks, right,
which again means more controllers, more drive bays, bigger physical servers
to hold all the additional drives, that kind of stuff.
So what is your primary level of failure to tolerance,
meaning how much are you willing to say you can handle failing?
If you're like, I'm fine if I can only handle one device failing,
then that would mean you have, you know, you're basically an N plus one.
If you want to say, I want to be able to handle surviving two fail.
And, again, the more hosts you have,
the more you probably want to be able to say you can handle failing, right,
because the more hosts you have, the more devices,
the more probable you have multiple failures.
It's not very common that you have two NICs and two different hosts go bad
at the same time, but things happen, right?
So you need to evaluate your environment.
What can you stand?
What can you handle, all right, for fault tolerance?
So for networking, you have failover and load balancing.
So, again, this is going to be VMware ESXi networking, right?
You're going to have a virtual switch, whether it be a distributed virtual switch
or a standard virtual switch, and you're going to have physical NICs
attached to this virtual switch, and you're going to create port groups.
You're going to create VM kernel ports.
You're going to need to create a VM kernel port that has vSAN enabled on it
and with an IP address and all of that.
So those NICs, hopefully you have multiple NICs.
How are you going to load balance across those NICs?
And you have the same algorithms available with vSAN networking that you would
have on a traditional, you know, port group for virtual machines
or for any other VM kernel service you're running.
So you can say a route based on the original virtual port,
so the virtual port on that virtual switch, you know,
will determine you'll create an algorithm based upon that route.
On an IP hash or based on a physical network adapter load,
that's if you're doing distributed virtual switches, not standard switches, right?
So after 6.7, unicast traffic is sufficient.
You don't need to have multicast with vSAN anymore.
You want to mark the SAN traffic.
So basically what we really want to do, we want a dedicated segment, right?
We want vSAN traffic on a dedicated segment, on a dedicated VLAN,
on dedicated NICs with switch ports that we're not sharing a bunch of
those services with, right?
We want all of this to be dedicated for vSAN, all right?
You don't want to be sharing a bunch of different types of traffic
and have vSAN competing with other services on those ESXi hosts
or with virtual machines on those ESXi hosts.
You place hosts on the same subnet.
You don't want to be routing your vSAN traffic.
It's kind of surprising.
I've been, like, on calls with Cisco.
I've been on calls with different vendors designing, you know,
different storage networks.
Maybe we're, you know, doing a Cisco UCS deployment.
And I've literally had Cisco, and I swear this is true,
want to put the storage adapters for our storage array on the core switch
because it gets it closer to the routing interface.
And I'm like, who's routing this storage?
There's no routing happening here at all.
So you want all your vSAN traffic to be on the same subnet connected, you know,
to one switch as close to the host as we can get, fast interfaces, no routing.
It's just all right there.
That is obviously ideal, right?
You want dedicated bandwidth.
Probably, again, you want at least 10 gigabit.
One is acceptable.
VMware will accept that.
You really want at least 10, maybe even more,
depending on how much you use that storage, right?
Configure the port group, right?
Set whatever policies mean, your load balancing policies
and all that kind of stuff on there.
Set the VLAN tag for that virtual switch.
It will be on your physical network.
And then make sure that on the ESXi host firewall that vSAN traffic is
allowed through, all right?
So requirements to create a vSAN.
So what do you need?
You need at least three ESXi hosts.
You can do it with eight, with two, if you have a third witness.
You need a third device for a witness, all right?
And that's going to be a virtual machine that you can get from VMware for a witness.
But, really, you probably want at least three.
But just so you know, if you have a real small deployment for, like,
a little branch or something like that, you can do it with two and a witness, right?
At least eight gigs of RAM in the host.
Obviously, you're probably going to have a lot more than that.
You need your disk controller to be in pass-through mode, all right?
You need a cache disk and capacity disk.
Now, you need at least one of each.
Again, you can do it without the flash or the caching disk in version 8,
but it needs to be a new deployment, and you'll need to check,
you know, choose the option for that.
Most deployments are going to have a caching disk
and capacity drives per disk group, all right?
A host, of course, needs a network adapter, obviously.
You need a VMkernel port for vSAN.
Now, it says you may need a vMkernel network.
So, what they're talking about there is having a dedicated network for the vMkernel.
You don't necessarily need that,
but you will need a vMkernel adapter port, right, configured for vSAN.
But it doesn't have to be a dedicated VLAN for that, necessarily.
So, let's go ahead and jump over to our demo environment,
and I'll walk you through setting up vSAN.
So, here we are.
We're back in VMware's hol.vmware.com lab environment.
So, it's VMware hands-on labs.
So, it's hol.vmware.com.
And what we have is we have vCenter.
We have a data center and a cluster.
And this cluster happens to have three ESXi hosts in here.
So, what I want to do to configure vSAN is I want to choose my cluster,
and I'm going to go to configure.
I am going to scroll down to vSAN, and I'm going to choose services.
And because I don't have vSAN configured,
all I have right now is just that services option.
And when this comes up, I'll see I have some different options here.
And in these options, I can do a standard vSAN cluster.
I can do a two-node cluster.
And the reason why you see that little VM in the middle there is because that would be a witness.
We need a third location or third device that would run that witness
so we would know if vSAN is up and running or not.
Or we could do a stretch cluster.
But we're not going to do any of those.
We're going to just do a standard vSAN cluster.
And I'm going to say configure.
And one of the first things it wants to know is do we want to do an ESA vSAN or not.
And it's telling me I can't do an ESA vSAN.
And if you wonder, Walt, why can I not do an ESA vSAN?
If I scroll down, I'll see that my NVMe devices are not on VMware's hardware compatibility list.
Now, the device itself may be on the hardware compatibility list, and actually probably is,
but the driver that's currently using is not.
And there are some third-party drivers loaded, I know, in VMware's lab environment.
You might think it's kind of odd that VMware's labs have devices, drivers being used that aren't certified by them,
but that's the case here.
I'm going to go ahead and say next.
And now here I can enable services, right?
I can enable compression, or I can say I want to do deduplication and compression.
I can also enable encryption.
I could do data at rest encryption, which means I need to have a key provider configured already,
or I could do data in transit encryption.
And if I do data in transit, I can say how often do I want to rekey this.
By default, it's one day.
If I were to choose custom, then I would say how many minutes I wanted a key to be good for before I had to rekey.
Okay.
But we're not going to do that right now.
And so I'm just going to leave everything at its defaults, and I'm going to say next.
Now it's asking what disks I want to use, and it's telling me on three hosts I have one disk each for my caching tier,
and on my three hosts I have two disks each for my capacity tier.
And I can expand here and take a look and see what I've got configured.
Now these are all flash.
These happen to be all flash.
But they're still going to my capacity tier, right?
These above are going to go to my caching tier.
Now for some reason, if that's not what I wanted them to do, if I wanted one to be one way or the other, I can choose to change it.
Okay.
I'm going to leave everything the way it is at the defaults.
Again, that works.
I'm going to say next.
And now it's just telling me, okay, so my fault-tolerant domain, I have one domain that these three hosts are going to live in,
so this is going to be one fault-tolerant domain.
And I can review my settings, right?
I'm not doing any data at rest encryption.
I'm not going to do anything special to reclaim space, meaning I'm not going to enable compression
or deduplication or anything like that.
I'm just going to go ahead and say finish.
And now this is going to run through creating my vSAN cluster.
And you can see it kicks off a bunch of different jobs.
We'll go ahead and let this run through, and we'll just continue as soon as it's done in this configuration.
This can actually take, you know, five, ten minutes.
All right.
It looks like it's done running tasks.
What you can do, if you scroll through here, you would see that it went through and, you know, went to create vSAN.
When it created a bunch of disk groups, it added – you know, after it created the vSAN cluster,
it created the disk groups, and then it added disks to the disk groups.
And now we know it's done because it did that scan for our vSAN health check.
So, what the vSAN health check is going to be doing, now that it's kicked this off,
let's go ahead and wait for this to continue – or, I'm sorry, to complete.
So, once this is completed, I'll show you what this is – what this is building.
So, now that health check is completed, let me get this out of the way.
So, now for my cluster, I'm going to go to monitor, scroll down to vSAN, and check on health.
And this will give me a Skyline health check.
Oops, I'm sorry.
Skyline here.
vSAN Skyline health check.
So, my health is at 96%.
That's not horrible.
It's not ideal.
Of course, you want it to be 100.
But if I want to know, well, why am I not at 100?
100 would be better.
Let's scroll down.
And it'll tell me because my vSAN cluster isn't consistent yet.
So, if I let this run a little bit longer, that will probably get better just because it's a brand-new cluster still being, you know,
still syncing itself up.
Also, does not like my NVMe devices.
That's a problem.
And this performance service status here, what this is talking about is that basically all the devices in my vSAN cluster are
really small devices.
They're under – you know, my whole cluster is under 256 gigs, and it's not happy about that.
It doesn't like those tiny devices.
Let's scroll back up here.
Let's see if we do a retest, if that improves my score,
and if it has a chance to synchronize up a little bit more.
Okay, got a little bit better.
Went to 97.
All right.
So, it's starting to sync up.
Now, what we want to do is some time's gone by, and we're using our vSAN.
Everything's going great.
We've been throwing VMs in there.
And now we decide we need to add more capacity to our vSAN cluster.
Maybe we need to add more compute and things like that to our hosts.
So, I'm going to add a host to this vSAN cluster.
So, I'm going to take this host and just drop it on that cluster.
I'm going to say go ahead and bring all the virtual machines that are on that host along with you.
All right.
So, now my host, this new host, is in this cluster.
And if I go back to the cluster and to monitor skyline health, wow, it got way worse.
Now it's 92.
Why did it get so low?
What happened there?
Let's scroll down.
And it's saying that not all the hosts have a vSAN VM kernel adapter configured.
Well, that could be a problem.
So, let's scroll a little bit and just try and get some more information about this.
I say troubleshoot, and it tells me it's host 4.
So, okay, I know host 4 doesn't have the VM kernel configured for vSAN.
So, I'm going to click on that.
Now I'm on host 4.
I'm going to go to configuration, and I'm going to go to VM kernel adapters.
I'm going to add a VM kernel adapter for vSAN.
So, I'm going to say add networking.
My VM kernel adapter.
I'm going to use my vSAN port group.
I'm going to say next.
And I'm going to say I want to add the vSAN service.
Everything else is fine on this screen.
I'm going to assign it a static IP address.
And the IP address I'm going to give it is 192.168.130.54.
And so that masks 255.255.255.0.
And I'm going to override the default gateway.
And I want that to be 192.168.130.1.
I'm going to say next.
And it'll ask me are all my settings correct.
Yep, I don't want to enable any other services other than vSAN.
This is the IP address I wanted, 192.168.130.54.
Yep, 24-bit mask.
Everything looks good there.
I'm going to say finish.
And so now I have a VM kernel port enabled for vSAN,
and it happens to be on the correct network for vSAN.
So if I go back to my cluster
and jump back to my Skyline Health, let's retest that.
Let's give that a moment to run.
And now I'm at 98%, much better.
And the problems I have, that VM kernel adapter once gone,
but my NVMe devices have not changed.
All right, my vSAN max component size hasn't changed.
So it's much, much better.
But now that I've added that, what I haven't done is actually added
any storage to vSAN.
Now I go back down to vSAN, and now that I've installed vSAN on here,
other than just services, I also have disk management.
And under disk management, if I scroll down, I'll see my host is here,
but there's no disk in use.
Zero of the three disks it has are being used by vSAN.
So what I want to do is go and claim unused disks.
And it's showing me what's available to be claimed.
One cache disk is available, and two capacity disks are available.
So I'm going to add those.
This actually goes through pretty quick.
Let's just refresh the screen.
And now on host four, I actually have three of three disks in here.
If I go to my summary, I can see the capacity, right,
how much storage I've got going on here.
I can see my DRS score for this.
This is just a summary for the cluster.
And I can also go into my data stores, choose my vSAN data store.
And I can see the storage that's being used on that because there's overhead
just to create vSAN from those disks, right, the allocation here,
scroll down a little bit, see if there's any other good information.
Here I can see a little graph of my vSAN's 2% used.
All right.
We want to say view capacity.
We can get a little more information from here.
All right.
So we have gone in.
We created a vSAN cluster in the vSAN data store.
We added a host and increased it.
And now we also know how to go in and monitor the health using Skyline of
our vSAN.
And we know how to configure the vSAN services.
So if I want to do something like go into vSAN services and I want to enable
file services so that I can do something like create an SMB or an NFS share,
I can enable those services there.
I can also go in to data services, edit my data services,
and all the same options I had to configure when I was configuring that vSAN,
I can do it from here.
I can go in.
I can enable compression only, deduplication and compression.
I can enable encryption for data at rest or encryption for data in transit.
All the same options are here as when I was going through the installation.
So during the install, if you're not really sure what you want to use
and don't use, or if you change your mind later,
you can go in there and do those things here, right?
Don't forget about your file services from here.
And also don't forget about your data services.
And you can also enable iSCSI targets from here as well.
So that is configuring a vSAN cluster in vSER.
So, again, this is done at hol.vmware.com, and there's a lot more options,
a lot more things you can configure in that lab.
You can configure storage policies and things like that.
So I encourage you to go to hol.vmware.com and try these labs for yourself.
So I hope you enjoyed that demo.
It was informative for you.
So let's go back to continuing our lecture on vSAN.
So to create vSAN like you saw in the demo, there's different ways of doing it.
You definitely need to go to, you know, select your cluster,
and then you're going to want to go to configuration,
go down to vSAN, and you can create your cluster from there.
There are other options, other ways of doing it,
but the most common and easiest way is just from your inventory,
selecting your cluster, going to your configuration tab,
then going down to the vSAN option, and then choosing your services,
and you can run through the whole vSAN wizard there like we did in our demo.
You can also edit the vSAN settings by, again, you can choose your cluster,
go to the configuration tab, selecting services,
and then you can go in there and just choose what option you want to edit.
Do you want to edit your file services?
Do you want to edit iSCSI and make this an iSCSI target?
Do you want to edit data services and enable compression?
Or, you know, what exactly are you looking to do?
But you can do that, again, by choosing your cluster, going to configuration,
going down to vSAN, choosing services,
and then editing the option for each of our components you're looking to edit.
So there's also performance, you know, services.
So, for example, you can make an iSCSI target, enable file services,
not only the data services, which would be your compression,
your deduplication, and your encryption.
You can also enable file services,
which allow you to enable either an NSS share or an SMB share.
And you can also do capacity reservations from there, too.
So you can say capacity reservations are basically a way of saying that you
need to reserve a certain amount of capacity that's not going to be used
because vSAN might need that.
For example, if you know coming up pretty soon you're going to be doing an
upgrade, and so when you do your upgrade, you're going to upgrade vSAN,
and you're like, I'm getting really close on my capacity being utilized,
because you need at least one data store and available capacity, right?
You can actually reserve that capacity so people can't be throwing things on
there, like, say, for example, the shares or things like that,
and limit that usage.
The capacity reservations can give you the ability to save free space that's
actually in the vSAN capacity area so that you can upgrade data stores
or patch and things like that.
So some other vSAN settings, object repair timer,
so that's a way you can go in there and just schedule maintenance on your
vSAN cluster, read a location.
So what you can do if you have a stretch cluster,
a stretch cluster means you have members in different locations, right?
And so relocation will be which location do you want to be the primary space
where, like, a VM is read from.
Because if your hosts are going to be in a certain location running a certain
VM, you don't want them going to another member of vSAN to try
and run that, right?
So you can say which location you want the VM to be running from.
Thin swap provisioning, so you can make your swap space, right?
But thin swap provisioning means that rather than normally it makes a thick
file but uses a whole space for that file, it will thin provision it
and grow it as needed.
It's just a way of saving space for your VMs because every time the VM powers
on, it creates that swap file.
You can also say you want to allow large cluster support,
which means clusters of up to 64 hosts, so it's a pretty large vSAN cluster.
And automatic rebalance would just basically mean if the data stores aren't
balanced and, like, say one host is getting way more utilization than another,
you can automatically rebalance the data stores to even out the load
across the hosts.
So you view vSAN.
So vSAN has storage policies.
You're enabled a storage policy for vSAN the same way as you did in our
previous lesson.
Basically, you just go to your little Big Mac icon at the top, click on that,
and you're going to say your policies option,
and then there you'll have your storage policies.
You'll see ones for vSAN.
There's a default vSAN policy.
You can create custom vSAN policies, right?
You can also get properties about the vSAN itself.
Before you create policies for vSAN, you probably want to know what
the capabilities are.
You don't want to create a policy that requires encryption
and you have it enabled encryption, right?
So you can just browse to your vSAN data store, right,
and then you can go through and look and see what you've configured
or you can go to the summary
and you can see also from summary what you've configured.
So you can either do it from the configuration or from the summary.
Uploading and downloading files from vSAN.
So just like you would do from any other data store,
you can just browse to the vSAN data store and you can right-click on it
until you want to browse data store.
And from within there, you can choose to upload files to it
or download files from it.
So when you upload to the vSAN data store,
whatever you're uploading to there is going to inherit the properties
of the vSAN itself regardless of where it came from.
It's going to now have, if vSAN has encryption enabled,
it will be encrypted.
If it has compression enabled, it will be compressed, things like that.
You can also choose to download from there.
So let's say there's an ISO file you want up there.
You can click on it and choose to download it.
You can also download VMs themselves, but downloading VMs can get tricky
because even like thin provisioned VMs,
when you download them they become thick provisioned.
There's a bunch of things in there that make it kind of complicated.
It's not as straightforward as just right-clicking on it
and you get exactly what you're seeing.
So just be careful about that when you're trying to download from a data store.
You're probably better off going to the VM itself
and exporting it as like an OVA file.
That's usually a better way of getting a VM onto your hard drive
of your local computer versus in the storage on the host
is using like an OVA or OVF file.
But you can browse the data store
and you can download files from there directly too.
You can add storage to an existing disk group
or you can create new disk groups.
So if you're adding just disks to your existing host,
you can go in there and claim those disks
and add them to a data store or to a disk group,
or you can also create a new disk group.
You can also add a host.
If you add in a host, you can bring the disks from that host in
and you're adding additional capacity to vSAN by adding a host with the disks
in there and that will create new disk groups for that host.
When you add disks, you can add either flash disks or capacity disks.
If you're adding capacity, you might want to also add flash,
which means you might be creating another disk group.
Remember, you want to ratio about 30% of flash to capacity.
So you don't want to start adding a bunch of capacity disks
and not upgrading your flash capacity as well.
Another way you can add capacity is to add controllers,
which then allows you to add more disks.
Also, adding more controllers and moving existing disks to there
won't increase your capacity, but it might help with performance.
So that's another issue there.
Most controllers are, you know,
enterprise-grade brand-name controllers, they're very fast.
They can handle loading up the disks on them to their capacity.
But if you are seeing some kind of performance issue with caching on the
HBAs or their disk controllers or something like that,
then adding another controller might help.
And then, again, you can also add capacity.
You can also add caching by adding a host.
So adding device to vSAN, kind of like we saw before, you're going to go in,
you can add the disk to the devices, go to your configuration tab,
choose the vSAN, go to disk management,
and then you can either go to disk groups and add disks,
or you can say you want to just reclaim
or claim the unused disks that are in there.
When you are reclaiming or claiming disks,
you will get some questions like, do you want to clear residual data?
What that means basically is if you want to clear,
especially on a flash drive,
data that might be on pages that are in that flash disk,
generally considering it's a good idea to do that.
It might take a little bit longer because you're going to go through
and do a more thorough wiping,
but it's considered a good idea to do that,
to clear the residual data on there, right?
You might check any partition information.
And then you want to do a disk balance health check
because if you're adding disks to one host and not to another,
you might start seeing some imbalances on the number of VMs on a host
versus another and what the disk can handle for IO and things like that.
It's generally a good idea, though,
to keep all your hosts pretty much configured hardware-wise
and then for software vSAN-wise the same.
So you don't really want to have a host that's got three times the capacity
to the next closest host.
You want them to be balanced out as much as you can.
So vSAN policies, you can define storage policies for vSAN
just like you would do for a SAN storage or something like that.
And then you can assign storage policies to your VMs.
You can do it when they're created, when you go to create a new VM.
You can assign a storage policy.
You can also assign it to a virtual disk for a VM.
So two different virtual disks on a VM can have different storage policies
because maybe one's for the OS and that doesn't have to be on quite as fast
or backed up as often, whereas another disk maybe is for a SQL database
that needs to be very fast and you need it backed up much more frequently,
things like that.
So you can put different policies for different types of disks on a VM.
Service levels for the VMs, so that would just mean what services you want
available like we saw before.
Are we going to do compression, deduplication, things like that?
You can use it as a wizard to allow you to go through and enable rules
and things like that for your vSAN storage.
So your attributes are going to be very standard to what we saw before.
You can do dual-site mirroring.
So we want basically it to be replicated to a vSAN member at another location.
So we have a stretched vSAN cluster.
Stretched vSAN clusters do require really good network throughput,
so be careful about configuring things like that.
You want very low IO latency and very high throughput.
And then advanced storage policy rules, you can go through
and you can create any kind of additional rule that you really need.
Again, you don't want to go through and create policies, though,
until you are sure what your vSAN is configured for.
Again, don't want to create policies that require a service, say,
for example, encryption, and you haven't enabled that yet.
Space efficiency.
So there's unmap.
Unmap is one of those things that's been around forever.
Everyone who's a storage admin working with VMware especially is aware of it,
but not all VMware admins are really familiar with unmap.
So basically, unmap is a SCSI command that will go in and zero out blocks.
So if in a VMDK file within a VM you delete a bunch of files,
the VMDK file is still there.
The VM knows in this file allocation table, so let's say it's a Windows VM,
in the NTFS file allocation table, it's like, oh, yeah, those blocks are available.
But on the VMDK file, they're zeroed.
So you can run unmap in a Linux VM or a Windows VM,
and that will zero out the blocks or the data that it has deleted.
That might be a task you have to be able to run maybe once in the evening
or once a week on the weekend or something like that.
Maybe so often so that it zeroes out the blocks.
So that way your storage knows it can reclaim that space
and it can actually shrink the VMDK files.
Other things you can do to help make your space more efficient besides,
you know, running a task like unmap, you can enable compression.
You can enable deduplication.
Those are both possible.
Your fault tolerance method will determine also your capacity, right?
RAID 5 writes parity once, which means if you can lose one drive and you're okay.
You lose two, you've lost all your data.
RAID 6 writes the parity twice.
So you can lose two drives and still be okay.
You lose a third, you've lost your data, right?
So you don't actually configure RAID on your drives with vSAN,
as we discussed before, but you can configure vSAN to allow, you know,
the loss of a single host or the loss of multiple hosts,
like two hosts, things like that.
So your fault tolerance level, the higher it is,
the more hosts you can afford to lose,
then the less space you have available for actual capacity for storing
VM information.
So this kind of depends there on what your needs are,
what your corporate policy is as far as fault tolerance levels
and things like that.
And SAN encryption.
So we can do encryption, right?
We can encrypt the data at rest.
We can also encrypt data in transit.
If we want to encrypt our data at rest, we need to show it a key.
So we need to configure a key provider, right?
We need to configure a key store.
So if you're going to configure KMS key services,
then that key, you can be requested,
vCenter can have it installed on there,
then vCenter is going to push it to the host,
and the host will then know they have keys available for configuring
services like vSAN.
So you need a key management server that sends the key services to vCenter,
and vCenter will get them to the EFXI host so that we can encrypt our data.
So it's not just a matter of checking a box,
there's actually a whole entire KMS infrastructure behind it.
So some quick well-known issues.
HAVM failover or failures.
So what basically means if you remove a disk group from a host and then all
of a sudden you can't migrate the VMs and it fails to try to migrate,
you need to add the disk group back, right?
You need that storage available.
Post adapter to a cluster without a disk group cannot save via VSVM
deployment to the host.
What they're saying is we can't create an NFS share or we can't create an
SMB share on this host because we don't have disks on it.
We need to add a disk group and add disks to it.
That's all there is to it.
It needs disks in order to use those services, right?
Post-recovery VMs lose HA protection.
So what basically means is if a VSAN has gotten full,
then all of a sudden HA and things like that aren't going to work.
Then you add capacity, and after you've added capacity,
HA still isn't working, the HA services still aren't there.
You can go through and reconfigure HA or just remove and add back HA,
and that will reset the HA services, and that should fix the problem.
So just, you know, again, if a disk is full, HA is going to be kind of,
I don't want to say disabled, not working.
But when you add capacity, it doesn't also fix your HA, and it doesn't.
You just disable HA and add HA back, and that will usually fix that problem.