Welcome back to vSphere 8 in the real world. Now we're going to do an introduction to virtual storage.
So, some common vocabularies, some simple vocabulary to get started here.
Local storage. Local storage is storage that will be local to the ESXi host, meaning it's literally in the ESXi host.
So other hosts can't see it. It's going to be only available to that particular instance of ESXi.
SAN storage. SAN is storage area network. And a SAN storage is block level access to storage that can be accessed over a network.
So that's where the storage area network comes from. And it will be block level access, as I said.
So with SAN storage, you will have either an iSCSI or a fiber channel.
It can be fiber channel over Ethernet device that is connected to your ESXi host either directly, like fiber channel can be connected directly to the host, or it can be done through a switch.
And fiber channel will be done through a fiber channel switch. And iSCSI will be done through a regular Ethernet switch.
And you'll be sending packets over a network to a storage device that will then de-encapsulate those, either take out the iSCSI or take out the fiber channel.
And you will then have SCSI commands that the storage array can then execute for you.
iSCSI is sending SCSI commands over an IP network.
It says here Internet base. So it's Internet with a small i, and that means that it's not like the Internet, because the latency would probably be far too high.
You wouldn't probably have enough bandwidth in most instances, but the latency would really kill you.
But it's sending these SCSI packets encapsulated in IP. So that's where the lowercase i for Internet is coming from with iSCSI.
It says that it's over an IP network. Normally, your iSCSI traffic or any storage traffic is not going to be routed at all.
You don't want to have your VMs going through some kind of a routing device to get from the host to the storage array.
You want it to be on the same network for the minimal amount of latency that is possible.
There are new technologies out there for like metro storage clusters and things like that that make that not quite as true.
But those require very low latency still, even if you're going from one building to the next, it's going to be very, very low latency.
And so you need to be very careful about that. So it's not saying any WAN connection is going to work.
It's going to be a very expensive, high throughput, very low latency connection for those types of technologies.
A LUN is what a SAN is going to present to the iSCSI host.
So on your storage array, you're going to have a group of disks and most storage arrays are going to ask you to take those group of disks and create a pool.
And then you can carve up that pool into smaller blocks called LUNs.
Then you're going to present that LUN to whichever device server on the other side of the network is going to be using that LUN.
And then those devices will partition it up.
So for the Windows server, you might make it a dynamic disk and carve it up.
Or if it would be a basic disk, you could create partitions on it and things like that.
eSXI also can create partitions, but it is considered a best practice, a good practice to only have one partition on a LUN.
Or if it's going to be multiple partitions, it's all for the same data store, meaning that you're going to have extension or have a LUN on there with a data store.
And then you're going to add an extent onto that same data store.
You don't want to carve up a LUN into a bunch of partitions and then put a bunch of different data stores on there.
Ideally, you want it to be just one storage device that's presented to the eSXI host.
Virtual disks in a VMware environment, a virtual disk is a VMDK file.
And so this virtual disk or this VMDK file is going to be two files.
So it would be one VMDK file that is a descriptor, it's a small little like maybe like 1K file that describes the geometry and things like that of the virtual disk that the eSXI host should be expecting to see.
The second file is going to be a much larger file, and that is where the data is going to be stored by the VM.
When the VM says write this to disk, when it thinks it's sending commands to write to a hard drive, that data gets written to that VMDK file.
And that is the virtual disk for that virtual machine.
VMFS, so we have a LUN, right?
We present that LUN to the host, the host now needs to write to it.
Before it can write to it, it has to format it.
That is what VMFS is.
VMFS is the file system that eSXI uses to format both local storage and your network attached storage, meaning your SANs.
If you have a SAN and you're using vBall, then you don't actually have that VMFS file system, but that's a whole different conversation.
And vBall requires a SAN that supports vBall specifically.
Not everyone uses vBall.
You need a certain license level to use vBall, and so we'll talk about that in a later lesson.
But VMFS is the file system, the native file system for eSXI.
So if you're formatting a local disk or you're formatting a LUN that looks like a disk that eSXI hosts, you're formatting that as VMFS.
So NFS is a NAS protocol.
So we have SAN, right, which is a storage area network, and we're sending SCSI commands over a network, you know, block commands to a block device on the network.
NFS is a file level protocol.
So we don't send SCSI commands and think that we're working at a block level.
We're going to say, I want to access that file or write this file, copy this file.
It's all file based.
So that's kind of what you're used to looking at.
When you go to Windows Explorer, you don't look at the blocks on the disk, you look at the file.
So NFS presents to the eSXI host the same way.
It presents at a file level.
So the eSXI host will just say, here's my VMFS file, here's my VMX file.
It's a file level storage.
It's like SMB.
So if you're used to Windows shares, Windows shares are SMB.
Only NFS is what would be native to like a Unix system.
And eSXI being, you know, Linux space, NFS is the network protocol that it uses for file level communication over a network.
And a raw device map.
So a raw device map means we have a SAN.
The SAN has a pool and it has a LUN.
The LUN is presented to the host.
But rather than format that LUN with VMFS, we present it to the virtual machine and that VM is going to own that LUN.
And then that VM can format it.
So with a Windows server, for example, the Windows server would format that LUN as NTFS.
And we would still have a VMX file that's attached to the virtual machine that describes and identifies that LUN that is that raw device map.
But the raw device map allows the Windows system or Linux, whatever the guest OS of that VM is, to write commands, SCSI commands to the LUN.
So there's different types of raw device maps.
There's one type where there is no interpretation.
There's no pair of virtualization happening.
So the Windows machine or the Linux machine, whatever the VM is, is going to send commands directly uninterpreted by the eSXI host.
To the SAN for that LUN.
And there's another version that will allow us to actually interpret that eSXI host interprets the SCSI commands from the virtual machine and then translates them and sends them to the storage array.
And it's still formatted as NTFS or, you know, ESP3 or ESP4 or whatever it is.
But with the virtualized one where the host can do some interpretation, we can do things like snapshots and things like that.
When it's just a true raw device map, then we can't take snapshots or anything like that for the virtual machine.
So it's just going to be writing raw right directly to that disk.
So shared storage design, vSAN.
So what vSAN is, we have a group of eSXI hosts that are, you know, in a cluster.
And maybe these hosts all have a lot of storage available on them.
And we want to utilize that storage.
And we don't want to go spend, you know, a million dollars on a SAN.
We want to just use the storage that's in these hosts.
And maybe the hosts all have 16 hard drives each and the hard drives are, you know, three or four or five terabytes each or whatever size.
They could be huge.
And they're all the same across all the hosts.
So now what I want to do is I want to take it and have it all look like it's shared SAN storage to my eSXI host.
Even though it's local storage on this host and it's local storage on this host and, you know, each one has their own local storage, it's going to look like a virtualized SAN.
So what's going to happen, and this is kind of important to remember for later lessons, is with a vSAN, we create what are called disk groups.
So on one host, maybe I have 16 drives.
So maybe I'm going to create four disk groups of four disks each.
And I will also have a flash drive.
And before version 8, you had to have at least one flash disk for cache because these spinning disks might not be fast enough to process, you know, and give you the kind of performance you need for virtual machines on this shared storage like this.
So you have a flash disk that the system writes to, and then it gets offloaded from the flash drive, which is cache, to the actual disks that are going to be used for the actual storage long term.
So this host has the 16 drives, the four storage groups, or four disk groups of four, you know, four disks each, and they're all the same, right?
They all are configured with four storage, four disk groups of four disks each across all 15 of my hosts.
And so I actually end up having 60 different disk groups across these 15 hosts with all these different disks.
Then it can take these disk groups, and it can virtualize them into a partition of this virtual SAN, and all of the hosts do the same thing.
So now when I have a VM, I think I'm going to create a new VM, and I'll put it on, you'll hear on my vSAN, the system will say, OK, what are the rules you use when you create a vSAN?
I said I wanted a N plus one.
N plus one simply means that when I create a virtual disk on this host, I'm going to create a copy of it someplace else.
I give you an N plus two, which means when I create a virtual disk on this host, I create two more copies somewhere on this network.
So I always have three copies, the active one that I'm working on, plus two additional ones that are available.
So if this one goes down, then it immediately spins up on another host, right?
I'll move the VM to another host, and I'll run that VM on that host, and I have my other copy, and it's going to say, well, my rule is I want N plus two.
And right now I have my N, and I have one.
So now it's going to make another copy on another host, because I have 15 hosts in this big network.
I have plenty of room to create additional.
But one important thing to remember is a host is not a disk group necessarily.
You could create one disk group.
You probably won't.
You'll create multiple disk groups on a host, breaking all those disks up into smaller groups of disk groups.
And that'll be important later on when you're looking at things like fault tolerance, when you're looking at upgrading, when you're making changes to your vSAN.
Having smaller chunks in these disk groups is going to make it really helpful, because let's say on here I had 16 terabytes all in one disk group.
And I want to make an upgrade to a file system, or I want to upgrade vSAN, or I want to start encrypting my data, I have to have 16 terabytes of free space someplace on my network to do that with.
If I have four disk groups of four terabytes each, now I only have to have four terabytes of free disk space in my vSAN to start doing upgrades and things like that.
So, small disk groups make it easier for things like, like I said, upgrading to new versions of vSAN and enabling encryption and things like that.
So, vSAN is a way to allow you to take local storage and present it to this ESXi host, and it looks like it's iSCSI storage.
And some of the more recent versions, like with 8, you can actually create sys shares on these and present them to Windows machines, and you can do all kinds of things like that too.
But the really important thing, I think it's going to be most often used for, is I'm able to use local storage on my hosts to store my virtual machines, and there'll be copies on other hosts.
And I can still do DRS, I can do HA, I can do vMotion, I can do all these things that we require a more expensive shared storage, you know, SAN device.
All right.
So, here we're seeing we have a bunch of VMs.
The VMs live on vSAN.
vSAN, this looks like vSAN.
You don't, you won't see that, oh, well, this VM lives on this host, and that's a copy here and here.
You can dig in and get that information, but when you just look at it, you'll see a data store and you'll see a VM on it.
And just from that, you know, regular, this initial view, you're not going to see all the hosts it's on and things like that.
But you can obviously dig in and get information about it.
So, we're going to have that vSAN.
We show the SSD, the SSD drive is going to be my cache disk, and I have regular hard disk.
That's my disk group.
And then I'll have multiple disk groups, hopefully, probably multiple disk groups per host, but they all get put together and I create a data store from that.
So, it's a pretty cool, it's a pretty cool concept.
I remember when it first came out, I think it was like vSphere 5.5, I could be wrong, I think it was somewhere around 5.5.
It was a really, it was 6, something like that.
It was a really big deal.
And honestly, like a lot of traditional SAN vendors, EMC, Hitachi, NetApp, a bunch of them, their sales plummeted that following year as people started looking at vSAN more closely.
And they started looking at how can they get into the hyper-converged business because they saw their SAN business was going to start shrinking.
And they're all still around, they all still sell SANs, but their business definitely took a hit.
And I know that because I worked for our SAN vendor during that time.
So, local storage design, this is a local disk on a host.
So, again, I have my ESXi host that's running the VM.
That VM has that VMDK file.
That VMDK file is going to be written on a VMFS volume.
That VMFS volume is on a disk that's on that host, that local attached disk.
All right.
So, the ESXi does support a lot of different drives than what it used to.
When it first came out, it only supported SCSI drives.
Then they added IDE drives.
And then they support SAS, they support SATA, they even now support USB devices, not for production VMs and things like that, but they support USB.
They didn't for a long time.
So, we are able to connect to external storage devices like SAS and SATA storage devices, and they can be treated like they're local internal storage on those machines.
We can also have SCSI devices or, you know, SATA device, SAS devices in that machine as well, in that box.
So, examples of some simple storage design.
So, again, your ESXi host is running your VM.
The VM is on a SCSI disk that's formatted as a VMFS, and it has that VMDK file.
VMDK is just one file of many.
We talk about that with disks the most often because that is what is that virtual disk for the VM.
But the VM also has, you know, VMX files, and it has swap files, and it has a lot of different files associated with it.
All of them are going to live on that VMFS datastore with that VM.
So, storage characteristics.
So, once you create a block device, you're going to want to get some information about that block device, right?
So, you're going to give it a name.
You want to know what the name is.
You'll have an identifier.
It's going to be a unique identifier.
Your universally unique identifier needs to be unique to the device, but you're going to want it to be the same to all the ESXi hosts.
They're all going to see it with the same name.
Operational state.
So, is it attached or is it detached?
The LUN number.
You want your LUNs to all be seen by your ESXi host with the same LUN number.
If it's LUN 5 on one host, you want it to be LUN 5 on all the hosts.
You want all the identifiers for your disks to be the same across all your hosts.
Because if a VM needs to move from one host to another or be picked up from one host to another, like, for example, in an HA situation, you want the host to know where to find it.
And so, you want those identifiers to always be the same.
So, what type of device?
Is it flash?
Is it a hard disk drive, meaning a spinning disk?
Is it a CD-ROM drive or something like that?
Again, if it is a disk versus like a CD-ROM drive, what kind of disk is it?
Is it flash or hard disk?
Transport.
So, what transport protocol are you using?
Is it a fiber channel?
Is it iSCSI?
Are you using iSCSI, the software initiator?
How are you connecting to it?
Capacity.
How big is it?
Who owns it?
By owner, they mean the plug-in that's managing it.
So, it could be a native plug-in for the ESXi host or it could be a third-party plug-in that's managing the access to that disk device, that LUN.
Hardware acceleration.
Does this device support hardware acceleration?
Meaning, does the storage array support vSphere APIs for managing the disk?
And by hardware acceleration, we're saying we want to offload tasks from the host to the hardware to the storage array to make things go faster.
So, for example, it used to be like back in the 3, 5 days and 4 that, let's say, I have a hard disk and I want to make a copy of that.
I could just copy it on the ESXi host and the ESXi host would read the block and write the block and read the blocks and write the blocks all over your network.
Now, what it can do is it can just tell the storage array, make a copy of that and make a copy of it from this LUN to that LUN if both LUNs are on the same storage array.
Or if you want to create a new virtual disk, this is probably the most common reasons or times you'll see this, is when you're creating a new virtual disk, you can either make it thin provisions,
which means it'll just be a really simple, empty file.
And as you write to it, the file grows, grows, grows, grows.
So, in that scenario, every time the file grows, we're going to put a lock on that virtual file.
We used to have to write a lock to the entire LUN, but VMware improved that and now we can just lock files and not have to lock the entire LUN.
The reason why that's important is because if I had two different virtual machines on the same LUN, only one of them can grow at a time because there'd be a lock on it, I'd grow that file, and then the lock would be released.
And then another host might need to put a lock on it, grow a file, the file's done, then that lock could be released.
But only one device can have a lock on a storage device like the LUN at a time.
So, now we put locks at the file level and we offload that to the storage array.
The storage array can create a lock, grow a file, release a lock much faster internally than the host is going to do it over the network.
So, we can offload things like that, copying files, growing files.
If we have a thick provisioned file, then that means when we create the virtual disk, we're going to create it all in one time, all one great big file.
It's all allocated at the beginning.
And if we want to make it either zeroed, that means we want to zero out all the blocks on that virtual disk when we create it.
The SXI host used to go in and create a 700-gig file and then write 700 gigs of zeros, and it would take a long time.
Now, if we have hardware acceleration enabled with our SAN vendor, the SXI host can communicate to the SAN and say,
on this data store, create this file, make it this size, and zero out all the blocks.
And now the host is done. The host is no longer worried about that.
The SAN is taking care of it, and the SAN will let it know, I completed that task.
You go back to doing what you were doing, and the host can just keep trucking along.
So, that's hardware acceleration.
We're offloading tasks the host used to have to do itself to the storage array to speed up the performance of the VMs and the ESXi hosts.
Sector format.
So, sector format is talking about – there's different types of sectors.
And so, you can see the sectors – basically, you're talking about the size of the sector.
And with disks getting so huge and LUNs getting so huge and, you know, storage getting so massive,
it used to be that we could only write one file per sector, right?
But now what we're able to do is we create larger sectors.
Our sectors are always 4K now, but we can break that sector up into smaller virtual pages,
and I can write a file to a virtual page.
So, I don't have to think of a 4K sector to write a simple little, you know, file that only has,
you know, a small bit of text and bits of information in it.
So, we can break it up, and we can write multiple small files inside those sectors, right?
So, basically, we can have these little virtual pages inside of a sector.
So, it looks like they're small sectors.
The reason why that matters is because you only have so many sectors on a disk.
And so, if you have small sectors, that means they're the maximum size that your disk can be.
If you have larger sectors and the same number,
obviously, your disk can be much, much larger now, right?
So, that's what we're talking about.
Partition format – is it an MBR, GUID, or GPT?
Is it a primary or logical partition?
Used to be you can only create one primary partition.
That was it.
Now, you know, VMware gives us all these options.
But best practice is you want to create as few partitions as possible.
If you have multiple partitions, you want them for one data store.
So, like, you need to add an extent to a data store.
So, maybe you created a LUN.
The LUN was 500 gigs.
The data store is starting to fill up.
So, you can increase the size of the LUN on the storage array.
That doesn't increase your file system.
And I can go ahead and add an extent to that file system.
So, now it's a terabyte, right?
But the whole entire LUN is only controlled by one VMFS data store.
Multipathing policies – so, multipathing policies is basically
VMware allows some different policies.
There's ones where we can do – which is kind of like a very common one
most people try to do – is a round-robin.
So, if you have multiple data paths, it sends packets down one path,
and then packets down the next, and packets down the next,
and packets down the next.
And this keeps going around and around on those paths.
Because it can only send so many before it has to get an acknowledgment back.
So, while I'm waiting for the acknowledgment back, I'll use the next path.
And I'll wait for an acknowledgment back, and I'll use the next path.
Otherwise, I have to send down a path, and I'll wait,
got an acknowledgment, send some more,
got an acknowledgment, send some more.
If that path goes down, I can fail over to a backup path.
But I then only have, again, that one path.
So, round-robin gives you more paths, gives you more bandwidth, okay?
So, we've covered some basic vocabulary of VMware storage, ESXi storage.
We covered, like, what is a SAN?
What is a storage area network?
What is a network-attached storage?
We covered data stores.
We covered file systems.