So regarding deploying an NSX Manager
or a cluster of NSX Managers,
let's first of all put a game plan together.
So I've currently got a Management Cluster,
and it contains exactly two ESXi hosts.
That's ESXi-C, and ESXi-D, and at the moment,
ESXi-D is currently running my vCenter Server.
And what we're gonna do now
is we're gonna plan on the implementation
of our first NSX Manager,
and we'll go ahead and use ESXi-C as the home
for that little VM.
Or I guess I shouldn't say little VM.
We have some options regarding, you know, WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
We have some options regarding, you know,
extra small, small, medium, or large,
regarding how much resources we wanted it to take.
But in any case, as far as our plan goes,
let's use ESXi-C in our Management Cluster
as the home for this NSX Manager VM.
Now, in a production environment,
it's very likely we're not just gonna have one NSX Manager.
We'd very likely have three.
So let me go ahead and add a couple more ESXi hosts here
based on a plan for some fault tolerance
and some redundancy.
And this one we'll call ESXi-E.
And this will be ESXi-F. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
And this will be ESXi-F.
So then what we'd do is we'd go ahead
and for our three NSX VMs that we're using for management,
we'd put them on separate hosts.
So this could be NSX.11, and this could be NSX.12,
and this could be NSX.13.
And the three of those collectively would be in a cluster.
And ideally, each of those members
of this NSX Management Cluster
would be in different ESXi hosts,
again, for the benefit of fault tolerance.
That way, if we lose one of our ESXi hosts,
we still have a couple NSX Managers in our cluster WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
we still have a couple NSX Managers in our cluster
that are still up and active.
So let's imagine that we have deployed
three NSX Manager appliances as part of a cluster,
and then you and I are sitting on the management network,
or we have access to the management network,
and this is our management PC right here.
So one of the challenges is that if we do have a failure,
let's say ESXi-C fails,
or for some reason this VM fails,
it's no longer functioning,
either way, that's out of commission,
how do we know which IP address to connect to,
to use the graphical user interface for the NSX Manager?
And the solution to that problem
regarding which IP address do we connect to WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
regarding which IP address do we connect to
is to use a virtual IP.
So let's imagine that we set up a virtual IP address,
and here on my management network,
this is the 192.168.1.0 network.
It happens to be VLAN 1 in my studio here.
And let's set up a virtual IP address of .10.
And that would be for the NSX cluster.
So what we would do is we would connect to .10
for the graphical user interface.
Or for using APIs, we'd go to the .10 address.
And then one of the NSX Managers
inside that Management Cluster
would be responsible for responding to WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
would be responsible for responding to
and interacting with us as we communicate with .10.
So in that case, it wouldn't matter too much
if we had one individual NSX Manager
that wasn't available as part of the cluster,
because we're connecting that virtual IP address,
and behind the scenes,
one of the existing running NSX Manager nodes
is gonna support that.
So let me bring back ESXi-C into a running state.
And behind the scenes, what's happening
with these NSX Managers inside the cluster
is they are sharing a common database,
and it's called CorfuDB. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
So the word Corfu is not regarding the Greek island,
but rather it's regarding
a common, object-oriented, real-time framework,
some acronym regarding a common database
that's maintained in memory,
that's based on a shared log.
So long story short, the NSX Managers in the cluster,
behind the scenes, they are sharing
and interacting with the CorfuDB,
which is distributed across them.
So, again, if we have one NSX Manager that goes away,
the shared database regarding NSX is being shared
across the remaining two,
which is gonna provide fault tolerance
inside of our NSX Management Cluster. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
inside of our NSX Management Cluster.
Another important aspect to be aware of
with the Corfu database is,
because it's being maintained in real time
across these devices,
we don't wanna do snapshots on our NSX Manager VMs.
In fact, as part of the deployment,
it's disabled by default.
If you try to do a snapshot, it won't be available to you.
And let's chat for a moment about the integration
that's required between our NSX Management and our vCenter.
So I'm gonna put a little line here
just representing that they need to work together.
And one of the questions might be,
well, why do we need to work together with vCenter? WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
well, why do we need to work together with vCenter?
Well, NSX needs to know several things.
One of 'em, for example, might be,
which ESXi hosts do we want to add the NSX software to,
to go ahead and make them host transport nodes
as part of NSX?
And so the NSX Manager communicates with vCenter
to identify and determine which ESXi hosts are out there.
Or if we want to deploy, for example,
a second or a third NSX Manager, once again,
we're gonna be leveraging the vCenter Server,
which knows about which hosts are out there and reachable.
Or if we wanna deploy an Edge node WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
Or if we wanna deploy an Edge node
to support our gateway services inside of NSX,
once again, we'll rely on the vCenter.
And the vCenter, once again, is gonna provide us information
about which ESXi hosts are available
that can be the home for that Edge transport node,
that VM that we're gonna deploy as part of NSX.
Because at the end of the day, if we're deploying a VM,
it needs to run somewhere.
And vCenter knows about all the compute resources
that it's managing.
And that's one of the other reasons
why we're integrating the two together.
Another reason why we're integrating the NSX Manager WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
Another reason why we're integrating the NSX Manager
with the vCenter is because we are gonna be leveraging
a distributed switch,
which is presided and controlled by vCenter.
And as we add NSX to our environment,
we're gonna be adding logical segments.
Think of 'em like Layer 2 broadcast domains
that are gonna look like port groups available
under the distributed switch.
So, once again, the NSX Manager, working with vCenter,
can identify what those existing distributed switches are.
And as we create additional overlay segments,
and I think of an overlay segment WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
and I think of an overlay segment
like a Layer 2 broadcast domain,
as we create additional Layer 2 overlay segments in NSX,
those will be visible under our distributed switch
as we look at that distributed switch via vCenter.
So over here, let me draw a few NSX nodes.
Now, these could be host transport nodes,
which is an ESXi host, which has been enabled for NSX,
or they could be an Edge node,
whether it's a Bare Metal Edge node,
or it's a VM acting as an Edge node.
So for short, I'm gonna just refer to these
as NSX nodes for a moment here.
I'll label 'em 1, 2, 3, and 4. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
I'll label 'em 1, 2, 3, and 4.
And these are the objects that the NSX Manager
would be responsible for interacting with.
So once we have a cluster set up
and a virtual IP address,
effectively the NSX Manager Cluster
is the NSX Manager service or function.
And let's imagine we have this NSX Manager here,
and it's at .10 on the management network,
the 192.168.1.0 network here in my studio.
So the NSX Manager VM or VMs
are acting as a centralized control plane.
And if you and I wanna manage
or manipulate anything as far as NSX goes, WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
or manipulate anything as far as NSX goes,
we are gonna log in to the NSX Manager,
connecting to the virtual IP address of the cluster.
And as NSX Manager works with the nodes it's managing,
it's going to instruct them on how to behave effectively.
It's a centralized control plane for NSX nodes.
And then each of the NSX nodes,
they're gonna keep a local copy
regarding what they need to know about
regarding the control plane.
So there's a local control plane stored here on this node
and on this node and on this one and on this one.
And that way, if the centralized control plane,
for example, all the NSX Managers are unavailable, WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
for example, all the NSX Managers are unavailable,
these nodes can still operate and behave
because they have a local copy of the information they need
for the control plane to do their job.
And when I refer to doing their jobs,
one of the primary jobs of our NSX nodes,
whether it's a ESXi-based host transport node,
or if it's a VM-based or a Bare Metal Edge transport node,
one of the main functions of those nodes
is to actually move traffic.
For example, a user's forwarding traffic,
or a VM is forwarding traffic
to another location or to another device.
It's the transport node's responsibility,
based on the control plane, to forward the data. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
based on the control plane, to forward the data.
And that's referred to as the data plane.
So a centralized control plane is at the NSX Manager.
Each of their respective nodes
has their appropriate local control plane.
And then each of the NSX nodes
that are participating as part of NSX
are gonna be involved in the data plane.
And one great way to remember that
is think of the word transport, like moving something.
So if we have an ESXi host that's been enabled for NSX,
that's referred to as a host transport node.
And if we've deployed an Edge node
that can provide the Layer 3 routing and gateway services, WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
that can provide the Layer 3 routing and gateway services,
that'd be referred to as an Edge transport node.
So when we think about the data plane,
just think of the word transport,
and it's the host transport nodes and Edge transport nodes
that are doing that work.
It's also the same nodes that have the ability
to build the overlay tunnels using Geneve
so they can forward through the tunnels
wherever the traffic needs to be forwarded to.
Now, when deploying and working with NSX,
if we wanna have success, and I think we do,
we wanna keep a few things in mind.
One is latency, or the delay involved, between two points. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
One is latency, or the delay involved, between two points.
So if we have NSX Managers,
let's go ahead and put a few here.
We'll put NSX 11 and NSX 12
and NSX 13 as managers, as part of a Management Cluster.
We wanna make sure the latency between them
is very, very low.
We want the latency to be 10 milliseconds or less.
So I'll put less than or equal to 10 milliseconds
for the communications between those NSX Managers
in the cluster.
And that's they're using CorfuDB,
and they're synchronizing that information near real time.
So we wanna make sure WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
So we wanna make sure
that they have high-speed connectivity to each other.
Also, for those VMs for accessing their storage,
we'd also wanna make sure the latency
is 10 milliseconds or less.
So that'd be traffic between these VMs
and also for these VMs to their storage,
whether that's local on an ESXi host
or, more likely, on some shared network storage.
The next part about latency else we need to be aware of
is latency between the NSX Managers
and the NSX nodes that they're managing.
And that should be less than or equal to 150 milliseconds. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
And that should be less than or equal to 150 milliseconds.
So let's manage this as a host transport node,
and this is a host transport node,
and this is a Edge transport node,
and this is an Edge transport node.
In any case, we'd wanna make sure
we don't have extreme latency or delay
between the NSX Managers and those nodes.
And when I mean extreme,
we'd want it to be less than or equal to 150 milliseconds
for that communication path.
Also, as far as the protocols that are being used
between the NSX Managers and the nodes,
it is gonna be using TCP ports 1234 and 1235. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
it is gonna be using TCP ports 1234 and 1235.
So on our network infrastructures,
if we're blocking TCP ports 1234 or 12345,
that would also cause a problem between our NSX Managers
and the NSX nodes they're trying to manage.
Again, that includes host transport nodes
or Edge transport nodes.
We'd wanna make sure that TCP port 1234 and 1235
are open and available between the manager and the nodes.
Now, for the deployment of our NSX Managers,
including the first one and the second and third one
for fault tolerance here in our cluster,
we have some options regarding what size WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
we have some options regarding what size
of VM do we want to deploy.
And that's gonna boil down to how much CPU and RAM
is going to be required based on our deployment.
So there's four choices.
We have, like, Goldilocks and the three bears plus one.
So there's extra small, small, medium, and large.
So based on the expected number
of devices we're gonna be managing,
we'd wanna choose the appropriate appliance size
for the deployment of our NSX Manager.
So if we deploy the first NSX Manager as size medium,
then as we deploy our second and third in the cluster, WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
then as we deploy our second and third in the cluster,
we'd wanna use medium there as well.
And that way, we have a nice even balance of resources
across each of the managers in our NSX Manager Cluster.
And let's put a game plan together
for what we want to configure and then verify together.
So I'll label this as Game Plan.
The first thing I think we ought to do
before we start deploying an NSX Manager
is let's just verify our current infrastructure.
So as part of that, we'd wanna verify the port group
that we're gonna be using on ESXi-C for the NSX Manager,
for its management network connectivity. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
for its management network connectivity.
We'd also wanna verify things like NTP is available,
and also DNS services are available.
And to that end, I also have
a little management server right here,
and it's at 192.168.1.100.
And that's the little Windows server
that's providing NTP services and DNS services as well.
So we wanna verify that for whatever NSX Managers
we're gonna put in place,
we're also gonna take another moment
and just verify that we also have DNS resolution in place. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
and just verify that we also have DNS resolution in place.
And if we don't, we'd wanna fix that
by adding the DNS resolution as A records
or, in the case of IP.6, with AAAA records.
I'd also recommend, beside the forward lookups,
also to have pointer records set up
for reverse lookups as well.
We'd also wanna verify our current vCenter Server
and verify what its IP address is.
That IP address for our vCenter Server appliance,
which is currently running on ESXi-D,
it's at the IP address of 185 on the 192.168.1 subnet here
in my home office.
So once we've verified our current infrastructure, WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
So once we've verified our current infrastructure,
the next step would be logically deploying
the first NSX manager in our vSphere environment.
So as a home for our first NSX Manager,
let's go ahead and place it on ESXi-C.
And for its name, let's go ahead
and call it nsx-mgr-11.
And I've also got DNS resolution
for the domain of ogit.local.
So that would be the fully qualified domain name
for our first NSX Manager.
And then we can put this manager at .11. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
And then we can put this manager at .11.
And then as a subsequent step,
we can also go ahead and create a virtual IP address
for the NSX Management Cluster.
And for that virtual IP, let's use 10 for the cluster.
Also, while we're talking about NSX Managers and planning,
let's also plan for a second NSX Manager
called NSX Manager 12, and it'll be at .12,
and also NSX Manager 13, it'll be at .13.
Then we can put them in the same cluster.
Now, in a lab environment,
you can get away with just one NSX Manager.
However, in a production environment for fault tolerance, WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
However, in a production environment for fault tolerance,
you'd wanna have three NSX Managers
and have them across three different ESXi hosts
for not only the compute and memory resources
but also for the benefit of some fault tolerance.
So once we deploy the NSX Manager,
we'd also wanna go ahead and verify it.
Additionally, we're gonna need licensing
on the NSX managers.
So we'll license them.
And if you're building a home lab,
one of the easiest and most effective ways of doing this
is to join up with VMUG Advantage.
As part of that, they'll give you a one-year eval license WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
As part of that, they'll give you a one-year eval license
that you can use for vSphere and vCenter and NSX Manager
and load balancers and a lot more.
And the cost of that currently, I think,
is around $200 US a year and well worth the investment,
'cause you can go ahead and play and use the features
and not get stopped because there's no licensing available.
So VMMUG Advantage will give you the licensing you need
to implement the NSX.
And the licensing available as part of VMUG Advantage
are intended and specified for evaluation purposes,
not for production.
Then we wanna go ahead and link up the NSX Manager WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
Then we wanna go ahead and link up the NSX Manager
and tell it which vCenter to interact and work with.
So we'll say right here, register with vCenter.
And then if we do have three NSX Managers
that are deployed as part of a cluster,
we'd also want to set up some anti-affinity rules
to cause them to not run on the same exact ESXi host.
And that way, if we're using DRS,
we can distribute the load for those NSX Managers
across three separate ESXi hosts.
So I'll make the last little note there,
set up some anti-affinity rules for those three VMs
that are part of our cluster. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
that are part of our cluster.