If you're working your way through the course sequentially, then in the last
skill, we just
saw that we were able to get the proper colors deployed onto our tunnel
interfaces, which
now we have T-locks with colors and they should all be trying to form tunnels
with one another.
And so in theory, when we look at our topology, we would expect there to be
data plane tunnels
forming all over the place.
But here's the thing, if we were to log in, let's say to the graphical user
interface,
we click on tunnels here, we can scroll up and down through the tunnel list and
we see
a couple of interesting things.
First of all, we do see that we're connecting from the Biz Internet color or T-
lock on edge
12 to edge 13 here, Biz Internet to Biz Internet.
We see lots of Biz Internet representation, which is good.
That was one of our two colors.
But again, two things to notice here.
First of all, I don't see anything about MPLS here.
There are no MPLS tunnels that are being formed, even though we followed the
same process for
getting MPLS deployed as we did Biz Internet.
So clearly something is going on there.
The other thing we notice is that edge 14 is showing up here with the default
color.
We got rid of the default color everywhere except for that one interface that
allows
it to communicate with the control components.
And that's fine.
We want edge 14 to form tunnels to the control components, but at the same time
now, it's
forming tunnels through the data plane as well, which we actually do not want.
And so we've got a couple of problems that we want to resolve here by the end
of this
skill.
And we'll take a look at the edge 14 case later on.
The bigger issue right now is that MPLS isn't forming any connections.
Now this is the graphical user interface.
What does the CLI look like?
Well, let's log into edge 11 here.
But if I do a show SD win BFD sessions, we will see that we really only have
three tunnels
and they are all Biz Internet to Biz Internet or again, Biz Internet to Biz
Internet here,
which is good.
But then we do see that Biz Internet to default, which is targeting edge 14
from edge 11.
And so we don't see any MPLS connections here either.
Let's pull up the topology and take a quick look at what it might be going on.
But when we look at this lab topology, it is designed with a couple of things
in mind.
First of all, we are trying to emulate the fact that we have a couple of
different types
of circuits here.
So we've got our Internet and we've got our MPLS that's represented by the
switch.
And then we see that our edges are connected into both of these.
But notice that MPLS, as it might be in the real world, isn't directly
connected to the
Internet.
Now we know that in the SD-WAN world, that it likes to form data plane
connections all
over the place.
And so we would expect that edge 11 and edge 13 are forming connections through
the Internet
and that's what we saw.
We saw lots of data plane connections happening through the Internet.
Even again, this gig 2 that is used by edge 14 to communicate out to the
control components,
that is also forming data plane connections, which is not ideal.
That's not what we want to accomplish with that link.
We'd really rather it not be forming any tunnel connections at the data plane.
Control plane and orchestration plane and management plane, that's all fine.
It needs to form those connections up here, but it shouldn't be forming
connections to
each other.
Again, at the data plane.
But now we have to ask why then are our MPLS connections not forming tunnels?
We should see data plane tunnels down here.
So there's a really simple answer to why this isn't forming.
And it's because when we're looking at a T-lock, again, the physical interface
is not the T-lock.
The T-lock just represents a connection point that, yeah, we use this physical
interface
to form, but it's just sort of this logical construct.
It's not a logical interface.
It doesn't have an IP address, but it is an attachment point as we've called it
.
So we have all these attachment points that are coming into the MPLS cloud, but
there
is an underlying assumption in the SD-WAN environment, which says that if I
want to
advertise this connection point, if I want to advertise this T-lock to the V-
smart, I
can't do it just through another interface.
So I'm trying to come up here and advertise it to the V-smart or the controller
.
I'm not going to use a different color to do that or a different T-lock.
Instead, I'm trying to make sure that I can do it out this interface, which of
course
we see isn't possible.
There isn't a connection between our MPLS and our internet in our lab
environment.
Now in the real world, this might be the case.
A lot of MPLS circuits are going to consist of private IP addresses that are
forming an
underlay, which means that Edge 14 in the real world connecting to a real MPLS
service
provider.
Yeah, the MPLS will allow us to maybe get to an internet circuit through our
network,
but that interface might not actually be able to get out to the internet.
And so if we're hosting our control components in the cloud, which is exactly
what this represents,
we've got control components that are connected through the internet into our
environment.
In this case, it's like we deployed it into Cisco's cloud space, which is in
the AWS
cloud for Amazon.
And so if we have all of our control components connected to the internet, and
then we try
to do something like this where we're spinning our lab, or if not lab, our real
world edges
up on MPLS and those edge interfaces can't connect to the internet, then they
're not
going to be able to reach those control components.
And again, the underlying assumption, which we can change fortunately, but the
default
configuration is for SD-WAN to say, hey, if I can't form control plane
connections,
then I'm not going to bother creating data plane connections.
And that is the problem that we're hitting right now.
So we need to resolve this in one of two ways.
We could try to get rid of this issue and connect these two together and allow
these
control connections to form at which point data plane connections would also
start to
form.
Or what we could do is say, you know what, I know that we're supposed to be
requiring
these control plane connections in order to form data plane connections.
But what if we just said, hey, you know, there's going to be zero connections
for our
control components.
So if we say that there are zero control connections, then the SD-WAN
environment is
going to expect there to be no connections to the controller environment.
And therefore, it will start to form data plane connections.
This might be required in the real world as well.
Again, in the situation where we're trying to form data plane tunnels where it
's not
possible to form control plane connections.
So that is what we're going to do.
And we'll pull that up in the next video to get that configuration in place,
both via
the CLI and the graphical user interface.
But we do have to ask a question before we move on here, which is why does
Cisco make
this requirement?
That's a big fat question mark there.
I'm not sure why that changed.
But why is Cisco requiring this by default and what are the dangers in
switching off
of this?
Well, let's switch over to the chalkboard and talk about that.
So here we have a rough estimation of our lab topology.
And again, what might look to be something we deploy in the real world as well.
So our controllers might be up in the cloud.
So they're only accessible via the internet.
Meanwhile, we have edge devices.
We'll just call this edge 11 on the left and edge 12 on the right.
And they have connections to the internet and MPLS.
And again, this connection might not actually have access to the internet, so
it can't reach
the controllers.
So let's say that in the real world, we hit this problem and we say, all right,
you
know what, we're just going to use that concept of saying, hey, there's going
to be zero control
connections out the MPLS circuit.
And therefore, there's no reason for us to wait for them.
We're just going to form data plane tunnels.
So what happens is exactly what we would want to have happen is we will
successfully
form data plane tunnels through these different service providers.
We're not going to be able to go from T lock on the internet to the T lock on
the MPLS
and form a complete full mesh in this case, because there's no way to get from
the internet
to the MPLS.
This connection doesn't exist.
So I can't then connect it to the other T lock.
So in this case, we'd form two data plane tunnels and that would be perfectly
fine.
Everything would come up and we'd have good redundancy or would we?
So that's the really thing we have to ask is what does redundancy look like in
this scenario?
Redundancy can be a lot of things.
It can be the fact that we've got two circuits here.
It can be the fact that maybe we've got multiple control components that are
redundantly deployed,
like a backup manager.
It could even be maybe a site that has two edge devices and therefore we're
going to
make sure that if one of the edge devices goes down, then we don't need to
worry anymore.
About losing full access to that site even though we have two circuits.
So what type of redundancy is this trying to resolve?
Well, we're trying to resolve redundant connections to the control components.
If we look at this topology, we recognize that we only have one way to get to
my controller
or my V smart right here, it's through the internet.
So what happens if this internet service provider, this ISP is having a bad day
and
all of a sudden maybe our two internet or two remote sites, they're connecting
to the
internet via the same service provider and the service provider goes down?
Well, we still have wide area network connectivity through the MPLS service
provider, but we
have, we'll see very clearly here, that we are not able to access our control
components
anymore.
We have lost access to all of those.
So our data plane is online.
We have three other planes to worry about.
We have the control plane, we have that orchestration plane and we have the
management plane and
all of these are effectively down.
Meaning that I as the network admin can't act, well, I could log into manager,
but it's
not going to see my edge devices because they don't have internet access.
So the management is down for this environment.
Orchestration is down.
If there's new devices trying to come online, while this is happening, then
they're not
going to be successful and the control plane is down as well.
And this is what's really a kind of a big deal is if we have more networks back
here
and there's an additional failure, well, then I can't advertise that failure to
the
control plane to then have it get propagated down to the other sites.
So this is what happens in a lot of software defined environments when we lose
access to
the controllers, especially when we're talking about Cisco as software defined
environments
because Cisco doesn't like to make it so that the data plane is tied to the
controller.
Some vendors do that, but Cisco says, hey, we make really good and really smart
devices,
like these edge devices, routers, switches, all of the components, they have
all of the
information that they need to establish the data plane and keep all of those
tunnels
online.
So the data plane does stay up and users will continue to be able to operate
for as long
as there are no major changes on the network, because by losing the control
plane, if there's
anything that causes any kind of domino effect where we need to advertise
routes or we need
to form new tunnels or anything like that, that can't happen because we've lost
all
of these planes that are behind my head, but we've lost all of these
connections to the
control plane.
So that is why Cisco requires redundancy by default.
And so we have to disable that requirement very intentionally understanding
that yes,
that will bring up my MPLS circuits and keep my make it so I can form data
plane connections.
But we are sacrificing the redundancy that we like to see to our control
components in
the event that we do lose access to one of our service providers.
What's the alternative?
Well, the alternative is to dig into the underlay and see what we can do to
hopefully
make this work a little bit better.
For example, could I give internet access to these interfaces so that they can
get out
to the internet and they can form control plane tunnels?
That would be ideal.
And maybe that involves forming some kind of external connection, maybe it
involves connecting
our MPLS cloud to the internet, which some service providers can do.
There's a lot of different options there.
But as we see in the real world when it comes to those kinds of designs that as
we are exploring
SD-WAN, we want to be asking these questions.
I have maybe a second or a third service provider, what am I going to do if the
internet goes
down if I have cloud controllers or if I'm hosting the controllers in my data
center,
can all of those control components be reached in the event that certain
service providers
go down?
So that's a little bit of a tangent, but it helps us understand what's
happening here,
why Cisco requires that and what that looks like outside of the lab world.
But as far as this skills concerned, we're going to get back to the lab world
and we're
simply going to say let's not form control connections out MPLS, let's simply
form the
data plane tunnels and get our network online.