When we look at traditional networking,
we find that the control plane
and the data plane are tied together.
So for example, if I've got a router
and it's communicating with another router,
and let's just say that they're running OSPF
at the control plane.
Well, the control plane is responsible
for handing routes back and forth.
So we exchange a bunch of routes,
we know how we can reach those specific routes,
and those routes are used to populate tables
at the data plane.
So let's say our IP routing table, for example,
or our Ceph table.
So now we have tables of information.
And so when a packet comes in to this router,
it's going to check the tables
and get forwarded at the data plane to the next hop.
So if we've got, for example,
another router and another subnet this way,
I might know that subnet A is this direction.
So I'm going to forward this packet
with a destination of A,
I'm going to forward it to the next hop,
which is that next router.
Now, traditionally, we would say
that if the control plane were to go down,
that the data plane would have to go down as well.
In other words, if I lose this OSPF connection
for any reason, that I have to remove the information
that's in the routing table that I learned via OSPF.
That is a little bit of an antiquated way
of looking at things,
because certainly that makes sense
if this router were to go down,
or if this link were to go down.
Then sure, I lose the OSPF adjacency,
and I should pull those routes from the routing table.
But what if OSPF isn't actually down,
but just doing a restart?
Maybe this is a software upgrade,
or maybe we made an OSPF config change.
In some capacity, OSPF is changing,
and so we're saying that we're going to lose
control plane access because OSPF is restarting.
So the question is, should we then take down the data plane?
Should I pull these routes from the routing table?
And the answer is no.
Actually, in a lot of cases,
we could leave this in place
and allow the data plane to continue to function.
This means that instead of taking the data plane down,
we actually leave it in an up state.
When we perform this type of operation,
this is known as a graceful restart.
And at this point, we haven't even talked about SD-WAN.
This is in traditional networking.
We have the capability of configuring graceful restart,
and indeed, most of our routing protocols
are configured for that in production networks
in the modern era.
The idea is essentially this,
that we again have these two routers
and we have a control plane connection between them,
and I send a graceful restart message to one router
saying, hey, I'm gonna go down,
and usually there's a timer associated with this.
I'm gonna go down for this long,
and go ahead and leave your data plane online.
So while OSPF or EIGRP or whatever our control plane is,
while that's cycling,
this device knows that the data plane is still online,
and I can still forward packets through that router.
Now, graceful restart does require a couple of things.
First of all, it requires that the data plane
on this router stay online.
If we're completely power cycling that device,
or again, if it's just hard down,
well, then none of this actually works.
The data plane must be accessible,
and so this device here must be configured
for graceful restart,
which if it's sending graceful restart messages,
it would make sense that it's configured for this.
The other concern is possible change on the network.
For example, let's say that this router
does have a downstream connection,
and we know that subnet A lives that direction.
At that point, this router,
whose control plane is down right now,
cannot update the upstream router,
in our case, this router here on the right,
to let it know that A is no longer down.
Changes are going to potentially cause us problems.
That's why we have this timer in place.
We don't wanna make our graceful restart take too long,
because changes are going to result in drops.
I'm going to forward packet A to this router,
who has no ability to tell me that it's path to A is down,
and it's going to drop that packet.
Now, generally speaking, these types of changes are rare,
and so we'd hope that it doesn't happen
during a graceful restart,
which shouldn't take that long for us to do.
But at the same time, it's something we need to be aware of.
So now let's apply this to our SD-WAN environment.
We've said a few times throughout this course
that the concept of software-defined networking,
which SD-WAN falls under that umbrella,
is that we have control plane and data plane separation.
In our traditional network,
we have control plane and data plane
together on one device.
It's running the control plane,
and it's running the data plane.
But in SD-WAN, we don't have that.
We have edge devices
that form IPSec tunnels to one another,
and this is the data plane.
Meanwhile, I have an SD-WAN controller up here,
which is processing OMP messages,
and this is where the control plane lives.
So kind of an interesting concept
that we would have a situation
where we could completely lose the control plane,
but then we have to ask ourselves
whether we should, by extension,
totally take down the data plane.
We can see even more clearly in the SD-WAN scenario
that if we lose access to the SD-WAN controller,
that we should probably keep the IPSec tunnels up,
at least for a little while,
hoping that the control plane comes back online.
But loss of the control plane does not immediately mean
that this edge no longer has access to subnet A,
or whatever the situation is.
In fact, probably, if the control plane goes down,
the edge over here still does have access to subnet A.
That is the most likely scenario.
So just like with traditional networking,
we're going to implement the concept of graceful restart
here in our SD-WAN environment,
and this can happen in two different ways, really.
It can happen where an edge tells the SD-WAN controller
that it's going to initiate a graceful restart.
That's scenario number one.
Scenario number two is the opposite of that.
The controller could actually say
that we're going to reload,
and we're letting the edge know
that we're going to initiate a graceful restart.
So at this point, we, in theory,
are propagating that we are performing graceful restarts
to our upstream peers.
So at this point,
we are not going to expect any kind of OMP updates,
and we're going to keep our routes in the routing table,
knowing that we're not going to get updated
if there are changes on the network.
Now, this part right here,
the fact that we're not going to get updates to changes,
or really be alerted to changes on the network,
this is a big deal,
because the longer the restart time,
the more likely there are that changes will occur.
If I'm down for a few minutes,
there's probably no changes in the network.
But what if I'm down for hours, or even days?
At that point, we would expect
that there'd be a lot of changes on the network
that I have not been made aware of,
because the control plane has been down.
So this is going to lead us to a conversation
around the graceful restart timer.
This timer, as we mentioned over here,
is that we're going to make sure
that we don't wait forever
for the control plane to come back online.
And this timer is configured in seconds.
Now, the way this works is if a device, like the Edge,
is telling the controller
that it's going to perform a graceful restart,
it will tell it what the timer is,
based on its own configuration.
Meaning, if we have it configured for,
let's say, 600 seconds,
then the controller,
which might be configured for 300 seconds,
will be told, hey, wait 600 seconds.
So this controller is going to wait 600 seconds,
even though it's configured for 300.
I wanna be very crystal clear on that.
This is a timer that we tell the other side
how long to wait.
And so if this goes the other direction,
then the Edge is going to end up waiting 300 seconds,
because it's told by the controller to wait 300 seconds.
Again, the configured value
is what the other side will wait,
because I will tell the other side how long to wait.
So what is the default timer?
Well, believe it or not,
the default timer is way higher than 300 or 600.
It is sitting at 43,200 seconds.
That might seem like a random number.
It's not.
This is equal to 12 hours.
Well, it has to do with the fact
that the control plane is separated from the data plane.
And yeah, we generally don't see a whole lot of changes
out in the SD-WAN environment,
especially not meaningful changes.
Because if, for example,
subnet A were to go down because of this Edge,
well, do we have another path to subnet A?
And maybe we don't.
And if that's the case,
then it doesn't really matter
whether we become aware of the fact
that this Edge goes down.
And so we wait for a very long time,
because maybe we have a problem in the data center,
we have a problem with our cloud connectivity,
and it's down for several hours.
Well, at that point,
we'd want our SD-WAN to stay online as best as could be.
And so we're willing to wait up to 12 hours
before we start pulling routes.
In fact, some organizations,
we might want to increase this to 24 hours or 48 hours
or something along those lines.
But there is a very important caveat
to increasing this value.
If we want to increase this value,
then we need to talk about a different timer
called the IPsec rekey timer.
This tells us how often our keys need to be changed
at the data plane.
This is an interesting intersection
of control and data plane operations.
Because if we think about it,
up here, we see that the control plane is down
and the data plane is online, and that's great.
But what happens if we're supposed to be waiting 12 hours,
and after about 15 minutes or so,
we need to rekey our data plane connection?
Well, that is going to involve the controller,
which isn't available.
And therefore, after 15 minutes,
we're going to tear this tunnel down
because we're supposed to be changing IPsec keys,
that's a security concept,
and we can't do so without the SD-WAN controller.
As a result, the data plane does end up going down,
even though we've just started our 12-hour timer.
So Cisco requires that this be twice
the graceful restart timer.
And yes, this is a requirement.
We will see later when we're configuring this
that if we try to configure anything less than twice
the value of the graceful restart timer,
that an error is going to be thrown.
And so this is something to be aware of,
but at the same time,
it's nothing we can misconfigure by mistake
because Cisco blocks us from doing that,
which is very nice.
The idea here is that if graceful restart is going to begin,
we might already be halfway through the IPsec rekey timer,
and because of that,
we're still going to maintain those tunnels
throughout the entire duration of the graceful restart.
At least that's the hope.
So always keep that in mind,
that it must be at minimum twice the graceful restart timer.
And if we do our math correctly,
that means that the IPsec rekey timer
which is 24 hours.
Now, the last thing to be aware of in all of this
is that graceful restart is enabled by default.
We don't have to configure it.
It's already configured.
We already have default timers in place.
And as a result,
it's more that we just need to understand how this works
and whether or not we need to actually tune the timers.
So we'll take a look at how it's configured
and how we change those timers coming up.
But for now, our takeaway here
is understanding why graceful restart matters,
especially in an environment
where our control plane and data plane are separated
and what some of the implications are to changing the timer.