When we look at first-hop redundancy protocols, most of our focus is going to
be on direct failures.
We've given the example every time of the active node going down, for example,
or maybe the interface fails,
or something directly in the path goes down and prevents gateway access.
But what about an indirect failure, especially in the upstream direction?
FHRP's by themselves aren't going to necessarily help us out here. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
And this can actually lead to at a minimum inefficient forwarding of packets,
meaning that we might have to receive a packet on the active node and then send
it over to the backup node anyways,
but at worst, it can actually result in the black holding of traffic.
And so we need to address the fact that an indirect upstream failure could
cause us issues in our FHRP environment.
So let's see this as an example. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
Usually what we're going to talk about here is a pair of routers running an FH
RP.
We'll just call this the active and the standby.
So in this case, just like usual, we've got active standby, running HSRP, life
is good.
We have our endpoints down here and they're all forwarding up to the active.
And we know if the active goes down, the standby takes over, we redirect it at
layer two to the standby gateway.
So that would be the direct failure. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
But now, let's consider the possibility that both of these routers have
upstream connections to a layer three environment.
This could be our network core, for example, where it could be a single
upstream router.
It could be a bunch of upstream routers, same hardware, different hardware,
either way.
We are talking about a layer three converged environment with routing protocols
and the like.
So now, let's say that we have that indirect upstream failure, meaning that it WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
So now, let's say that we have that indirect upstream failure, meaning that it
's not that we lost the node or the interface on the node,
but rather we lost the connection to the upstream network.
Maybe that was the failure of the link. Maybe there's a router and a router
here and we lost access to this router here.
And so the active node no longer has a direct link to the layer three
environment, but the standby does.
Well, by default, FHRP's are going to look at this and say, "All right, well,
that's well and good, except what do I care most about? WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
that's well and good, except what do I care most about?
Whether my interface is online, and because my interface is online, we're not
going to perform a failover."
Meanwhile, what is the traffic path we're going to take?
Well, we're going to perform our ARP if we need to and layer two is going to
forward us to the active.
And then the active, assuming it's running routing protocols out this layer two
domain, is going to forward us over to the standby. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
And then we get forwarded upstream from a layer three perspective.
Interestingly, the return traffic doesn't quite follow the same pattern because
of the directly attached route when the return traffic comes back to the
standby.
The standby is going to immediately forward it down to the endpoint.
And so we not only have an inefficient path going outbound, we have asymmetric
routing happening because the forwarding path for the initial traffic and the
return traffic
look very different. So this is a situation where we have a massive amount of WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
look very different. So this is a situation where we have a massive amount of
inefficiency.
We're going to have to come up with a way for us to failover even if the active
is online.
Now we mentioned this could perform black holding of traffic as well. And that
could absolutely happen because what if we don't actually have routing?
So maybe we're running OSPF in this environment. So OSPF is running upstream WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
So maybe we're running OSPF in this environment. So OSPF is running upstream
this way, but we have passive interface on our downstream interfaces.
Well, at that point, we don't actually have OSPF routes coming in, which means
that I'm going to send my traffic up to the active and the active is going to
just drop it because it doesn't have a route.
It lost all of its routes upstream and we don't have a routing relationship
with the standby. So even worse, this truly could cause us to start black
holding our traffic. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
And that is absolutely the worst case scenario here. For any reason, we're
dropping traffic at this point. We know that we have a big problem.
And the crazy thing in all of this is that it doesn't require something that's
completely unlikely to happen, but rather just a simple interface going down on
the upstream link or a device upstream failing.
We've got all of that's what caused this. And we've got all the redundancy of
our routing protocols. We've got all the gateway redundancy of our FHRP. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
But as configured, a simple interface failure could result in all of our
traffic getting black hold. Imagine trying to explain that to management after
we justify the cost of the redundant gateways and the deployment of all of
these protocols.
It's not going to be a fun conversation. So what do we have as a solution to
this?
Well, fortunately, we have one and it is known as FHRP object tracking. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
Well, fortunately, we have one and it is known as FHRP object tracking.
FHRP object tracking is going to allow for the detection of an upstream issue.
And at that point, we're going to trigger a failover. Now what's fascinating
about this is in this case, our active node is still online.
We're going to failover from the active and importantly, still online node. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
We're going to failover from the active and importantly, still online node.
So this is a voluntary, a voluntary surrender of the active role.
Now, a natural question at that point is how, how can we defer our role to the
other device?
And the answer here is going to involve two existing mechanisms across our FHRP
's.
First of all, the priority value and second of all, the concept of preemption. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
First of all, the priority value and second of all, the concept of preemption.
So remember how priority works. If I have my two nodes here, I'm going to
assign a priority value.
Now, the default values for HSRP and VRRP are going to be 100.
So if I have a priority of 100 on one side, I could lower the priority of the
other side to let's say 90.
And that means that the left side will become the active because it has a WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
And that means that the left side will become the active because it has a
better priority.
And so that's the how priority works and how the election of the active node
works.
Then from a preemption perspective, if this node were to go down, the active
will fire up on the right side, but the priority value is still 90.
So that when the active node comes back online and it has preemption enabled,
it's going to be able to take back over because it has a higher priority level. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
Now, in this example, the priority level remains static.
And that's typically how we deploy our FHRP's is with a static priority value.
But what happens if priorities start to change? And that's exactly the
mechanism we're going to use here is we're going to use dynamic priority values
.
So now look at the same situation. Let's say that we have preemption actually
enabled on both sides. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
Normally, we wouldn't need preemption enabled on the backup node because its
priority is lower.
But it will never be able to preempt with a lower priority. But what happens if
we both have upstream connections?
This one goes down and we decide because that one went down, we're going to
change our priority value to 70.
Well, now not only do I have preemption enabled, but I do have the higher
priority value, which allows me to take over as the active with my better WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
priority value, which allows me to take over as the active with my better
priority.
Now, if that link comes back online, I'll probably go back to a priority of 100
and because I have preemption, I can take back over as the active.
But the idea here is that our priority values can be a little more fluid.
They can increase or decrease depending on the scenario and especially based on
what's happening upstream.
So the way that we're going to perform this mechanism set is we're going to
again use what we call object tracking. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
again use what we call object tracking.
I'm going to create an object in the system.
And this object is going to be associated with a basically a Boolean or a true
and false statement.
We'll just say it's up or down.
And if it's down, we're going to have a priority decrement value associated
with it.
The word decrement simply means I'm going to decrease something. It's the WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
The word decrement simply means I'm going to decrease something. It's the
opposite of incrementing. If I want to increment something, I increase it.
Decoramenting is decreasing it.
So I might say, for example, that if that link goes down, then I would decre
ment my priority by 30.
And that would take me from that 100 to 70. And when the link comes back online
, I'm back at 100.
So then the question is simply, what are these objects? We actually have quite
a few options available to us.
First of all, we can simply track the line protocol of an interface. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
First of all, we can simply track the line protocol of an interface.
This would be an interface up down status.
Meaning that if I actually lose this link, or maybe over here with this example
, the link that we said went down,
if I lose a link, I detect that. It's direct on me. It's upstream, but it's
still a direct failure at that point.
And now I can say that, all right, I've lost a connection. I should go ahead
and decrement that by whatever I have configured.
So that's option number one. Option number two is I can track reachability. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
So that's option number one. Option number two is I can track reachability.
So I can figure out if I can reach a particular IP address.
This will be useful in the scenario, like we described over here, really the
black hole scenario, where I actually lost my access to the rest of the layer
three network.
I could pick an IP address up there, and I could say, hey, I want to make sure
that I can reach this. And if I can't great, and if I can't, I need to decre
ment my priority value. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
ment my priority value.
So that would be a second option.
A third option is I can actually look at my routing table directly. And so I
can check for route availability. Does a route exist in my routing table? If so
, our object is up.
And if the route doesn't exist, our object is down.
And lastly, we have a whole other world available at our fingertips. It says go
refers to as IP service level agreement or IPSLA.
And IPSLA is its own configuration set. It allows me to do all kinds of WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
And IPSLA is its own configuration set. It allows me to do all kinds of
different performance checks and ultimately trying to establish whether a
service level agreement has been violated.
And I can use TCP tests, UDP tests, ping tests, all kinds of things. And when I
'm using this as my option, so we've got our four different options here.
And using IPSLA as my option, it's either going to be in an upstate or a down
state, which will match to the object. So I have to configure this object that's
going to be at the heart of all of this. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
going to be at the heart of all of this.
And then in my FHRP configuration, I perform my decrement command and I relate
it back to this object.
So it's a mixture of several different mechanisms here. We're leveraging FHRP
mechanisms in our ability to dynamically change priority and use preemption in
order to take over as an active node.
And then we're also using object tracking and FHRP decrements in order to make WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
And then we're also using object tracking and FHRP decrements in order to make
the priority values dynamic.
And in the event that my network is impaired, and again, meaning I'm not down
necessarily, you might be black holding, but we might also just be running in
an efficient manner.
Then at that point, our FHRP object tracking can come in here, detect the issue
and perform a failover thereby keeping our network in really good shape,
despite the indirect or upstream outage.