So, let's say we have a pair of switches and these are 10 gig switches.
I've got four of these 10 gig connections between the two switches and I bundle
them
together in an ether channel.
And so, if you're like me, I will refer to this as a 40 gig link.
A lot of us probably do.
And it's easy to think of this as almost taking these four separate connections
and
just sort of filling them in almost so that we've got this gigantic pipe and WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
just sort of filling them in almost so that we've got this gigantic pipe and
that we
can send 40 gig through it.
And the question is, is this really accurate?
Is this a good way to describe what's really happening?
And the answer is no.
That's not the best way to describe what this is.
Yeah, we're taking four separate 10 gig connections and yes, our total
aggregate throughput is
40 gig.
But the thing about this is if we imagine a highway, let's say there's a two-
layer
lane highway here and we've got traffic going and I live in the US, so I'll use
miles per WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
miles per
hour here.
Let's say that the speed limit is 60 miles per hour, 60 mph.
And so that's great and all, but now we're starting to see some congestion on
this two-lane
highway.
There's lots of cars.
And so what might we do?
Well, we might add a third lane or even a fourth lane.
And this is going to allow us to place more cars on the highway at a time and
hopefully
relieve some of that congestion.
But here's the question, did the speed limit change?
And the answer is no. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
And the answer is no.
So if I were to say, okay, well, well, that's great and all, but how fast can
this one car
get through now that we've added two other lanes?
Well, it can still only go 60 miles an hour.
And this has the equivalence of a single traffic flow as it goes through the
ether channel.
Because when I take a traffic flow and a traffic flow is typically traffic that
is flowing
from one host to another as part of one process. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
from one host to another as part of one process.
So we think about TCP and UDP ports, source and destination IP addresses.
This is going to define for us a traffic flow.
This one flow is going to arrive on the switch and this switch recognizes it
needs to send
it out the ether channel.
So what does it do?
Well, it's going to pick a member interface and it's going to send that traffic
flow through
that one member interface, which means that this traffic flow has a total
throughput capability
of only 10 gigabits per second. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
of only 10 gigabits per second.
If these two hosts happen to be sending 25 gigabits per second between the two,
we might
look at it and say, that's fine.
I've got 40 gig between these switches.
25 is a lot of it, but it's not all of it.
Well, it actually is.
It's going to be severely bottlenecked by the fact that it's only going to be
placed on
one of those member links.
So the question we might say is, well, why are we doing this?
Why wouldn't we take this traffic flow and split it up among these links?
And there's a lot of reasons for that, but the primary reason is because this WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
And there's a lot of reasons for that, but the primary reason is because this
is going
to create major out of order issues, meaning that as I take packets one, two,
three and
four and I split them up like this, depending on the inner mechanisms of the
switch and
the number of packets in each individual queue, we might actually get these
packets in the
wrong order.
And so we receive them as, you know, let's just say three, four, one, two,
something like
that.
And then from there, it's going to take whatever path out towards the WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
And then from there, it's going to take whatever path out towards the
destination and that packet
order is going to be the same at that point.
Now, if this is a TCP stream, maybe not that big of a deal, but UDP does
nothing to rearrange
those packets.
And this is going to cause a disaster, not just because it might happen once or
twice,
but because it's going to happen constantly, it's going to happen all of the
time if we're
splitting this up on a per packet basis, which is why it makes sense to take
one traffic
flow and place it onto a single member interface. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
flow and place it onto a single member interface.
We're not going to even consider using this out of order type of method.
So how do we decide which member interface to use?
Well, this is going to require us to talk about creating a hash.
The goal here is for us to identify a traffic flow.
We need to actually figure out which packets belong to which traffic flow.
So we need to identify this, an individual traffic flow, and then we need to
select a
member link. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
member link.
Now the goal here is to decide which member link to use.
But in the middle here, we're going to place this through a hashing algorithm.
And this hashing algorithm is going to give us a number, which tells us the
member link.
So let's give an example here.
So let's say a particular packet arrives and it goes through this process. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
So let's say a particular packet arrives and it goes through this process.
And because of the hash, the resulting hash, let's just say it's a two digit
number here.
Actually, let's just say it's between zero and seven because that's actually
how these
switches work.
And so it picks one of these numbers based on the hash.
And so we end up with a number of four.
And so it gets placed onto link number four in the port channel.
So that's how we determine it.
Now there are a few questions to ask here.
First of all, again, how do we identify this traffic flow? WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
First of all, again, how do we identify this traffic flow?
This is going to be based on certain parameters.
For example, we're going to be taking a look at the source and destination MAC
address.
Since this is a layer two flow, the source and destination MAC will look
different among
the different traffic flows.
Although keep in mind, if we're going to the gateway, then that might not be
the case
in all situations.
But either way, we're going to look at the source and destination IP address.
We're going to look at the QOS tag, even the IP protocol, which will tell us
what's happening WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
what's happening
inside of the layer three packet and the input interface.
So these are the qualities that we're going to be taking a look at to identify
traffic
flow.
These match, then they're part of the same flow.
So this way, as these hosts, well, let's just say as host A here is sending
traffic
to host B, all of the packets that match on all of these parameters are going WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
to host B, all of the packets that match on all of these parameters are going
to be,
it's going to be part of the same traffic flow because all of the packets in
this flow,
well, working backwards, has all of those same properties.
So that's how we identify the traffic flow.
But now we need to also draw a little bit deeper here into this concept of
identifying
a hash ID as well as identifying a link number.
So how exactly does this work?
Well, I just mentioned earlier, a little bit of a spoiler alert that the result
of the
hash is going to be between zero and seven. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
hash is going to be between zero and seven.
This is because once again, our port channels support a max number of links of
eight.
So zero through seven gives us eight different values.
Furthermore, when we have, let's say, let's just start with eight.
Let's say we've got eight different connections here in our port channel
between our devices.
We've got a device here and a device here.
We're going to bundle these together.
Well, at that point, all of these are going to get an ID.
So zero, one, two, three, four, five, six, and seven. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
So zero, one, two, three, four, five, six, and seven.
So when a packet needs to go out this port channel, we're going to run the hash
, find
out the hash equals something, zero through seven, based on these parameters.
And it's going to therefore get sent out link number five, let's say, in the
bundle.
So that probably makes sense.
But what happens if we don't have eight links that are in the port channel?
Well, we can only get these eight different values.
And that leads to some interesting problems with certain numbers of links. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
And that leads to some interesting problems with certain numbers of links.
So for example, if I have two links here, oh, this isn't a big deal.
I'll just evenly divide these numbers up.
So maybe zero, one, two, three on one link, four, five, six, and seven on this
other link.
So if a packet has a result of five again, it's going to go on this link.
If as a result of six, it'll also go on that link.
Does this always result in it being perfectly load balanced between the two?
Probably not, but it's going to get pretty close, at least. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
Probably not, but it's going to get pretty close, at least.
At least that's the hope.
So now what happens if we have four?
Well, same thing.
I'm going to take these four links.
I'm going to divide the numbers up.
Maybe it's something like zero, one, and then two, three, four, five, six, and
seven.
So now if I have a five come through, it goes on this link.
But if I have a six, it will go on the different link.
And so again, as long as we've got enough, maybe we call entropy in the amount
of data
and amount of traffic flows that are coming through these switches. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
and amount of traffic flows that are coming through these switches.
Ideally, we end up load balancing across all four links, although a perfect
load balance
is probably still not realistic.
But then things get a little interesting.
What happens if I have something other than two, four, and eight?
What if I have three links?
One, two, and three.
So I start to divide my numbers up and say, okay, well, let's do zero, one, two
, three,
four, five.
And maybe I could go six and seven, something like that. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
And maybe I could go six and seven, something like that.
Well, regardless of which numbers are used on which link, we see that I've got
three
numbers on the top link, three numbers in the second link, and two numbers on
the bottom
link.
This means that this bottom link, if all things are created equal, will get two
-thirds of
the traffic of the other links.
This isn't ideal.
This is not an ideal load balancing situation.
And we're going to find that this happens as well with five, six, and seven,
because regardless
of the number of links that I have in this range, I'm going to end up with some WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
of the number of links that I have in this range, I'm going to end up with some
unbalanced
number.
So if I have five, for example, one, two, three, four, five, let's just start
splitting
up the numbers, one, two, zero, one, two, three, four, five, and six, and seven
.
So these two links down here will get one half of the traffic of all of these
just because
there are more hash results that can land on the top links.
The idea, generally speaking, is that I should get an even number of all of WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
The idea, generally speaking, is that I should get an even number of all of
these different
hash results, again, so long as there's enough chaos and entropy in the data
flows that exist
in my network.
So if I take an even number of these numbers, and then I divide them up among
the different
links in this way, well, again, this is where you might see twice as much
traffic on one
link compared to another.
Because of this, in a perfect world, we only either have eight links in a port
channel,
or four links, or two links.
That's what we'd want to aim for in any network design. WEBVTT X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000
That's what we'd want to aim for in any network design.
Now granted, I might deploy four links into my ether channel, and one of those
links goes
down for an extended time.
Well, then we're operating as if we have three links and there will be
imbalance there.
But at least in steady state operation, we should be as balanced as possible.
That is certainly going to be an element of network design and implementation
that we're
going to want to take with us when we're talking about real world networks.