In this video, you'll learn what you can do if you need to troubleshoot Linux memory.
To start with, use 3-M. That will give you a generic overview of current memory usage.
And you should check if a minimum of 20% of RAM is available as free or cached. If it's
not, you need to further investigate what is going on. Also, consider the use of swap
space. Are you currently using any swap? Do you even have swap? And if swap is used, use
vmstat to get usage information. Also, check logs to find out if the own killer is being
issued.
Let me show you. In order to investigate memory problems, it would be cool if you have memory
problems. And I'm going to reboot and limit the amount of available memory while rebooting
so that we can see that actually something is happening. Because right now, on this four
gig system, nothing really is happening. So once you are in a group menu, at the end
of the line that loads the Linux kernel, temporarily remove rhdb and quiet and then use mem equals
1g. That will make that you have one gigabyte of memory. And that makes it all a little
bit more interesting. Next, control X to start. So to the terminal to investigate, and a good
first step is to use 3 minus m. And now we see interesting stuff. So one gigabyte of
memory is what I passed while starting up. Unfortunately, for my system, part of that
memory is used for the graphics card. So we only have 708 remaining. And that means that
we fail all sanity tasks. So 529 used, only 56 free. And you can also see that 770 mebibytes
of memory is currently in swap. What does that mean? Well, we have more memory in swap.
Then we have a total memory available. And that is perfect for troubleshooting. So I
would like to use a VM stat to five, two second interval, five polling loops in total. And what
I'm observing is swap activity. And here you can see the swap activity, which is pretty close to
block activity. Now the thing that you are not supposed to like here is that swap in and swap
out, it's continuously happening. And I wasn't even doing something. And that is really an indicator
of a poor state of your memory. Of course, we have 10 times the amount of swap as the amount
of RAM. So it's going to take a while before you are running out of swap. But you are seriously
running out of memory. I would also advise you grab on active memories in PROC MEMINFO.
I'm skipping the A, by the way, so that I don't have to worry about is that an uppercase or
lowercase. And this is showing what I want to see. So what I see is that I have this amount of
active memory. And here I have my inactive memory. And this is application memory. And the thing is
104244 kilobytes. That means we have 104 megabytes of inactive application memory. The amount of
memory that we have in swap is way higher. And that means that active memory is being swapped
out. As it doesn't show as active and inactive anonymous memory, which is your typical application
memory, we are most likely talking about kernel procedures and routines. So beyond any doubt,
this system is way out of memory. Then I'm going to do something fun to trigger an out of memory.
I want you to be able to recognize an out of memory condition. And in order to do so, first,
I'm echoing an H to the file PROC SYSRQ trigger. The SYSRQ trigger is giving access to some magical
controls on your Linux. Now, if you echo an H, you get help. And this help is only visible
in the D message output. And the help I'm looking for is the memory full ohm kill. You can see that
on the bottom, it's the letter F that we need to echo. So this time, I'm going to echo an F
to PROC SYSRQ trigger. And oh, boy, I get, oh, do you see that? Device memory is almost full.
Terminal was using a lot of memory. And I'm going to start my terminal again. At least I'm going to
try, because I would like to show you what you can find in the logs if this happens.
So journal CTL, uppercase G to go all the way to the end. And then I'm scrolling up a little bit.
And here, I can see out of memory, out of memory, killed process, whatever. But you should be
looking for this ohm kill. Now, by echoing an F into this PROC SYSRQ trigger, I manually force
an ohm kill. What you should be worried about is if you can see it in any other circumstances.
Now, this is the same ohm kill than the one that I just triggered. And if there is only one of them,
then it's not an issue. But if you have multiple out of memory killers that occurred, then really,
you have a problem. So that is how you can identify poorly behaving memory. Let's boot
back into a normal system. So one solution, if you are running out of memory, is to allocate some
swap. So swap space can be created on a block device or on a swap file. And if you are using
a partition, you must set the partition type to swap. And next, you use mkswap to format it
as a swap partition. And you swap on to activate the swap space. Normally, swap is mounted through
etcfsstep. And from there, systemd will pick it up and mount it as a swap unit type. Let me show
you how this works. So let's talk about swap. We are back in a normal operational state where
we have 8 gigs of swap, which is not used at all. But I want to show you how to use swap.
So what am I going to do? Well, I'm going to use dd. If is dev0. Of is slash swap file. We're
going to make a swap file this time. Bs is 1M. Oops. Bs is 1M. And count is 1024 to make for a
one gigabyte swap file. Good. Then what are we going to do? Well, I don't have to create a partition
this time. But I do need mkswap on slash swap file to format the swap file as a swap device.
And there you can see that the mkswap command is kind enough to tell me about the insecure
permissions. So I need to fix it using chmod 600 on slash swap file. The reason is that only the
root user should have read and write access to the swap file, nobody else. You can even see that
the swap file has created a label. Now, as this is a file, and files typically don't change their
name spontaneously, I'm just going to put it in etcfsstep as such. So slash swap file. And I'm
going to mount it the way you should mount swap. And for swap, we have seen it before. There's a
couple of examples here. So how do I mount it? Well, I mount it on none. The type is swap. The
mount options are defaults, and then the zero and zero. And next, I can use swap on minus A.
That's comparable to mount minus A. And it will activate all swap that is in etcfsstep and not
yet initialized. So when I use free minus M, I can see that I have another gigabyte of swap
available. Swap on minus S, by the way, is showing all the different swap devices.
So here we can see that we really got it all. We have a swap logical volume, a swap partition,
and a swap file. Also interesting to see is the priority. Minus two, minus three, minus four.
The minus two priority is used first. And that's how you can add swap to give your system
the opportunity to calm down if it is running out of memory.