Enabling Linux OOM Killer

Sometimes we do want to have some process killed to save the overall other works left not saved on Linux.

In Linux, there are two kernel parameters that serves the purpose:

1. vm.admin_reserve_kbytes
2. vm.oom_kill_allocating_task

vm.admin_reserve_kbytes

In the kernel doc, it says that the minimal suggestion for vm.admin_reserve_kbytes is 128MB. However, when using desktop, the minimal preservation is at least 512MB so that you can switch to console using CTRL+ALT+F1 and kill the ideal process.

==============================================================
admin_reserve_kbytes

The amount of free memory in the system that should be reserved for users
with the capability cap_sys_admin.

admin_reserve_kbytes defaults to min(3% of free pages, 8MB)

That should provide enough for the admin to log in and kill a process,
if necessary, under the default overcommit 'guess' mode.

Systems running under overcommit 'never' should increase this to account
for the full Virtual Memory Size of programs used to recover. Otherwise,
root may not be able to log in to recover the system.

How do you calculate a minimum useful reserve?

sshd or login + bash (or some other shell) + top (or ps, kill, etc.)

For overcommit 'guess', we can sum resident set sizes (RSS).
On x86_64 this is about 8MB.

For overcommit 'never', we can take the max of their virtual sizes (VSZ)
and add the sum of their RSS.
On x86_64 this is about 128MB.

Changing this takes effect whenever an application requests memory.
==============================================================

vm.oom_kill_allocating_task

By default, vm.oom_kill_allocating_task is set to 0 and it'll trigger a scan-through in the task list and choose the task that takes up the most amount of memory to kill. However, it is usually impossible to finish the whole process since we do not have that much memory left to do it. The overall system is hanging and no process will actually get killed. Setting it to non-zero value can help us quickly trigger a kill and save the rest of our works.

==============================================================
oom_kill_allocating_task

This enables or disables killing the OOM-triggering task in
out-of-memory situations.

If this is set to zero, the OOM killer will scan through the entire
tasklist and select a task based on heuristics to kill.  This normally
selects a rogue memory-hogging task that frees up a large amount of
memory when killed.

If this is set to non-zero, the OOM killer simply kills the task that
triggered the out-of-memory condition.  This avoids the expensive
tasklist scan.

If panic_on_oom is selected, it takes precedence over whatever value
is used in oom_kill_allocating_task.

The default value is 0.
==============================================================

Setup Temporarily

# Preserve 1GB for system admin
sudo sysctl -w vm.admin_reserve_kbytes=1048576
# Don't scan, but just kill the task that causes the OOM
sudo sysctl -w vm.oom_kill_allocating_task=1

Setup that is Persistent Across Reboots

# /etc/sysctl.conf

# Preserve 1GB for system admin
vm.admin_reserve_kbytes     = 1048576
# Don't scan, but just kill the task that causes the OOM
vm.oom_kill_allocating_task = 1