skip to content

Department of Genetics

 

We have some linux boxes used for small-scale HPC by novice users. These can become unresponsive under extreme load, so I use cgroups to try to reserve some I/O, RAM and CPU for system processes

 

/etc/cgconfig.conf -- Used by cgconfigparser this is one way of creating cgroups.

group inter {
        cpu {
                cpu.shares="50"; # weighting of CPU usage. Interactive jobs are penalised
        }
        memory {
                memory.limit_in_bytes="200G"; # Hold back some RAM for system processes and caches
                memory.memsw.limit_in_bytes="250G"; # Don't go crazy on the swapfile
        }
}

group sys {
        cpu {
                cpu.shares="250"; # System stuff shouldn't need much, but really needs it.
        }
}

group batch {
        cpu {
                cpu.shares="500"; # As batch should only start jobs when the load is low we'll let these jobs have higher priority.
        }
        memory {
                memory.limit_in_bytes="2G";
                memory.memsw.limit_in_bytes="2G";
                memory.oom_control="1"; # Pause, don't kill OOM processes
        }
}

/etc/cgrules.conf -- this classifies processes into cgroups. cgruleengd uses it every time a process starts or changes UID. Processes inherit their cgroups from their parents.

# user:process cgroup type cgroup name

root:xrdp-sesman        cpu,memory      inter
root            cpu             sys
*:atd           cpu,memory      batch
*:screen        cpu,memory      inter
co:sshd         cpu             sys
*:sshd          cpu,memory      inter
*:rserver       cpu,memory      inter
*:Xvnc          cpu,memory      inter
*:xfce4-session cpu,memory      inter

To start things off at boot I append to /etc/rc.conf (I should do this in a better way I'm sure...)

# Read cgconfig.conf and create the cgroups
cgconfigparser -l /etc/cgconfig.conf
# Start the automatic classifier
/usr/sbin/cgrulesengd -f /var/log/cgrulesengd.log