Zram

zram is a Linux kernel module that creates a RAM backed block device with features including on-the-fly compression.

Because zram can transparently compress data, it may be used as a swapfile which effectively allows RAM to be compressed as it is paged out. This is particularly useful for memory constrained systems.

zswap is closely related with the biggest difference that less frequently used pages can be swapped out from RAM to physical disk.

Quick usage guide

Firstly, load the zram kernel module as root. This should create /dev/zram0 which you can configure with zramctl. You may set the size and compression algorithm when creating the block device.

Supported compression algorithms include: lz4 (more efficient compression but slower), lzo (less efficient but faster), zstd. More on the supported algorithms below.

## Load the zram module. It should have created /dev/zram0 by default.
# modprobe zram
# ls -al /dev/zram*
/dev/zram0

## Define the disk size and algorithm of zram0
# zramctl --size 128m --algorithm lz4 /dev/zram0

Using zram with a Ext4 filesystem

Once the block device is created (as /dev/zram0), you can use it as any normal block device and create a normal filesystem or swap space with it. We'll first make a ext4 filesystem to show off some additional features with zram.

## Make a filesystem and mount it
# mkfs.ext4 /dev/zram0
# mount /dev/zram0 /mnt
# df -h /mnt
Filesystem      Size  Used Avail Use% Mounted on
/dev/zram0      108M   84K   99M   1% /mnt

The actual memory consumption of the block device can be viewed with zramctl. After running dd if=/dev/zero of=test.bin bs=512 count=100000, we now see the filesystem being approximately 50% full whilst only using 4.1KB of actual memory.

## Write all zeros to a file, which should compress very well
# dd if=/dev/zero of=/mnt/test.bin bs=512 count=100000
100000+0 records in
100000+0 records out

## The filesystem is now 50% used
# df -h /mnt
Filesystem      Size  Used Avail Use% Mounted on
/dev/zram0      108M   49M   51M  50% /mnt

## But it's only using 4.1KB of memory
# zramctl
NAME       ALGORITHM DISKSIZE  DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0 lzo           128M 53.9M  4.8K   60K       4 /mnt

You may define an upper-limit to the amount of memory that the zram device can use by writing to /sys/block/zram0/mem_limit. For example, to limit the total memory to 10MB:

## Maximum of 10MB of memory (after compression)
# echo 10m >  /sys/block/zram0/mem_limit

Use this with care since exceeding this value will cause writes to fail and will most likely corrupt the filesystem using this device. Continuing on with the example above, even though /mnt appears to be 128MB in size, because zram0 can only use 10MB of ram, we cannot write data that compresses larger than 10MB. So, if we were to run:

## Writing 50MB of random data
# dd if=/dev/urandom of=test.bin bs=512 count=100000
# df -h /mnt
Filesystem      Size  Used Avail Use% Mounted on
/dev/zram0      108M   49M   51M  50% /mnt

## Actual memory usage is capped to 10MB, so something got truncated...
# zramctl
NAME       ALGORITHM DISKSIZE  DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0 lzo           128M 10.1M   10M   10M       4 /mnt

## Uh-oh
# dmesg | tail
[ 2793.272512] EXT4-fs warning (device zram0): ext4_end_bio:323: I/O error 10 writing to inode 12 (offset 8388608 size 2101248 starting block 7194)
[ 2793.274915] buffer_io_error: 12490 callbacks suppressed
[ 2793.274917] Buffer I/O error on device zram0, logical block 7194
[ 2793.277186] Buffer I/O error on device zram0, logical block 7195
[ 2793.278343] Buffer I/O error on device zram0, logical block 7196
...

Unmounting

You may unmount the filesystem with umount /mnt. To ensure everything is wiped from memory, you can use zramctl --reset /dev/zram0 to zero out the block device.

Using zram as swap space

Continuing on, we can also use it as swap space by first initializing the swap space with mkswap and then enabling the swap space with swapon. We will specify a higher priority to ensure that the system uses the zram swap device before other devices (such as disk which may be slower).

# mkswap /dev/zram0
# swapon /dev/zram0 -p 10

To disable the swap device, use the swapoff command.

# swapoff /dev/zram0

To ensure everything is wiped from memory, you can use zramctl --reset /dev/zram0 to zero out the block device.

Persistent setup using udev

To make the zram swap space come up with the system, you can use a udev rule:

## Load the zram module on startup
# echo "zram" > /etc/modules-load.d/zram.conf

## Create a udev rule that automatically sets up the swap space
# echo 'KERNEL=="zram0", ATTR{disksize}="4G" RUN="/sbin/mkswap /dev/zram0", TAG+="systemd"' > /etc/udev/rules.d/99-zram.rules

## Add the zram block device into fstab
# echo "/dev/zram0 none swap defaults,pri=10 0 0" >> /etc/fstab

Persistent setup using systemd

There is a zram-generator service which you may install which can be used to set up the zram devices. See: https://github.com/systemd/zram-generator/blob/main/README.md

Persistent setup with NixOS

Just add zramSwap.enable = true; to your NixOS configuration in /etc/nixos/configuration.nix. This will create a zram device half the size of your system memory by default.

If you need to adjust the size of the zram device from the default 50% of the installed system memory, set the memoryPercent parameter: zramSwap.memoryPercent = 100;

A word of caution

Because zram allocates memory as pages start swapping in, you have to be careful and ensure your system doesn't run out of memory. If you do run out of memory and zram is unable to allocate additional memory, the block device will become unwritable and you may see filesystem or swap space corruption. You can verify this scenario by looking at your kernel logs (by running dmesg) and look for zram complaining about 'Cannot allocate memory'.

In the best case, you'll see your filesystem error out as you attempt to write a file. In the worst case, your system starts locking up as it's unable to swap pages in and out from the device.

If you're using zram as swap and run into this situation, you'll likely need to forcibly reboot the server. Next, you need to reassess the size of the zram block device you created (is it too big for the amount of memory your system has or for the amount of memory pressure your system is experiencing?) and possibly look into another solution such as zswap which can swap pages to disk as a last resort.

Tasks

Check on zram statistics

Use the zramctl command to show the state of zram:

# zramctl
NAME       ALGORITHM DISKSIZE  DATA  COMPR  TOTAL STREAMS MOUNTPOINT
/dev/zram0 lzo-rle         4G  2.1G 636.4M 677.1M       8 [SWAP]

Columns are:

DISKSIZE - this device will store up to this size
DATA - amount of uncompressed data stored
COMPR - amount of compressed data stored
TOTAL - total memory used, including metadata.
STREAMS - number of threads in use to support this device. Should be equal to number of CPUs.

In this case, we swapped out 2.1G which only uses 677MB which means we saved ~1.4GB of memory.

Show available compression algorithms

The algorithm that is available can be listed from the comp_algorithm pseudofile.

# cat /sys/block/zram0/comp_algorithm
lzo [lzo-rle] lz4 lz4hc 842 zstd

As per this Reddit post, you can see the differences in compression ratios.

Clearing the zram block device

Run: zramctl --reset /dev/zram0

Optimizing and tuning swap on zram

The ArchLinux wiki suggests the following kernel options (set in /etc/sysctl.d/99-vm-zram-parameters.conf):

vm.swappiness = 180
vm.watermark_boost_factor = 0
vm.watermark_scale_factor = 125
vm.page-cluster = 0

Based on what I was able to find, the reasons are:

Higher swappiness is good for fast swap devices like zram. Lower swappiness should only be used for slower IO devices.
Small page-cluster helps with higher IOPs
The watermark_boost_factor controls the level of reclaim when memory is being fragmented. 0 disables this feature.

Resizing the zram block device

You will have to stop the zram device in use first before you can change its size.

# swapoff -a
# zramctl -s 8G zram0
# swapon -a

Changing the zram compression algorithm

You will have to stop the zram device in use before you can change its algorithm. You can find all the supported algorithms with cat /sys/block/zram0/comp_algorithm.

You must also specify the thread count (-t) as well as the size.

# swapoff -a
# zramctl -a zstd -t 8 -s 8G zram0
# swapon -a

If you're using udev, you will also want to change the udev rule:

KERNEL=="zram0", ATTR{comp_algorithm}="zstd", ATTR{disksize}="8G", ATTR{max_comp_streams}="8", RUN="/sbin/mkswap /dev/zram0", TAG+="systemd"

You can test the udev rule change by running:

# rmmod zram
# udevadm control --reload
# modprobe zram
## This should automatically trigger the udev rule. If it doesn't work, you can also trigger the udev rule with:
## udevadm trigger