Zram - Revision history

Leo at 01:46, 18 November 2023

2023-11-18T01:46:34Z

@@ Line 188: / Line 188: @@
 # zramctl -a zstd -t 8 -s 8G zram0
 # swapon -a
 | lang = terminal
 }}

Leo at 01:37, 18 November 2023

2023-11-18T01:37:03Z

@@ Line 170: / Line 170: @@
 * Small page-cluster helps with higher IOPs
 * The watermark_boost_factor controls the level of reclaim when memory is being fragmented. 0 disables this feature.
 == See Also ==
@@ Line 176: / Line 196: @@
 * Swap to zram. More at https://ikarus.sg/using-zram-to-get-more-out-of-your-raspberry-pi/
 * Proxmox's guide to zram is also very useful: https://pve.proxmox.com/wiki/Zram

Leo: NixOS

2023-11-06T17:08:36Z

NixOS

← Older revision		Revision as of 17:08, 6 November 2023
Line 113:		Line 113:
	There is a zram-generator service which you may install which can be used to set up the zram devices. See: https://github.com/systemd/zram-generator/blob/main/README.md		There is a zram-generator service which you may install which can be used to set up the zram devices. See: https://github.com/systemd/zram-generator/blob/main/README.md

	Persistent setup with NixOS		==== Persistent setup with NixOS ====
			Just add <code>zramSwap.enable = true;</code> to your NixOS configuration in <code>/etc/nixos/configuration.nix</code>. This will create a zram device half the size of your system memory by default.

	~~Just add~~ <code>zramSwap.~~enable~~ = ~~true~~;</code> ~~to your NixOS configuration. This will create a zram device half the size of your system memory by default.~~		If you need to adjust the size of the zram device from the default 50% of the installed system memory, set the memoryPercent parameter: <code>zramSwap.memoryPercent = 100;</code>

	=== A word of caution ===		=== A word of caution ===

Leo at 20:44, 18 August 2023

2023-08-18T20:44:59Z

@@ Line 8: / Line 8: @@
 Firstly, load the <code>zram</code> kernel module as root. This should create <code>/dev/zram0</code> which you can configure with <code>zramctl</code>. You may set the size and compression algorithm when creating the block device.
-Supported compression algorithms include: <code>lz4</code> (more efficient compression but slower), <code>lzo</code> (less efficient but faster).
+Supported compression algorithms include: <code>lz4</code> (more efficient compression but slower), <code>lzo</code> (less efficient but faster), zstd. More on [[Zram#Show available compression algorithms|the supported algorithms below]].
 {{Highlight
 | code = ## Load the zram module. It should have created /dev/zram0 by default.
@@ Line 125: / Line 125: @@
 ==Tasks==
 ===Show available compression algorithms===
@@ Line 130: / Line 148: @@
 {{Highlight
 | code = # cat /sys/block/zram0/comp_algorithm
-lzo [lz4]
+lzo [lzo-rle] lz4 lz4hc 842 zstd
 | lang = terminal
-}}
+}}As per [https://old.reddit.com/r/Fedora/comments/mzun99/new_zram_tuning_benchmarks/ this Reddit post], you can see the differences in compression ratios.
 === Clearing the zram block device ===
 Run: <code>zramctl --reset /dev/zram0</code>
 == See Also ==

Leo: /* Persistent setup */

2023-08-18T18:17:12Z

Persistent setup

← Older revision		Revision as of 18:17, 18 August 2023
Line 96:		Line 96:
	To ensure everything is wiped from memory, you can use <code>zramctl --reset /dev/zram0</code> to zero out the block device.		To ensure everything is wiped from memory, you can use <code>zramctl --reset /dev/zram0</code> to zero out the block device.

	==== Persistent setup ====		==== Persistent setup using udev ====
	To make the zram swap space come up with the system, you ~~will have to do~~ a ~~few things~~:		To make the zram swap space come up with the system, you can use a udev rule:
	{{Highlight		{{Highlight
	\| code = ## Load the zram module on startup		\| code = ## Load the zram module on startup
Line 109:		Line 109:
	\| lang = terminal		\| lang = terminal
	}}		}}

			==== Persistent setup using systemd ====
			There is a zram-generator service which you may install which can be used to set up the zram devices. See: https://github.com/systemd/zram-generator/blob/main/README.md

			Persistent setup with NixOS

			Just add <code>zramSwap.enable = true;</code> to your NixOS configuration. This will create a zram device half the size of your system memory by default.

	=== A word of caution ===		=== A word of caution ===

Leo at 18:13, 18 August 2023

2023-08-18T18:13:19Z

← Older revision		Revision as of 18:13, 18 August 2023
Line 89:		Line 89:
	\| lang = terminal		\| lang = terminal
	}}		}}
	To disable the swap device, use the swapoff command.		To disable the swap device, use the <code>swapoff</code> command.
	{{Highlight		{{Highlight
	\| code = # swapoff /dev/zram0		\| code = # swapoff /dev/zram0
Line 109:		Line 109:
	\| lang = terminal		\| lang = terminal
	}}		}}

			=== A word of caution ===
			Because zram allocates memory as pages start swapping in, you have to be careful and ensure your system doesn't run out of memory. If you do run out of memory and zram is unable to allocate additional memory, the block device will become unwritable and you may see filesystem or swap space corruption. You can verify this scenario by looking at your kernel logs (by running <code>dmesg</code>) and look for zram complaining about 'Cannot allocate memory'.

			In the best case, you'll see your filesystem error out as you attempt to write a file. In the worst case, your system starts locking up as it's unable to swap pages in and out from the device.

			If you're using zram as swap and run into this situation, you'll likely need to forcibly reboot the server. Next, you need to reassess the size of the zram block device you created (is it too big for the amount of memory your system has or for the amount of memory pressure your system is experiencing?) and possibly look into another solution such as zswap which can swap pages to disk as a last resort.

	==Tasks==		==Tasks==

Leo at 17:56, 18 August 2023

2023-08-18T17:56:18Z

← Older revision		Revision as of 17:56, 18 August 2023
Line 1:		Line 1:
	==Quick ~~Usage~~==		zram is a Linux kernel module that creates a RAM backed block device with features including on-the-fly compression.
	~~Load~~ the <code>zram</code> kernel module. This should create <code>/dev/zram0</code> which you can configure with <code>zramctl</code>.
			Because zram can transparently compress data, it may be used as a swapfile which effectively allows RAM to be compressed as it is paged out. This is particularly useful for memory constrained systems.

			zswap is closely related with the biggest difference that less frequently used pages can be swapped out from RAM to physical disk.

			==Quick usage guide==
			Firstly, load the <code>zram</code> kernel module as root. This should create <code>/dev/zram0</code> which you can configure with <code>zramctl</code>. You may set the size and compression algorithm when creating the block device.

			Supported compression algorithms include: <code>lz4</code> (more efficient compression but slower), <code>lzo</code> (less efficient but faster).
	{{Highlight		{{Highlight
	\| code = ## Load the zram module. It should have created /dev/zram0 by default.		\| code = ## Load the zram module. It should have created /dev/zram0 by default.
Line 9:		Line 17:
	## Define the disk size and algorithm of zram0		## Define the disk size and algorithm of zram0
	# zramctl --size 128m --algorithm lz4 /dev/zram0		# zramctl --size 128m --algorithm lz4 /dev/zram0
			\| lang = terminal
			}}

	## Make a filesystem and mount it		=== Using zram with a Ext4 filesystem ===
			Once the block device is created (as <code>/dev/zram0</code>), you can use it as any normal block device and create a normal filesystem or swap space with it. We'll first make a ext4 filesystem to show off some additional features with zram.
			{{Highlight
			\| code = ## Make a filesystem and mount it
	# mkfs.ext4 /dev/zram0		# mkfs.ext4 /dev/zram0
	# mount /dev/zram0 /mnt		# mount /dev/zram0 /mnt
Line 18:		Line 31:
	\| lang = terminal		\| lang = terminal
	}}		}}
	~~The size of the <code>zram0</code> device is its blocksize which <code>df</code> reports is around 128MB (minus ext4 metadata and other overheads).~~ The actual memory consumption of the block device can be viewed with <code>zramctl</code>. After running <code>dd if=/dev/zero of=test.bin bs=512 count=100000</code>, we now see the filesystem being approximately 50% full whilst only using 4.1KB of actual memory.		The actual memory consumption of the block device can be viewed with <code>zramctl</code>. After running <code>dd if=/dev/zero of=test.bin bs=512 count=100000</code>, we now see the filesystem being approximately 50% full whilst only using 4.1KB of actual memory.
	{{Highlight		{{Highlight
	\| code = ## Write all zeros to a file, which should compress very well		\| code = ## Write all zeros to a file, which should compress very well
Line 64:		Line 77:
	...		...
	\| lang = terminal		\| lang = terminal
	}}<br />		}}

			==== Unmounting ====
			You may unmount the filesystem with <code>umount /mnt</code>. To ensure everything is wiped from memory, you can use <code>zramctl --reset /dev/zram0</code> to zero out the block device.

			=== Using zram as swap space ===
			Continuing on, we can also use it as swap space by first initializing the swap space with <code>mkswap</code> and then enabling the swap space with <code>swapon</code>. We will specify a higher priority to ensure that the system uses the zram swap device before other devices (such as disk which may be slower).
			{{Highlight
			\| code = # mkswap /dev/zram0
			# swapon /dev/zram0 -p 10
			\| lang = terminal
			}}
			To disable the swap device, use the swapoff command.
			{{Highlight
			\| code = # swapoff /dev/zram0
			\| lang = terminal
			}}
			To ensure everything is wiped from memory, you can use <code>zramctl --reset /dev/zram0</code> to zero out the block device.

			==== Persistent setup ====
			To make the zram swap space come up with the system, you will have to do a few things:
			{{Highlight
			\| code = ## Load the zram module on startup
			# echo "zram" > /etc/modules-load.d/zram.conf

			## Create a udev rule that automatically sets up the swap space
			# echo 'KERNEL=="zram0", ATTR{disksize}="4G" RUN="/sbin/mkswap /dev/zram0", TAG+="systemd"' > /etc/udev/rules.d/99-zram.rules

			## Add the zram block device into fstab
			# echo "/dev/zram0 none swap defaults,pri=10 0 0" >> /etc/fstab
			\| lang = terminal
			}}

	==Tasks==		==Tasks==

	===~~Available~~ compression algorithms===		===Show available compression algorithms===
	The algorithm that is available can be listed from the <code>comp_algorithm</code> pseudofile.		The algorithm that is available can be listed from the <code>comp_algorithm</code> pseudofile.
	{{Highlight		{{Highlight
Line 76:		Line 120:
	}}		}}

	===~~Mapping swap to~~ zram===		=== Clearing the zram block device ===
	~~You can try to 'increase' your system's memory by putting your swap file in zram.~~		Run: <code>zramctl --reset /dev/zram0</code>

	<br />

	== See Also ==		== See Also ==
Line 85:		Line 127:
	* Log2ram, which is a script that mounts zram/tmpfs for logs: https://github.com/azlux/log2ram/blob/master/log2ram		* Log2ram, which is a script that mounts zram/tmpfs for logs: https://github.com/azlux/log2ram/blob/master/log2ram
	* Swap to zram. More at https://ikarus.sg/using-zram-to-get-more-out-of-your-raspberry-pi/		* Swap to zram. More at https://ikarus.sg/using-zram-to-get-more-out-of-your-raspberry-pi/
			* Proxmox's guide to zram is also very useful: https://pve.proxmox.com/wiki/Zram

Leo: Added categories and navbox

2021-12-30T01:12:37Z

Added categories and navbox

← Older revision		Revision as of 01:12, 30 December 2021
Line 85:		Line 85:
	* Log2ram, which is a script that mounts zram/tmpfs for logs: https://github.com/azlux/log2ram/blob/master/log2ram		* Log2ram, which is a script that mounts zram/tmpfs for logs: https://github.com/azlux/log2ram/blob/master/log2ram
	* Swap to zram. More at https://ikarus.sg/using-zram-to-get-more-out-of-your-raspberry-pi/		* Swap to zram. More at https://ikarus.sg/using-zram-to-get-more-out-of-your-raspberry-pi/


			{{Navbox Linux}}
			[[Category:Linux]]
			[[Category:LinuxUtilities]]

Leo at 02:10, 16 February 2021

2021-02-16T02:10:30Z

← Older revision		Revision as of 02:10, 16 February 2021
Line 1:		Line 1:
	== Quick Usage ==		==Quick Usage==
	Load the <code>zram</code> kernel module. This should create <code>/dev/zram0</code> which you can configure with <code>zramctl</code>.		Load the <code>zram</code> kernel module. This should create <code>/dev/zram0</code> which you can configure with <code>zramctl</code>.
	{{Highlight		{{Highlight
Line 66:		Line 66:
	}}<br />		}}<br />

	== Tasks ==		==Tasks==

	=== Available compression algorithms ===		===Available compression algorithms===
	The algorithm that is available can be listed from the <code>comp_algorithm</code> pseudofile.		The algorithm that is available can be listed from the <code>comp_algorithm</code> pseudofile.
	{{Highlight		{{Highlight
Line 76:		Line 76:
	}}		}}

	=== Mapping swap to zram ===		===Mapping swap to zram===
	You can try to 'increase' your system's memory by putting your swap file in zram.		You can try to 'increase' your system's memory by putting your swap file in zram.

	See: https://ikarus.sg/using-zram-to-get-more-out-of-your-raspberry-pi/		<br />

			== See Also ==

			* Log2ram, which is a script that mounts zram/tmpfs for logs: https://github.com/azlux/log2ram/blob/master/log2ram
			* Swap to zram. More at https://ikarus.sg/using-zram-to-get-more-out-of-your-raspberry-pi/

Leo: Initial Content

2021-02-16T02:05:24Z

Initial Content

New page

== Quick Usage ==
Load the <code>zram</code> kernel module. This should create <code>/dev/zram0</code> which you can configure with <code>zramctl</code>.
{{Highlight
| code = ## Load the zram module. It should have created /dev/zram0 by default.
# modprobe zram
# ls -al /dev/zram*
/dev/zram0

## Define the disk size and algorithm of zram0
# zramctl --size 128m --algorithm lz4 /dev/zram0

## Make a filesystem and mount it
# mkfs.ext4 /dev/zram0
# mount /dev/zram0 /mnt
# df -h /mnt
Filesystem Size Used Avail Use% Mounted on
/dev/zram0 108M 84K 99M 1% /mnt
| lang = terminal
}}
The size of the <code>zram0</code> device is its blocksize which <code>df</code> reports is around 128MB (minus ext4 metadata and other overheads). The actual memory consumption of the block device can be viewed with <code>zramctl</code>. After running <code>dd if=/dev/zero of=test.bin bs=512 count=100000</code>, we now see the filesystem being approximately 50% full whilst only using 4.1KB of actual memory.
{{Highlight
| code = ## Write all zeros to a file, which should compress very well
# dd if=/dev/zero of=/mnt/test.bin bs=512 count=100000
100000+0 records in
100000+0 records out

## The filesystem is now 50% used
# df -h /mnt
Filesystem Size Used Avail Use% Mounted on
/dev/zram0 108M 49M 51M 50% /mnt

## But it's only using 4.1KB of memory
# zramctl
NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0 lzo 128M 53.9M 4.8K 60K 4 /mnt
| lang = terminal
}}
You may define an upper-limit to the amount of memory that the zram device can use by writing to <code>/sys/block/zram0/mem_limit</code>. For example, to limit the total memory to 10MB:
{{Highlight
| code = ## Maximum of 10MB of memory (after compression)
# echo 10m > /sys/block/zram0/mem_limit
| lang = terminal
}}
Use this with care since exceeding this value will cause writes to fail and will most likely corrupt the filesystem using this device. Continuing on with the example above, even though <code>/mnt</code> appears to be 128MB in size, because zram0 can only use 10MB of ram, we cannot write data that compresses larger than 10MB. So, if we were to run:
{{Highlight
| code = ## Writing 50MB of random data
# dd if=/dev/urandom of=test.bin bs=512 count=100000
# df -h /mnt
Filesystem Size Used Avail Use% Mounted on
/dev/zram0 108M 49M 51M 50% /mnt

## Actual memory usage is capped to 10MB, so something got truncated...
# zramctl
NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0 lzo 128M 10.1M 10M 10M 4 /mnt

## Uh-oh
# dmesg {{!}} tail
[ 2793.272512] EXT4-fs warning (device zram0): ext4_end_bio:323: I/O error 10 writing to inode 12 (offset 8388608 size 2101248 starting block 7194)
[ 2793.274915] buffer_io_error: 12490 callbacks suppressed
[ 2793.274917] Buffer I/O error on device zram0, logical block 7194
[ 2793.277186] Buffer I/O error on device zram0, logical block 7195
[ 2793.278343] Buffer I/O error on device zram0, logical block 7196
...
| lang = terminal
}}<br />

== Tasks ==

=== Available compression algorithms ===
The algorithm that is available can be listed from the <code>comp_algorithm</code> pseudofile.
{{Highlight
| code = # cat /sys/block/zram0/comp_algorithm
lzo [lz4]
| lang = terminal
}}

=== Mapping swap to zram ===
You can try to 'increase' your system's memory by putting your swap file in zram.

See: https://ikarus.sg/using-zram-to-get-more-out-of-your-raspberry-pi/