Linux block device in ram

zram

zram, formerly called compcache, is a Linux kernel module for creating a compressed block device in RAM, i.e. a RAM disk with on-the-fly disk compression. The block device created with zram can then be used for swap or as a general-purpose RAM disk. The two most common uses for zram are for the storage of temporary files ( /tmp ) and as a swap device. Initially, zram had only the former function, hence the original name «compcache» («compressed cache»).

Usage as swap

Initially the created zram block device does not reserve or use any RAM. Only as files need or want to be swapped out, they will be compressed and moved into the zram block device. The zram block device will then dynamically grow or shrink as required.

Example (System has 32 GiB RAM, zram is configured with 16 GiB RAM, we assume a 1:4 compression ratio):

• Worst case (RAM and zram completely «filled»): 16 GiB RAM + 64 GiB zram
• Normal usage without swapping: 32 GiB RAM + 0 GiB zram
• Normal usage with light swapping: 30 GiB RAM + 8 GiB zram
• Without any zram configuration: 32 GiB RAM

Therefore, zram always offers the advantage of being able to store more content in RAM.

• If the related zswap kernel feature remains enabled, it will prevent zram from being effectively used. zswap would function as a swap cache in front of zram, intercepting and compressing evicted memory pages before they can reach zram. Despite the output of zramctl(8) , most of zswap is unused in this circumstance.
• When configuring zram, the size of the zram device controls the maximum uncompressed amount of data it can store, not the maximum compressed size. You can configure the zram’s size to be equal to or even greater than your system’s physical RAM capacity.
• Hibernating to swap on zram is not supported, even when zram is configured with a backing device on permanent storage. logind will protect against trying to hibernate to a swap space on zram.

Manually

# echo 0 > /sys/module/zswap/parameters/enabled

Then, to set up one zstd compressed zram device with 32GiB capacity and a higher-than-normal priority (only for the current session):

# modprobe zram # zramctl /dev/zram0 --algorithm zstd --size 32G # mkswap -U clear /dev/zram0 # swapon --priority 100 /dev/zram0

To disable it again, either reboot or run:

# swapoff /dev/zram0 # modprobe -r zram

A detailed explanation of all steps, options and potential problems is provided in the official documentation of the zram module.

For a permanent solution, use a method from one of the following sections.

Using a udev rule

The example below describes how to set up swap on zram automatically at boot with a single udev rule. No extra package should be needed to make this work.

Before starting, permanently disable zswap using the kernel parameter or sysctl knob.

Create the following udev rule adjusting the disksize attribute as necessary:

ACTION=="add", KERNEL=="zram0", ATTR="zstd", ATTR="4G", RUN="/usr/bin/mkswap -U clear /dev/%k", TAG+="systemd"

Add /dev/zram to your fstab with a higher than default priority:

/dev/zram0 none swap defaults,pri=100 0 0

Note: The swap space on zram cannot be referred to by its LABEL or UUID in fstab since udev does not create /dev/disk/by-label/* and /dev/disk/by-uuid/* symlinks for zram devices.

Using zram-generator

zram-generator provides a systemd-zram-setup@.service unit to automatically initialize zram devices without users needing to enable/start the template or its instances. See zram-generator(8) and zram-generator.conf(5) .

Before starting, permanently disable zswap using the kernel parameter or sysctl knob. [1]

Then, to create a zram swap device using zstd and half of the entire available ram, install zram-generator , then create /etc/systemd/zram-generator.conf with the following:

/etc/systemd/zram-generator.conf

[zram0] zram-size = ram / 2 compression-algorithm = zstd swap-priority = 100 fs-type = swap

Run daemon-reload, then start your configured systemd-zram-setup@zramN.service instance(s).

You can check the swap status of your configured /dev/zramN devices by reading the unit status of your systemd-zram-setup@zramN.service instance(s), or by using zramctl(8) .

Using zramswap

zramswap AUR provides an automated script for setting up a swap with a higher priority and a default size of 20% of the RAM size of your system. To do this automatically on every boot, enable zramswap.service .

Using zramd

zramd AUR allows to setup zram automatically using zstd compression by default, its configuration can be changed at /etc/default/zramd . It can be started at boot by enabling the zramd.service unit.

Tips and tricks

Checking zram statistics

NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT /dev/zram0 zstd 32G 1.9G 318.6M 424.9M 16 [SWAP]

• DISKSIZE = 32G: this zram device will store up to 32G of uncompressed data.
• DATA = 1.9G: currently, 1.9G (uncompressed) of data is being stored in this zram device
• COMPR = 318.6M: the 1.9G uncompressed data was compressed to 318.6M
• TOTAL = 424.9M: including metadata, the 1.9G of uncompressed data is using up 424.9M of physical RAM

Multiple zram devices

By default, loading the zram module creates a single /dev/zram0 device.

If you need more than one /dev/zram device, specify the amount using the num_devices kernel module parameter or add them as needed afterwards.

Optimizing swap on zram

Since zram behaves differently than disk swap, we can configure the system’s swap to take full potential of the zram advantages:

/etc/sysctl.d/99-vm-zram-parameters.conf

vm.swappiness = 180 vm.watermark_boost_factor = 0 vm.watermark_scale_factor = 125 vm.page-cluster = 0

Explanation of the configuration:

These values are what Pop!_OS uses. That Pop!_OS GitHub PR also links to some testing done by users on r/Fedora, which determined that vm.page-cluster = 0 is ideal. They also found a high swappiness value to be ideal, which matches what is suggested by the kernel docs:

«The default value is 60. For in-memory swap, like zram or zswap, as well as hybrid setups that have swap on faster devices than the filesystem, values beyond 100 can be considered. For example, if the random IO against the swap device is on average 2x faster than IO from the filesystem, swappiness should be 133 (x + 2x = 200, 2x = 133.33).»

On a system with a hard drive, random IO against the zswap device would be orders of magnitude faster than IO against the filesystem, so swappiness should be ~200. Even on a system with a fast SSD, a high swappiness value may be ideal.

See also

Источник

Using the RAM disk block device with Linux¶

The RAM disk driver is a way to use main system memory as a block device. It is required for initrd, an initial filesystem used if you need to load modules in order to access the root filesystem (see Using the initial RAM disk (initrd) ). It can also be used for a temporary filesystem for crypto work, since the contents are erased on reboot.

The RAM disk dynamically grows as more space is required. It does this by using RAM from the buffer cache. The driver marks the buffers it is using as dirty so that the VM subsystem does not try to reclaim them later.

The RAM disk supports up to 16 RAM disks by default, and can be reconfigured to support an unlimited number of RAM disks (at your own risk). Just change the configuration symbol BLK_DEV_RAM_COUNT in the Block drivers config menu and (re)build the kernel.

To use RAM disk support with your system, run ‘./MAKEDEV ram’ from the /dev directory. RAM disks are all major number 1, and start with minor number 0 for /dev/ram0, etc. If used, modern kernels use /dev/ram0 for an initrd.

The new RAM disk also has the ability to load compressed RAM disk images, allowing one to squeeze more programs onto an average installation or rescue floppy disk.

2) Parameters¶

2a) Kernel Command Line Parameters

ramdisk_size=N

Size of the ramdisk.

This parameter tells the RAM disk driver to set up RAM disks of N k size. The default is 4096 (4 MB).

rd_nr

/dev/ramX devices created.

max_part

Maximum partition number.

rd_size

See ramdisk_size.

3) Using «rdev»¶

«rdev» is an obsolete, deprecated, antiquated utility that could be used to set the boot device in a Linux kernel image.

Instead of using rdev, just place the boot device information on the kernel command line and pass it to the kernel from the bootloader.

You can also pass arguments to the kernel by setting FDARGS in arch/x86/boot/Makefile and specify in initrd image by setting FDINITRD in arch/x86/boot/Makefile.

Some of the kernel command line boot options that may apply here are:

ramdisk_start=N ramdisk_size=M

If you make a boot disk that has LILO, then for the above, you would use:

append = "ramdisk_start=N ramdisk_size=M"

4) An Example of Creating a Compressed RAM Disk¶

To create a RAM disk image, you will need a spare block device to construct it on. This can be the RAM disk device itself, or an unused disk partition (such as an unmounted swap partition). For this example, we will use the RAM disk device, «/dev/ram0».

Note: This technique should not be done on a machine with less than 8 MB of RAM. If using a spare disk partition instead of /dev/ram0, then this restriction does not apply.

Decide on the RAM disk size that you want. Say 2 MB for this example. Create it by writing to the RAM disk device. (This step is not currently required, but may be in the future.) It is wise to zero out the area (esp. for disks) so that maximal compression is achieved for the unused blocks of the image that you are about to create:

dd if=/dev/zero of=/dev/ram0 bs=1k count=2048

dd if=/dev/ram0 bs=1k count=2048 | gzip -v9 > /tmp/ram_image.gz

dd if=zImage of=/dev/fd0 bs=1k

dd if=/tmp/ram_image.gz of=/dev/fd0 bs=1k seek=400

That is it. You now have your boot/root compressed RAM disk floppy. Some users may wish to combine steps (d) and (f) by using a pipe.

Changelog:¶

Updated to reflect changes in command line options, remove obsolete references, general cleanup. James Nelson (james4765 @ gmail . com)

Источник

Create a block device in RAM

Now, all of this works. The obvious improvement is to create a script that would do this all automatically. This can also be done, but one manual step will still remain: insert the USB pendrive.

I am thinking instead of the actual USB disk we could use a block device in RAM. This not only would eliminate the manual step of inserting the USB disk, but would also be faster.

So what I need is a plain block device in RAM. One that can be split into partitions just like a physical USB pendrive can.

I just researched this topic and seems like the only options are

1. tmpfs and ramfs, which obviously don’t suit me
2. ramdisk and the ‘brd’ kernel module, which, when loaded, creates a number of ‘partitions’: /dev/ram4

and each of them can individually be formatted and used, but like I said — I want a plain block device in RAM which I would then split into two partitions of 10MB and 500MB myself, and from where I would be then able to dd an image to a file on my HDD.

3 Answers 3

Just use brd and create one brd device (ram0). Use that device in place of your usb drive. You can partition it using sfdisk, use the partitions and then use dd to dump its contents to file.

There is no need to use one filesystem per brd device.

Or (though little hacky) you can use tmpfs, create image file and use that as loop device. That might be the easiest way to accomplish what you want. As a bonus, you have that image ready and can upload it straight away. No need to dd.

# Create mountpoint for tmpfs mkdir /tmp/tmpfs # Mount tmpfs there mount -t tmpfs none /tmp/tmpfs # Create empty file of 600MB # (it creates 599MB hole, so it does not # consume more memory than needed) dd if=/dev/zero of=/tmp/tmpfs/img.bin bs=1M seek=599 count=1 # Partition the image file cfdisk /tmp/tmpfs/img.bin # Create loop block device of it (-P makes kernel look for partitions) losetup -P /dev/loop0 /tmp/tmpfs/img.bin # Create filesystems mkfs.vfat /dev/loop0p1 mkfs.ext4 /dev/loop0p2 # Now it's your turn: # mount loop0p1 and loop0p2 and copy whatever you want and unmount it # detach the loop device losetup -d /dev/loop0 # May i present you with your image . ls -al /tmp/tmpfs/img.bin 

Modify to suit your needs.

Источник