Linux clear disk cache

How to clear the buffer/pagecache (disk cache) under Linux

Are you facing a performance issue and you suspect it might be related to cache usage? High cache usage should not normally cause performance issues, but it might be the root cause in some rare cases.

What is Memory Cache

In order to speed operations and reduce disk I/O, the kernel usually does as much caching as it has memory By design, pages containing cached data can be repurposed on-demand for other uses (e.g., apps) Repurposing memory for use in this way is no slower than claiming pristine untouched pages.

What is the purpose of /proc/sys/vm/drop_caches

Writing to /proc/sys/vm/drop_caches allows one to request the kernel immediately drop as much clean cached data as possible. This will usually result in some memory becoming more obviously available; however, under normal circumstances, this should not be necessary.

How to clear the Memory Cache using /proc/sys/vm/drop_caches

Writing the appropriate value to the file /proc/sys/vm/drop_caches causes the kernel to drop clean caches, dentries and inodes from memory, causing that memory to become free.

1. In order to clear PageCache only run:

# sync; echo 1 > /proc/sys/vm/drop_caches

2. In order to clear dentries (Also called as Directory Cache) and inodes run:

# sync; echo 2 > /proc/sys/vm/drop_caches

3. In order to clear PageCache, dentries and inodes run:

# sync; echo 3 > /proc/sys/vm/drop_caches

Running sync writes out dirty pages to disks. Normally dirty pages are the memory in use, so they are not available for freeing. So, running sync can help the ensuing drop operations to free more memory.

Page cache is memory held after reading files. Linux kernel prefers to keep unused page cache assuming files being read once will most likely to be read again in the near future, hence avoiding the performance impact on disk IO.

dentry and inode_cache are memory held after reading directory/file attributes, such as open() and stat(). dentry is common across all file systems, but inode_cache is on a per-file-system basis. Linux kernel prefers to keep this information assuming it will be needed again in the near future, hence avoiding disk IO.

Note: Starting with the sync command as shown in the above 3 commands is optional. The sync command allows the kernel write as many dirty cache pages to disk as it can (to maximize the number of data cache pages that can be dropped)

How to clear the Memory Cache using sysctl

You can also Trigger cache-dropping by using sysctl -w vm.drop_caches=[number] command.

1. To free pagecache, dentries and inodes, use the below command.

2. To free dentries and inodes only, use the below command.

3. To free the pagecache only, use the below command.

Note: Using vm.drop_caches can cause a deadlock if the system is under heavy memory and I/O load.

“Clean” cached data is eligible for dropping. “Dirty” cached data needs to be written somewhere. Using vm.drop_caches will never trigger the kernel to drop dirty cache.

Some more articles you might also be interested in …

Источник

How to purge disk I/O caches on Linux?

Sounds like you want the sync command, or the sync() function.

If you want disk cache flushing: echo 3 | sudo tee /proc/sys/vm/drop_caches

sync is 100% unrelated. I’m talking about long-lived multi-GB read caches, not trivial amounts of short-lived unwritten data which sync deals with (and which gets written to disk every 10 or so seconds anyway).

Actually even though you tell the OS to drop the caches, the hard drive doesn’t have to 🙂 The only way to force this to happen is to power down the machine, found this out the hard way (on disk cache)

@ChrisDennett I do the same thing when my code doesn’t compile, just to make sure the compiler knows what I’m doing, maybe it will get the message too

Please consider switching the 2 sentences of your answer. The drop_caaches part is what I wanted when I came to this page.

# sync # (move data, modified through FS -> HDD cache) + flush HDD cache # echo 3 > /proc/sys/vm/drop_caches # (slab + pagecache) -> HDD (https://www.kernel.org/doc/Documentation/sysctl/vm.txt) # blockdev --flushbufs /dev/sda # hdparm -F /dev/sda # NEXT COMMAND IS NOT FOR BENCHMARKING: # should be run before unplug, flushes everything possible guaranteed. # echo 1 > /sys/block/sdX/device/delete 

You may use strace to see that these are three different syscalls

Also, it may be desirable to turn off HDD cache using hdparm, not sure what thing you benchmarking.

In any way, you cannot prevent HDD to cache last 64/32/16 MB of recently used data. In order to kill that cache, just write some amount of zeroes (and flush) + read some unrelated place from HDD. This is required since cache may be divided to read-part and write-part. After that you can benchmark HDD.

blockdev —flushbufs /dev/sda works with my USB drive, but has no effect with SATA SSD drive. echo 3 | sudo tee /proc/sys/vm/drop_caches works with both drives.

The last command removes the disk from the kernel. You definitely won’t be able to benchmark the disk after (or use the disk in any way) and if the disk is in use it’s dangerous. There’s no reason to include it along with the other commands. Here’s an example of when to use this command and what it does: access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/…

Disk cache purging: echo 3 | sudo tee /proc/sys/vm/drop_caches

Writing to this will cause the kernel to drop clean caches, dentries and inodes from memory, causing that memory to become free.

To free pagecache:

echo 1 > /proc/sys/vm/drop_caches

To free dentries and inodes:

echo 2 > /proc/sys/vm/drop_caches

To free pagecache, dentries and inodes:

echo 3 > /proc/sys/vm/drop_caches

As this is a non-destructive operation, and dirty objects are not freeable, the user should run «sync» first in order to make sure all cached objects are freed.

Short good enough answer: (copy paste friendly)

DISK=/dev/sdX # /proc/sys/vm/drop_caches blockdev --flushbufs $DISK hdparm -F $DISK 

Explanation:

sync : From the man page: flush file system buffers. Force changed blocks to disk, update the super block.

echo 3 > /proc/sys/vm/drop_cache : from the kernel docs this will cause the kernel to drop clean caches

blockdev —flushbufs /dev/sda : from the man page: call block device ioctls [to] flush buffers.

hdparm -F /dev/sda : from the man page: Flush the on-drive write cache buffer (older drives may not implement this)

Although the blockdev and hdparm commands look similar according to an answer above they issue different ioctls to the device.

Long probably better way:

(I’ll assume that you have formatted the disk but you can adapt these commands if you want to write directly to the disk)

Run this only once before the 1st benchmark:

Run this every time you want to empty the caches:

DISK=/dev/sdX # /proc/sys/vm/drop_caches blockdev --flushbufs $DISK hdparm -F $DISK # read the file with pseudo-random data to fill any read-cache # the disk may have with garbage dd if=$MOUNT/temp-hddread.tmp of=/dev/null 

Explanation:

The disk will probably have some H/W cache. Some disks by design or due to bugs may not clear their caches when you issue the blockdev and hdparm commands. To compensate we write and read pseudo-random data hopping to fill these caches so that any cached data are removed from them. How much data you need to fill the cache depends on its size. In the commands above I’m using dd to read/write 16*64MB=1024MB, adjust the arguments if your HDD may have bigger cache (data sheets and experimentation are your friend and it doesn’t hurt to specify values above the actual size of the cache). I’m using /dev/urandom as a source for random data because it’s fast and we don’t care about true randomness (we only care for high entropy because the disk firmware may be using compression before storing data to the cache). I’m creating /mnt/test/temp-hddread.tmp from the start and use it every time I want to read enough random data. I’m creating and deleting /mnt/test/temp-hddwrite.tmp each time I want to write enough random data.

I’ve wrote this answer based on the best parts of the existing answers.

Источник

Очищаем буфер и кеш в системе Linux

Прежде чем приступить к практике, сперва поговорим о буферизации и кэш памяти в операционных системах Linux. Очистка кешей и буферов в Linux снижает производительность системы и в основном полезно только для бенчмаркинга.

Свободная память – неиспользуемая память – это потраченная память впустую.

Очистка кеша и буфер в Linux

Если вы хотите очистить в системе кеш и буфер, то вы можете использовать эту цепочку команд:

free && sync && echo 3 > /proc/sys/vm/drop_caches && free

total used free shared buff/cache available Mem: 24729752 12590140 2704836 354028 9434776 11361684 Swap: 8388604 27592 8361012 total used free shared buff/cache available Mem: 24729752 12555168 10743364 354028 1431220 11397284 Swap: 8388604 27592 8361012 

Вы можете сигнализировать ядру Linux, чтобы удалить различные аспекты кешированных элементов, изменив числовой аргумент на указанную выше команду.

echo 1 > /proc/sys/vm/drop_caches

echo 2 > /proc/sys/vm/drop_caches

echo 3 > /proc/sys/vm/drop_caches

ПРИМЕЧАНИЕ. Вышеупомянутые команды должны выполняться от root пользователя.

Если вы пытаетесь сделать это с помощью sudo , вам нужно немного изменить синтаксис, примерно так:

sudo sh -c 'echo 1 >/proc/sys/vm/drop_caches' sudo sh -c 'echo 2 >/proc/sys/vm/drop_caches' sudo sh -c 'echo 3 >/proc/sys/vm/drop_caches'

echo "echo 1 > /proc/sys/vm/drop_caches" | sudo sh echo "echo 2 > /proc/sys/vm/drop_caches" | sudo sh echo "echo 3 > /proc/sys/vm/drop_caches" | sudo sh

Увидеть, что находится в буферах и кеше

Посмотрите linux-ftools , если вы хотите анализировать содержимое буферов и кэша в вашей системе. В частности, если вы хотите посмотреть, какие файлы в настоящее время кэшируются.

fincore

С помощью этого инструмента вы можете видеть, какие файлы кэшируются в директории give.

fincore [options] files. Options: -J, --json use JSON output format -b, --bytes print sizes in bytes rather than in human readable format -n, --noheadings don't print headings -o, --output output columns -r, --raw use raw output format -h, --help display this help -V, --version display version

Очистка swap в Linux

первым делом смотрим сколько занимает наш swap места:

 total used free shared buff/cache available Mem: 24150 12258 10391 356 1500 11121 Swap: 8191 26 8191 

В данном случае swap занимает 26 Мб из 8 Гб

Очистим swap следующей командой:

sudo swapoff -a && sudo swapon -a

Затем используйте команду, чтобы посмотреть освободился ли swap:

 total used free shared buff/cache available Mem: 24150 12258 10391 356 1500 11121 Swap: 8191 0 8191

Из вывода видно что swap теперь занимает 0 Мб.

Очищаем сетевой кеш samba

Для очистки сетевого кэша Samba воспользуемся следующей командой:

Очистка memcached в Linux

Есть несколько путей очистить кеша memcached.

Первый – это очистка с помощью подключения через telnet. Используем следующие команды:

telnet localhost 11211
flush_all
quit

• localhost — хост где находится memcached (можно указывать и IP-адрес хоста).
• 11211 — порт который использует memcached.

Чтобы узнать порт и IP-адрес в терминале где установлен memcached наберите следующее:

netstat -natpl | grep "memcache[d]" 

tcp 0 0 127.0.0.1:11211 0.0.0.0:* LISTEN 755/memcached 

sudo ps aux | grep memcache

memcache 755 0.0 0.2 420652 17608 ? Ssl апр14 2:14 /usr/bin/memcached -m 64 -p 11211 -u memcache -l 127.0.0.1 -P /var/run/memcached/memcached.pid

Второй – перезапустить сервис memcached:

sudo service memcached restart

Третий – установить утилиту memcflush для очистки кешей:

sudo apt install libmemcached-tools -y

yum install libmemcached-* -y

Теперь очищаем кеш memcached

memflush --servers=localhost:11211

Если есть вопросы, то пишем в комментариях.

Также можете вступить в Телеграм канал, ВКонтакте или подписаться на Twitter. Ссылки в шапке страницы.
Заранее всем спасибо.

RSS

Если вы нашли ошибку, пожалуйста, выделите фрагмент текста и нажмите Ctrl+Enter.

Сегодня в статье настроим и русифицируем Ubuntu Server 16.04/18.04/20.04. Чтобы поддерживался русский язык, и перевод системы стал русским

Начиная с сентября 2017 года удостоверяющим центрам предписано обязательно проверять CAA-записи в DNS перед генерацией сертификата

В этой статье рассмотрим пример обновления Ubuntu Server 16.04 до Ubuntu Server 18.04 Все наши действия нам придется выполнять из Читать

В связи с последними блокировками IP-адресов Роскомнадзором, встала необходимость завести свой собственный VPN сервер. Если VPN у вас ещё не Читать

Источник