How do I minimize disk space usage
One of my machines is the 2GB EeePC Surf, a neat netbook with very limited resources. So limited that right now, I have 22MB free space left. On it, I’m running Arch Linux with the Openbox DE and a host of needed applications for it to function as a mobile PC. What methods are available to stamp out some unnecessary used space?
@Stefan also. if you’re willing to go the experimental route. btrfs supposedly supports a compressed file system (or is going to?) maybe check that out
@Stefan so it is. oops. of you can cd / and run du -sh * and drill down. (you don’t really have enough ram to store stuff in it)
5 Answers 5
Here are some points you could start with:
- Have a look at the packages installed on your system with pacman -Q and remove the ones you don’t need. A good start may be to append the -t switch:
pacman -cc to clean the entire cache including installed packages I think. Not normally recommended as you may want those as backups later.
@xeno, I already mount a diretory on my desktop for —cachedir , so I don’t need to keep any pkg files.
For clearing space on / , I found pacgraph really useful. It can sort your installed packages by size, so you can better prioritise what it makes sense to remove. (It can even draw SVG dependency graphs, but is useful from just the command line.)
ncdu is a cool command-line alternative to xdiskusage . Baobab is by far the best graphical app for this purpose though.
WARNING: These ideas are only for users who are well-versed in both Linux as a whole and Arch Linux.
If you’re willing to tread into dangerous territory, you can slim a base Arch install down to less than 500MB installed. This requires doing some very dangerous things:
- removing all unnecessary locales (already covered)
- removing any firmware files not needed to run your system (from /usr/lib/firmware )
- removing any kernel modules not needed to run your system (from /usr/lib/modules/. )
- removing any .a files in /usr/lib (only if you never use the system to compile software. note: this includes using makepkg )
- removing everything in /usr/include (only if you never use the system to compile software)
- removing unneeded documentation from /usr/share/doc and /usr/share/info
- (VERY BAD IDEA unless maybe for a server) removing man pages from /usr/share/man
- (also a bad idea) removing unneeded terminal descriptors from /usr/share/terminfo and unneeded timezone files from /usr/share/zoneinfo
- (DANGEROUS) running strip * on all folders containing executable binaries ( /usr/bin and /usr/sbin )
- (in extreme situations) using a tool such as upx to compress larger binaries (the Samba binaries lend themselves to this well as they tend to be quite large since they are often compiled statically.) Also note that using upx means the entire uncompressed binary must fit in RAM during execution, so be weary on systems with low RAM.
Before you do ANY of this, MAKE A FULL BACKUP of your system. Linux thankfully makes this relatively easy — if you can attach and mount an external volume (e.g. a USB drive) you can do something like cd / && tar -cf /mnt/usb/mySystem.tar / to backup the entire system.
Once again, note that I do not actually recommend doing the above (especially 7 through 9) unless you have in-depth knowledge, experience and understanding of Linux internals and Arch Linux. Playing with just about any of the files I’ve listed can damage a system in horrible ways, so you’ve been warned. If you don’t know if your system needs a certain firmware file, module, etc. then do your research before you mess with it. (Be warned that removing kernel modules that your system needs can result in an unbootable system, or a system with no keyboard/network card/sound/display/etc. support, or all sorts of other unexplainable behavior.)
Also please note that any package upgrades can and will restore many of the files you remove above. If you do decide to go this route, you may wish to eventually script the removal of unneeded files and run your script after every major package upgrade. (Example: upgrading the kernel will bring back all the kernel modules as well as upgrade via dependency the linux-firmware package bringing back all of the firmware in /usr/lib/firmware .)
Finally, keep an eye on /var/log as the journal files will grow over time. You can remove past journals but keep the current ones by doing something like rm *\@*.journal in your journal folder.
I’ve successfully ran a NAS server off a 512MB Disk-on-Module device for months using these techniques, however they’re still not for the faint of heart. (I’ve also used LinuxFromScratch to build a similar project in only 128MB of storage, but that’s another story. )
Here’s a few more methods you can use to try to get some extra space:
- Getting rid of libgo . The libgo library comes with gcc-libs and is AFAIK only used by applications written in the Go language. I can’t think of any application I use that needs that library. On my system it’s 40MB in size. When you’re trying to slim way down, that’s a lot of space. I’ve removed it from my «mini» installs with no ill effects on anything I do (but again, that’s me, YMMV!)
- Shrinking libicudata.so . It’s 27MB on my system. It’s basically a ton of unicode/locale data compressed into a library object. There is a tool online that can make smaller versions of this file, but it hasn’t been updated for the current version (and you can’t use older files in newer releases.) I haven’t tried doing this by hand, but if you can figure out how, you can shave about 20-22MB off this file.
- If you use Python, you can save 37MB or so by removing the test library from python2: rm -r /usr/lib/python2.7/test and about 66MB by removing it for python3: rm -r /usr/lib/python3.6/test .
- Again for Python, you can get rid of the .pyo files and the .py files. The .pyo files are «optimized» files, but Python never really uses them. The .py files are the raw source code for the standard library. The only files Python normally reads when running Python code is the .pyc (python compiled) files. cd /usr/lib/python2.7 && find . -name «*.pyo» -exec rm -v <> \; and cd /usr/lib/python3.6 && find . -name «*.pyo» -exec rm -v <> \;
- Removing unnecessary locale data. There’s an AUR package called localepurge which automates this. Otherwise, you have to play around in /usr/share/locale . You need to keep your own locale and locale.alias . For me here in the US, keeping en_US and locale.alias and removing everything else shaved off about 80MB.
Now, what I want to see is a tool that analyzes your system and determines which kernel modules you need, and also which firmware files you need. That would be a nice way to «safely» clean out those folders.
Or, maybe someone should put together an Arch «distro» that uses uClibc or diet-libc or something similar. That might be a fun summer project. 🙂
Disk quota
«A disk quota is a limit set by a system administrator that restricts certain aspects of file system usage on modern operating systems. The function of setting quotas to disks is to allocate limited disk-space in a reasonable way.«
This article covers the installation and setup of disk quota.
Installation
Configuration
Set up the filesystem
Edit fstab to enable the quota mount option(s) on selected file systems, e.g.:
/dev/sda3 /home ext4 defaults,usrquota 0 2
To additionally enable the group quota mount option:
/dev/sda3 /home ext4 defaults,usrquota,grpquota 0 2
If supported by the kernel and file system it is recommended to use journaled quota instead:
/dev/sda3 /home ext4 defaults,usrjquota=aquota.user,jqfmt=vfsv1 0 2
Append grpjquota=aquota.group to enable group quota.
Remount the partition to apply the change:
Create quota index
To create the quota index for /home :
Append the -g parameter to also create a group index.
To enable disk quotas for the desired file system:
To disable disk quotas for the file system:
Usage
Enable quota for user/group
- To find out how many 1 kilobyte blocks are there for a partition use $ df .
- You may use a online bytes converter to calculate the correct amount of blocks [1].
- The command # setquota may be used as an alternative of # edquota [2].
Quotas are configured using edquota (as the root user) that will be opened in the default configured text editor:
Disk quotas for user user (uid 1000): Filesystem blocks soft hard inodes soft hard /dev/sda3 24 0 0 6 0 0
blocks Indicates number of 1k blocks currently used by the user/group. soft Indicates max number of blocks for the user/group before a warning is issued and grace period countdown begins. If set to «0» (zero) then no limit is enforced. hard Indicates max number of blocks for the user/group can use. If maximum amount has been reached, no further disk space can be used. If set to «0» (zero) then no limit is enforced. inodes Indicates the current inodes amount used by the user/group. soft Indicates the soft inode limit for the user/group. hard Indicates the hard inode limit for the user/group.
Consider the following configuration for ftpuser1:
Disk quotas for user ftpuser1 (uid 1000): Filesystem blocks soft hard inodes soft hard /dev/sda3 24 1000000 1048576 6 0 0
In this case if ftpuser1 uses over 976MB of space a warning will be issued. If the hard limit of 1GB has been reached the user will be unable to write any more data.
See #Specify a grace period to give users a specific amount of time to reduce storage usage when they hit their soft limit.
Warning: The hard limit applies to all files written by and for the respective user/group, including temporary files by started applications, which may crash at this point.
Specify a grace period
To give current users some time to reduce their file usage, a grace period can be configured. This specifies the allowed time a user/group can exceed their soft limit and while under their hard limit:
Grace period before enforcing soft limits for users: Time units may be: days, hours, minutes, or seconds Filesystem Block grace period Inode grace period /dev/sda3 7days 7days
The grace period can be set in seconds, minutes, hours, days, weeks or months.
Reports
Shows all configured quotas:
Shows quotas on a specific partition:
Show quotas that apply to a user/user group:
Copy quota settings
To one or several users
To copy quota settings from user1 to user2 :
# edquota -p user1 user2
To copy quota settings to several other users, append user3 user4 .
To groups
To copy quota settings from group1 to group2 :
# edquota -g -p group1 group2
To all users
The idea is to modify the quota settings for one user and copy the setting to all other users. Set the quota for user1 and apply the quota to users with a UID greater than 999:
# edquota -p user1 $(awk -F: '$3 > 999 ' /etc/passwd)
Tips and tricks
Quota warnings
The command warnquota can be used to warn the users about their quota. Configuration is available in /etc/warnquota.conf .
Stats
The command quotastats can be used to give more information about the current quota usage:
Number of dquot lookups: 101289 Number of dquot drops: 101271 Number of still active inodes with quota : 18 Number of dquot reads: 93 Number of dquot writes: 2077 Number of quotafile syncs: 134518740 Number of dquot cache hits: 7391 Number of allocated dquots: 90 Number of free dquots: 2036 Number of in use dquot entries (user/group): -1946