Linux сумма размеров файлов

How to list the size of each file and directory and sort by descending size in Bash?

I found that there is no easy to get way the size of a directory in Bash? I want that when I type ls — , it can list of all the sum of the file size of directory recursively and files at the same time and sort by size order. Is that possible?

What exactly do you mean by the «size» of a directory? The number of files under it (recursively or not)? The sum of the sizes of the files under it (recursively or not)? The disk size of the directory itself? (A directory is implemented as a special file containing file names and other information.)

@KeithThompson @KitHo du command estimates file space usage so you cannot use it if you want to get the exact size.

@ztank1013: Depending on what you mean by «the exact size», du (at least the GNU coreutils version) probably has an option to provide the information.

12 Answers 12

Simply navigate to directory and run following command:

OR add -h for human readable sizes and -r to print bigger directories/files first.

du -a -h --max-depth=1 | sort -hr 

du -h requires sort -h too, to ensure that, say 981M sorts before 1.3G ; with sort -n only the numbers would be taken into account and they’d be the wrong way round.

This doesn’t list the size of the individual files within the current directory, only the size of its subdirectories and the total size of the current directory. How would you include individual files in the output as well (to answer OP’s question)?

@ErikTrautman to list the files also you need to add -a and use —all instead of —max-depth=1 like so du -a -h —all | sort -h

Apparently —max-depth option is not in Mac OS X’s version of the du command. You can use the following instead.

Unfortunately this does not show the files, but only the folder sizes. -a does not work with -d either.

To show files and folders, I combined 2 commands: l -hp | grep -v / && du -h -d 1 , which shows the normal file size from ls for files, but uses du for directories.

(this willnot show hidden (.dotfiles) files)

Use du -sm for Mb units etc. I always use

because the total line ( -c ) will end up at the bottom for obvious reasons 🙂

PS:

• See comments for handling dotfiles
• I frequently use e.g. ‘du -smc /home// | sort -n |tail’ to get a feel of where exactly the large bits are sitting

du —max-depth=1|sort -n or find . -mindepth 1 -maxdepth 1|xargs du -s|sort -n for including dotfiles too.

@arnaud576875 find . -mindepth 1 -maxdepth 1 -print0 | xargs -0 du -s | sort -n if some of the found paths could contain spaces.

This is a great variant to get a human readable view of the biggest: sudo du -smch * | sort -h | tail

Command

Output

3,5M asdf.6000.gz 3,4M asdf.4000.gz 3,2M asdf.2000.gz 2,5M xyz.PT.gz 136K xyz.6000.gz 116K xyz.6000p.gz 88K test.4000.gz 76K test.4000p.gz 44K test.2000.gz 8,0K desc.common.tcl 8,0K wer.2000p.gz 8,0K wer.2000.gz 4,0K ttree.3 

Explanation

• du displays «disk usage»
• h is for «human readable» (both, in sort and in du)
• max-depth=0 means du will not show sizes of subfolders (remove that if you want to show all sizes of every file in every sub-, subsub-, . folder)
• r is for «reverse» (biggest file first)

ncdu

When I came to this question, I wanted to clean up my file system. The command line tool ncdu is way better suited to this task.

Just type ncdu [path] in the command line. After a few seconds for analyzing the path, you will see something like this:

$ ncdu 1.11 ~ Use the arrow keys to navigate, press ? for help --- / --------------------------------------------------------- . 96,1 GiB [##########] /home . 17,7 GiB [# ] /usr . 4,5 GiB [ ] /var 1,1 GiB [ ] /lib 732,1 MiB [ ] /opt . 275,6 MiB [ ] /boot 198,0 MiB [ ] /storage . 153,5 MiB [ ] /run . 16,6 MiB [ ] /etc 13,5 MiB [ ] /bin 11,3 MiB [ ] /sbin . 8,8 MiB [ ] /tmp . 2,2 MiB [ ] /dev ! 16,0 KiB [ ] /lost+found 8,0 KiB [ ] /media 8,0 KiB [ ] /snap 4,0 KiB [ ] /lib64 e 4,0 KiB [ ] /srv ! 4,0 KiB [ ] /root e 4,0 KiB [ ] /mnt e 4,0 KiB [ ] /cdrom . 0,0 B [ ] /proc . 0,0 B [ ] /sys @ 0,0 B [ ] initrd.img.old @ 0,0 B [ ] initrd.img @ 0,0 B [ ] vmlinuz.old @ 0,0 B [ ] vmlinuz 

Delete the currently highlighted element with d , exit with CTRL + c

ls -S sorts by size. Then, to show the size too, ls -lS gives a long ( -l ), sorted by size ( -S ) display. I usually add -h too, to make things easier to read, so, ls -lhS .

Ah, sorry, that was not clear from your post. You want du , seems someone has posted it. @sehe: Depends on your definition of real — it is showing the amount of space the directory is using to store itself. (It’s just not also adding in the size of the subentries.) It’s not a random number, and it’s not always 4KiB.

find . -mindepth 1 -maxdepth 1 -type d | parallel du -s | sort -n 

I think I might have figured out what you want to do. This will give a sorted list of all the files and all the directories, sorted by file size and size of the content in the directories.

(find . -depth 1 -type f -exec ls -s <> \;; find . -depth 1 -type d -exec du -s <> \;) | sort -n 

[enhanced version]
This is going to be much faster and precise than the initial version below and will output the sum of all the file size of current directory:

echo `find . -type f -exec stat -c %s <> \; | tr '\n' '+' | sed 's/+$//g'` | bc 

the stat -c %s command on a file will return its size in bytes. The tr command here is used to overcome xargs command limitations (apparently piping to xargs is splitting results on more lines, breaking the logic of my command). Hence tr is taking care of replacing line feed with + (plus) sign. sed has the only goal to remove the last + sign from the resulting string to avoid complains from the final bc (basic calculator) command that, as usual, does the math.

Performances: I tested it on several directories and over ~150.000 files top (the current number of files of my fedora 15 box) having what I believe it is an amazing result:

# time echo `find / -type f -exec stat -c %s <> \; | tr '\n' '+' | sed 's/+$//g'` | bc 12671767700 real 2m19.164s user 0m2.039s sys 0m14.850s 

Just in case you want to make a comparison with the du -sb / command, it will output an estimated disk usage in bytes ( -b option)

As I was expecting it is a little larger than my command calculation because the du utility returns allocated space of each file and not the actual consumed space.

[initial version]
You cannot use du command if you need to know the exact sum size of your folder because (as per man page citation) du estimates file space usage. Hence it will lead you to a wrong result, an approximation (maybe close to the sum size but most likely greater than the actual size you are looking for).

I think there might be different ways to answer your question but this is mine:

ls -l $(find . -type f | xargs) | cut -d" " -f5 | xargs | sed 's/\ /+/g'| bc 

It finds all files under . directory (change . with whatever directory you like), also hidden files are included and (using xargs ) outputs their names in a single line, then produces a detailed list using ls -l . This (sometimes) huge output is piped towards cut command and only the fifth field ( -f5 ), which is the file size in bytes is taken and again piped against xargs which produces again a single line of sizes separated by blanks. Now take place a sed magic which replaces each blank space with a plus ( + ) sign and finally bc (basic calculator) does the math.

It might need additional tuning and you may have ls command complaining about arguments list too long.

Источник

Посчитать размер файлов, созданных 21 июля

Есть папка в /Archive — там 10000 файлов, надо найти все файлы созданные 21 июля и посчитать их сумму.

$ for i in ls -l | grep "Jul 21" | awk ''; do AB=0; AB=du -sh $i | awk '' | sed 's/K//g' | cut -d. -f 1; SUM=expr $AB + $SUM ; done 

for i in ls -l | grep «Jul 21» | awk ‘‘ ; do AB=0; AB= du -sh $i | awk ‘‘ | sed ‘s/K//g’ | cut -d. -f 1 ; SUM= expr $AB + $SUM ; done не работает

2 ответа 2

найти файлы, созданные 21 июля 2016-го года можно, например, так:

$ find /archive -newerct 2016-07-21 \! -newerct 2016-07-22 

подсчитать же общий их размер можно, например, с помощью программы du. примерно так:

$ find /archive -newerct 2016-07-21 \! -newerct 2016-07-22 -print0 | xargs -0 du -ch | tail -n 1 

спасибо, сработало. Просил двух индусов из саппорта — оба дали неправильные комманды. Если бы не вы то не сносить бы мне головы.

@Kwaker: 1- вы не получите суммарный размер файлов, если du -с больше одного раза вызывается (что возможно если find печатает много имён файлов) 2- Дата создания на Linux не всегда доступна ( newerct использует дату последней модификации метаданных файла, что может отличаться). ext2/3 не хранят дату создания, ext4 файловая система хранит, но специальное API может быть нужно. Посмотрите на Birth time в выводе stat filename , а не Change time, используемое newerct 3- Существует разница между местом, занимаемым на диске и отображаемым размером

Источник

bash script: calculate sum size of files

I’m working on Linux and need to calculate the sum size of some files in a directory. I’ve written a bash script named cal.sh as below:

#!/bin/bash while IFS='' read -r line || [[ -n "$line" ]]; do echo $line done'` 

./cal.sh: line 6: `ls -l | grep opencv | awk ''`: ambiguous redirect 

And if I execute it with sh cal.sh , it seems to work but I will get some weird message at the end of output:

25 31 385758: File name too long 

3 Answers 3

option -b will display each file in bytes and -c will print the total size.

I would recommend against using that pipeline to get the sizes of the files you want — in general parsing ls is something that you should avoid. Instead, you can just use *opencv* to get the files and stat to print the size:

The format specifier %s prints the size of each file in bytes.

You can pipe this to awk to get the sum:

stat -c %s *opencv* | awk ' < sum += $0 >END < if (sum) print sum >' 

The if is there to ensure that no input => no output.

Ultimately, as other answers will point out, it’s not a good idea to parse the output of ls because it may vary between systems. But it’s worth knowing why the script doesn’t work.

The ambiguous redirect error is because you need quotes around your ls command i.e.:

while IFS='' read -r line || [[ -n "$line" ]]; do echo $line done < "`ls -l | grep opencv | awk ''`" 

while IFS='' read -r line || [[ -n "$line" ]]; do echo $line done '`" 

This works as expected, but has some drawbacks. When using a "here string" the command must first execute in full, then store the output of said command in a temporary variable. This can be a problem if the command takes long to execute or has a large output.

IMHO the best and most standard method of iterating a commands output line by line is the following:

ls -l | grep opencv | awk ' '| while read -r line ; do echo "line: $line" done 

Источник

How do I determine the total size of a directory (folder) from the command line?

The -h flag on sort will consider "Human Readable" size values.

If want to avoid recursively listing all files and directories, you can supply the --max-depth parameter to limit how many items are displayed. Most commonly, --max-depth=1

du -h --max-depth=1 /path/to/directory 

I use du -sh or DOOSH as a way to remember it (NOTE: the command is the same, just the organization of commandline flags for memory purposes)

There is a useful option to du called the --apparent-size. It can be used to find the actual size of a file or directory (as opposed to its footprint on the disk) eg, a text file with just 4 characters will occupy about 6 bytes, but will still show up as taking up ~4K in a regular du -sh output. However, if you pass the --apparent-size option, the output will be 6. man du says: --apparent-size print apparent sizes, rather than disk usage; although the apparent size is usually smaller, it may be larger due to holes in (‘sparse’) files, internal fragmentation, indirect blocks

This works for OS X too! Thanks, I was really looking for a way to clear up files, both on my local machine, and my server, but automated methods seemed not to work. So, I ran du -hs * and went into the largest directory and found out which files were so large. This is such a good method, and the best part is you don't have to install anything! Definitely deserved my upvote

@BandaMuhammadAlHelal I think there are two reasons: rounding ( du has somewhat peculiar rounding, showing no decimals if the value has more than one digit in the chosen unit), and the classical 1024 vs. 1000 prefix issue. du has an option -B (or --block-size ) to change the units in which it displays values, or you could use -b instead of -h to get the "raw" value in bytes.

Источник