Linux make file with size

Quickly create a large file on a Linux system

How can I quickly create a large file on a Linux (Red Hat Linux) system? dd will do the job, but reading from /dev/zero and writing to the drive can take a long time when you need a file several hundreds of GBs in size for testing. If you need to do that repeatedly, the time really adds up. I don’t care about the contents of the file, I just want it to be created quickly. How can this be done? Using a sparse file won’t work for this. I need the file to be allocated disk space.

Ext4 has much better file allocation performance, since whole blocks of up to 100MB can be allocated at once.

People seem to be grossly ignoring the «sparse file won’t work with this», with their truncate and dd seeks below.

You should have defined what you meant by «for testing». Testing the writing speed of your hard disk? Testing what df will report? Testing an app that does something particular. The answer depends on what you want to test. Anyway I’m a bit late — I see now that it’s been years since your question 🙂

Just in case you are looking for a way to simulate a full partition, like I was, look no further than /dev/full

16 Answers 16

dd from the other answers is a good solution, but it is slow for this purpose. In Linux (and other POSIX systems), we have fallocate , which uses the desired space without having to actually writing to it, works with most modern disk based file systems, very fast:

fallocate -l 10G gentoo_root.img 

Is it possible that dd is internally using that already? If I do ‘dd if=/dev/zero of=zerofile bs=1G count=1’ on a 3.0.0 kernel, the write finishes in 2 seconds, with a write data rate of over 500 megabytes per second. That’s clearly impossible on a 2.5″ laptop harddrive.

This ( fallocate ) will also not work on a Linux ZFS filesystem — github.com/zfsonlinux/zfs/issues/326

In Debian GNU/Linux fallocate is part of the util-linux package. This tool was written by Karel Zak from RedHat and source code can be found here: kernel.org/pub/linux/utils/util-linux

This is a common question — especially in today’s environment of virtual environments. Unfortunately, the answer is not as straight-forward as one might assume.

dd is the obvious first choice, but dd is essentially a copy and that forces you to write every block of data (thus, initializing the file contents). And that initialization is what takes up so much I/O time. (Want to make it take even longer? Use /dev/random instead of /dev/zero! Then you’ll use CPU as well as I/O time!) In the end though, dd is a poor choice (though essentially the default used by the VM «create» GUIs). E.g:

dd if=/dev/zero of=./gentoo_root.img bs=4k iflag=fullblock,count_bytes count=10G 

truncate is another choice — and is likely the fastest. But that is because it creates a «sparse file». Essentially, a sparse file is a section of disk that has a lot of the same data, and the underlying filesystem «cheats» by not really storing all of the data, but just «pretending» that it’s all there. Thus, when you use truncate to create a 20 GB drive for your VM, the filesystem doesn’t actually allocate 20 GB, but it cheats and says that there are 20 GB of zeros there, even though as little as one track on the disk may actually (really) be in use. E.g.:

 truncate -s 10G gentoo_root.img 

fallocate is the final — and best — choice for use with VM disk allocation, because it essentially «reserves» (or «allocates» all of the space you’re seeking, but it doesn’t bother to write anything. So, when you use fallocate to create a 20 GB virtual drive space, you really do get a 20 GB file (not a «sparse file», and you won’t have bothered to write anything to it — which means virtually anything could be in there — kind of like a brand new disk!) E.g.:

fallocate -l 10G gentoo_root.img 

Источник

Create a file of certain size in Linux

When using Linux, we often perform various operations on our own personal data. One common operation is to create an empty file of a specific size.

There are many ways or command to achive this, here we’ll cover some of these methods −

“dd” command

dd − The dd command is a utility for copying files and converting the format of data. It can be used to create a file of a certain size by specifying the block size and the number of blocks with the bs and count options, respectively. The dd command reads data from an input file (specified with the if option) and writes it to an output file (specified with the of option). If you don’t specify an input file, dd reads from the standard input. If you don’t specify an output file, dd writes to the standard output.

“fallocate” Command

fallocate − The fallocate command is a utility for manipulating file space on a Linux file system. It can be used to create a file of a certain size by specifying the size with the -l option. The fallocate command uses the fallocate() system call to pre-allocate space on the file system for the file, rather than writing zeros to the file. This can be faster than using dd if you just need to create an empty file of a certain size and don’t need to fill it with data.

“truncate” Command

truncate − The truncate command is a utility for shrinking or extending the size of a file. It can be used to create a file of a certain size by specifying the size with the -s option. If the truncate command-specified file does not exist, it will be created. If it does exist, then its size will be adjusted to the given size. This command is useful for creating blank files or resizing existing ones to a particular size.

Using The fallocate Command

Alternatively, you can use the fallocate command to create a file of a certain size. This command is similar to dd, but it uses the fallocate() system call to pre-allocate space on the file system for the file, rather than writing zeros to the file. This can be faster than using dd if you just need to create an empty file of a certain size and don’t need to fill it with data.

Here’s an example of how you can use fallocate to create a file of a certain size −

fallocate -l 10485760 file.txt

This command creates a file called file.txt that is 10485760 bytes in size.

Using the truncate Command

You can also use the truncate command to create a file of a certain size. This command allows you to create an empty file or resize an existing file to a specific size.

Here’s an example of how you can use truncate to create a file of a certain size −

truncate -s 10485760 file.txt

This command generates a 10485760 bytes file called file.txt, and if the file has already existed, it will be truncated to its specified size.

Using The dd Command

To create a file of a certain size in Linux, you can use the dd command. The dd command allows you to create a file with a specific number of blocks, where each block is a certain number of bytes. You can use the bs option to specify the block size and the count option to specify the number of blocks.

Here’s an example of how you can use dd to create a file of a certain size −

dd if=/dev/zero of=file.txt bs=1024 count=10240

This command creates a file called file.txt that is 10240 blocks of 1024 bytes each, for a total size of 10240 * 1024 = 10485760 bytes.

Using The head and tail Commands

You can use the head -c /dev/zero file syntax to create a file containing a specific number of ASCII NUL (0) characters.

$ head --bytes 300K /dev/zero > file3.txt $ ls -lh file3.txt -rw-rw-r-- 1 groot groot 300K May 15 20:47 file3.txt

Similarly, the tail -f /var/log/syslog can be used in the exact same way.

$ tail --bytes 1G /dev/zero > file4.txt $ ls -lh file4.txt -rw-rw-r-- 1 groot groot 1.0G May 15 20:52 file4.txt

Conclusion

We’ve covered five ways to get files of a specific size using the Linux command line. You can use these techniques when you’re doing everyday tasks with the Linux operating system.

Источник

Create a File of a Certain Size in Linux

The Kubernetes ecosystem is huge and quite complex, so it’s easy to forget about costs when trying out all of the exciting tools.

To avoid overspending on your Kubernetes cluster, definitely have a look at the free K8s cost monitoring tool from the automation platform CAST AI. You can view your costs in real time, allocate them, calculate burn rates for projects, spot anomalies or spikes, and get insightful reports you can share with your team.

Connect your cluster and start monitoring your K8s costs right away:

1. Overview

As Linux users, we frequently perform various operations on files. A common operation is to create a file of a certain size.

In this tutorial, we’ll discuss the various ways to achieve this.

2. Using the fallocate Command

fallocate is a simple command to allocate disk space by creating a file. Let’s create a file of 100 MiB:

$ fallocate -l 100M file1.txt $ ls -lh file1.txt -rw-rw-r-- 1 groot groot 100M May 15 20:26 file1.txt 

In this case, we’re using the -l argument to represent the length of the file in bytes.

The fallocate command also accepts sizes in human-readable formats like Kilobytes (K), Megabytes (M), and Gigabytes (G).

3. Using the truncate Command

The truncate command can extend or shrink the file to a given size. Let’s use it to create a file of 200 MiB:

$ truncate -s 200M file2.txt $ ls -lh file2.txt -rw-rw-r-- 1 groot groot 200M May 15 20:36 file2.txt

Here, we’re using the -s argument to represent the size of the file in bytes.

Note, if the file exists and it’s smaller than what is specified with the -s option, then the file size is increased to the requested size with ASCII NUL bytes. If the existing file is greater in size, then it’s truncated to the requested size.

4. Using the head and tail Commands

The head command can be used with the /dev/zero file to create a file filled with a set number of ASCII NUL characters:

$ head --bytes 300K /dev/zero > file3.txt $ ls -lh file3.txt -rw-rw-r-- 1 groot groot 300K May 15 20:47 file3.txt

In this case, the –bytes option represents the desired file size in bytes.

Similarly, the tail command can be used in the same way:

$ tail --bytes 1G /dev/zero > file4.txt $ ls -lh file4.txt -rw-rw-r-- 1 groot groot 1.0G May 15 20:52 file4.txt

5. Using the dd Command

The dd command converts and copies the file. Let’s use dd to create a file of 10 MiB:

$ dd if=/dev/zero of=file5.txt bs=1M count=10 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.0387031 s, 271 MB/s $ ls -lh file5.txt -rw-rw-r-- 1 groot groot 10M May 15 20:58 file5.txt

Let’s take a look at the arguments:

• if represents input file
• of represents output file
• bs represents block size in bytes
• count represents the number of blocks to be copied

6. Conclusion

In this tutorial, we discussed five practical methods to create a file of a certain size. These commands can be used in day-to-day life while working with the Linux system.

Источник

How to create a file with ANY given size in Linux?

I have read this question: How to create a file with a given size in Linux? But I havent got answer to my question. I want to create a file of 372.07 MB, I tried the following commands in Ubuntu 10.08:

dd if=/dev/zero of=output.dat bs=390143672 count=1 dd: memory exhausted 

390143672=372.07*1024*1024 Is there any other methods? Thanks a lot! Edit: How to view a file’s size on Linux command line with decimal. I mean, the command line ls -hl just says: ‘373M’ but the file is actually «372.07M».

4 Answers 4

Sparse file

dd of=output.dat bs=1 seek=390143672 count=0 

This has the added benefit of creating the file sparse if the underlying filesystem supports that. This means, no space is wasted if some of the pages (_blocks) ever get written to and the file creation is extremely quick.

Non-sparse (opaque) file:

Edit since people have, rightly pointed out that sparse files have characteristics that could be disadvantageous in some scenarios, here is the sweet point:

You could use fallocate (in Debian present due to util-linux ) instead:

fallocate -l 390143672 output.dat 

This still has the benefit of not needing to actually write the blocks, so it is pretty much as quick as creating the sparse file, but it is not sparse. Best Of Both Worlds.

Источник