- How create a temporary file in shell script?
- 7 Answers 7
- zsh
- bash/zsh on Linux.
- POSIX
- Creating a Temporary File in Linux
- Creating a Temporary File in Linux
- Syntax
- Create a Random Temporary File in tmp Directory
- Specifying a Directory and Permissions
- Specify Template of temporary file
- Create multiple Temporary files
- Checking the Status of the Temporary File
- Removing the Temporary File
- mktemp output and file status
- Example
- Use mktemp in Bash Script
- Example
- Output
- Conclusion
- How to create temporary files on linux that will automatically clean up after themselves no matter what?
- 6 Answers 6
How create a temporary file in shell script?
While running a script, I want to create a temporary file in /tmp directory. After execution of that script, that will be cleaned by that script. How to do that in shell script?
7 Answers 7
tmpfile=$(mktemp /tmp/abc-script.XXXXXX) : . rm "$tmpfile" You can make sure that a file is deleted when the scripts exits (including kills and crashes) by opening a file descriptor to the file and deleting it. The file keeps available (for the script; not really for other processes but /proc/$PID/fd/$FD is a work-around) as long as the file descriptor is open. When it gets closed (which the kernel does automatically when the process exits) the filesystem deletes the file.
# create temporary file tmpfile=$(mktemp /tmp/abc-script.XXXXXX) # create file descriptor 3 for writing to a temporary file so that # echo . >&3 writes to that file exec 3>"$tmpfile" # create file descriptor 4 for reading from the same file so that # the file seek positions for reading and writing can be different exec 4&3 # your script continues : . # reading from that file descriptor head -n 1 &- » You can use cat
@HaukeLaging will you update this to show how to read from the file descriptor? Automatically deleting a file you write to and don’t read from isn’t much good.
Use mktemp to create a temporary file
or, to create a temporary directory:
At the end of the script you have to delete the temporary file or directory
mktemp creates file in the /tmp directory or in the directory given with the —tmpdir argument.
You can use trap «rm -f $temp_file» 0 2 3 15 right after creating the file so that when the script exits or is stopped with ctrl-C the file is still removed.
@HaukeLaging Then the trap doesn’t fire if the script is stopped with Ctrl+C. A thing to note is that TRAP doesn’t help if you kill -9 $somepid . That particular kill signal is insta-death with nothing else happening.
For those wondering, the trailing integers in trap «rm -f $temp_file» 0 2 3 15 are the signals upon which to run the first argument. 0: exit shell, 2: Interrupt, 3: Quit, 15: Terminate.
Some shells have the feature built-in.
zsh
zsh ‘s =(. ) form of process substitution uses a temporary file. For instance =(echo test) expands to the path of a temporary file that contains test\n .
$ &3; cat $file> 3<> $ test lrwx------ 1 stephane stephane 64 Jan 30 11:19 /dev/fd/3 -> /tmp/zshMLbER0 test2 That file is automatically removed, once the command has finished.
bash/zsh on Linux.
Here-documents or here-strings in bash versions prior to 5.1 and zsh are implemented as deleted temporary files (as was the case in the Bourne shell which introduced here-documents in the late 70s).
The file descriptor 3 is connected to a deleted temporary file that contains test\n .
You can get its content with:
If on Linux, you can also read or write to that file via /dev/fd/3 , though with bash version 5.0, you’d first to need to restore write permissions to it (which bash explicitly removes in that version):
$ exec 3 /dev/fd/3 $ cat /dev/fd/3 foo (some other shells use pipes, or may use /dev/null if the here doc is empty).
POSIX
There is no mktemp POSIX utility. POSIX however specifies a mkstemp(template) C API, and the m4 standard utility exposes that API with the mkstemp() m4 function by the same name.
mkstemp() gives you a file name with a random part that was guaranteed not to exist at the time the function was called. It does create the file with permissions 0600 in a race-free way.
tmpfile=$( echo 'mkstemp(template)' | m4 -D template="$/baseXXXXXX" ) || exit Note however that you need to handle the clean-up upon exit, though if you only need to write and read the file a fixed number of times, you could open it and delete it just after creating like for the here-doc/here-string approach above:
tmpfile=$( echo 'mkstemp(template)' | m4 -D template="$/baseXXXXXX" ) || exit # open once for writing, twice for reading: exec 3> "$tempfile" 4< "$tempfile" 5< "$tempfile" rm -f -- "$tmpfile" cmd >&3 # store something in the temp file exec 3>&- # fd no longer needed # read the content twice: cat You could open the file for reading once, and rewind in between two reads, however there's no POSIX utility that can do that rewinding ( lseek() ), so you can't do it portably in a POSIX script ( zsh ( sysseek builtin) and ksh93 ( <#((. )) operator) can do it though).
Creating a Temporary File in Linux
In Linux, it is often necessary to create temporary files for various purposes, such as storing intermediate data during processing or storing configuration information for a script. Temporary files are usually created in the /tmp directory, which is a standard location for storing temporary files on most Linux systems.
Creating a Temporary File in Linux
There are several ways to create a temporary file in Linux. One of the most common methods is to use the mktemp command, which creates a unique temporary file and prints the file name to the console.
Syntax
To create a temporary file using mktemp, use the following syntax.
The template argument is a string that specifies the name and location of the temporary file, and can include XXXXXX as a placeholder for a unique suffix that mktemp will generate. The options argument is optional and can be used to specify various options, such as the directory in which to create the file or the permissions to set on the file.
Create a Random Temporary File in tmp Directory
The simplest way to create a temporary file can be done by running mktemp command without any arguments. If you run the mktemp command without any arguments, it will create a unique temporary file in the default temporary file directory (usually /tmp) with a default template of tmp.XXXXXX.
This will create a file with a name like /tmp/tmp.qhgG9f.
You can also use the -q option to suppress the output of the mktemp command and store the file name in a variable instead.
$ temp_file=$(mktemp -q) $ echo "Temporary file: $temp_file"
This will create a temporary file with a name like /tmp/tmp.qhgG9f and store the file name in the temp_file variable. The echo command will then print the file name to the console.
Specifying a Directory and Permissions
To specify a different directory in which to create the temporary file, use the -d option.
$ mktemp -d /my/custom/dir/temp.XXXXXX
This will create a temporary file in the /my/custom/dir directory with a name like temp.qhgG9f.
$ mktemp -d /my/custom/dir/temp.XXXXXX
You can also use the -p option to specify a prefix for the file name, like this −
$ mktemp -p /my/custom/dir mytemp.XXXXXX
This will create a file with a name like /my/custom/dir/mytemp.qhgG9f.
Specify Template of temporary file
To create a temporary file in the /tmp directory with a unique suffix and default permissions, you can use the following command −
This will create a file with a name like /tmp/temp.qYhg9f, where qYhg9f is the unique suffix generated by mktemp.
You can also use the -t option to specify a template that includes the XXXXXX placeholder, like this −
This will create a file with a name like temp.qYhg9f in the default temporary file directory (usually /tmp).
Create multiple Temporary files
To create multiple temporary files at once, use the -u option followed by a template with multiple instances of the XXXXXX placeholder.
$ mktemp -u /tmp/temp1.XXXXXX /tmp/temp2.XXXXXX
This will create two temporary files with names like /tmp/temp1.qhgG9f and /tmp/temp2.qhgG9f.
Checking the Status of the Temporary File
After creating a temporary file, you may want to check its status to ensure that it was created successfully. You can use the stat command to display information about a file, including its size, permissions, and creation date.
To use stat, specify the name of the file as an argument, like this −
This will display information about the file, including its size, permissions, and creation date.
Removing the Temporary File
When you are finished with a temporary file, it is a good idea to remove it to free up disk space and prevent clutter. You can use the rm command to remove a file, like this:
This will delete the file permanently, so be sure that you no longer need the file before removing it.
mktemp output and file status
Here is an example of the mktemp command in action −
Example
$ mktemp /tmp/temp.XXXXXX /tmp/temp.qhgG9f $ stat /tmp/temp.qhgG9f File: /tmp/temp.qhgG9f Size: 0 Blocks: 0 IO Block: 4096 regular empty file Device: 802h/2050d Inode: 1234 Links: 1 Access: (0664/-rw-rw-r--) Uid: ( 1000/ user) Gid: ( 1000/ user) Access: 2022-12-16 11:07:35.000000000 -0500 Modify: 2022-12-16 11:07:35.000000000 -0500 Change: 2022-12-16 11:07:35.000000000 -0500 Birth: - $ rm /tmp/temp.qhgG9f
In this example, the mktemp command creates a temporary file in the /tmp directory with a unique suffix
Use mktemp in Bash Script
Once the temporary file has been created, you can use it like any other file on the system. When you are finished with the file, you can remove it using the rm command.
Example
Here is an example of creating a temporary file and then displaying its contents −
# Create the temporary file and store its path in a variable temp_file=$(mktemp) # Write some content to the file echo "This is a temporary file" > $temp_file # Display the contents of the file cat $temp_file # Remove the temporary file rm $temp_file
This code will create a temporary file, write the string "This is a temporary file" to it, display the contents of the file, and then delete the file. The output of this code will be −
Output
Conclusion
In Linux, temporary files can be created easily using the mktemp command. By specifying a template and optional options, you can create a unique temporary file in the desired location and with the desired permissions. You can then use the stat command to check the status of the file, and the rm command to remove it when it is no longer needed.
How to create temporary files on linux that will automatically clean up after themselves no matter what?
I want to create a temporary file on linux while making sure that the file will disappear after my program has terminated, even if it got killed or someone performs a hard reboot in the wrong moment. Does tmpfile() handle all this for me?
@JarrodRoberson I'll re-add the [race-condition] tag - this is all about the time window between open() and unlink() .
@JarrodRoberson: Uh, aren't you talking about a deadlock? My definition of "race condition" is something like "there are some parallel processes and depending on how fast they are in their race, different things happen". Let's look what the english wikipedia says: "A race condition or race hazard is a flaw in an electronic system or process whereby the output or result of the process is unexpectedly and critically dependent on the sequence or timing of other events."
"A race condition occurs when a program doesn't work as it's supposed to because of an unexpected ordering of events that produces contention over the same resource." I was assuming that from your comments your concern was specifically on a dead-lock which is a result of trying to remediate a race-condition ( contention of the shared resource ). It is still not clear what your concern is, calling tmpfile() and having the program exit abnormally before calling unlink() is the least of your worries if your application is really that fragile.
6 Answers 6
You seem pre-occupied with the idea that files might get left behind some how because of some race condition, I don't see an explanation of why this is a concern.
"A race condition occurs when a program doesn't work as it's supposed to because of an unexpected ordering of events that produces contention over the same resource."
I was assuming that from your comments on other answers your concern was specifically on a dead-lock which is a result of trying to remediate a race-condition ( contention of the shared resource ). It is still not clear what your concern is, calling tmpfile() and having the program exit abnormally before that function gets to call unlink() is the least of your worries if your application is really that fragile.
Given that there isn't any mention of concurrency, threading or other processes sharing this file descriptor to this temp file, I still don't see the possibility for a race condition, maybe the concept of an incomplete logical transaction, but that can be detected and cleaned up.
The correct way to make absolutely sure that any allocated file system resources are cleaned up is not solely on exit of an application but also also on start-up. All my server code, makes sure that everything is cleaned up from a previous run before it starts and makes itself available.
Put your temp files in a sub-dir in /tmp make sure your application cleans this sub-dir on startup and normal shutdown. You can wrap your app start up with a shell script that detects abnormal ( kill -9 ) shutdown based on PID existence and also does clean up activities.