- How to Use file descriptors to read from and write to multiple files in Linux
- User-Defined File Descriptors
- File Descriptors in the Bourne Shell
- Korn Shell File Descriptors
- read(2) - Linux man page
- Return Value
- Errors
- Conforming to
- Notes
- See Also
- Read file descriptor linux
- NAME
- SYNOPSIS
- DESCRIPTION
- RETURN VALUE
- ERRORS
- CONFORMING TO
- NOTES
- SEE ALSO
- COLOPHON
How to Use file descriptors to read from and write to multiple files in Linux
A script can receive input from a file and send output to a file so that the script can run without user interaction. Later, a user can review the output file, or another script can use the output file as its input.
File input and output are accomplished in the shell by integer handles that the kernel uses to track all open files in a process. These numeric values are file descriptors. The best-known file descriptors are 0 (stdin), 1 (stdout), and 2 (stderr)
The numbers 3 through 9 are for programmer-defined file descriptors. Use these to associate numeric values to pathnames. In a program, if there is a need to read or write multiple times to the same file, a shorthand file descriptor value might reduce errors in referencing the file name. Also, a change to the file name must be done only once if the file is subsequently accessed through the file descriptor. The table below shows the syntax for file redirection.
| Command | Description |
|---|---|
| < file | Takes standard input from file |
| 0 < file | Takes standard input from file |
| > file | Puts standard output to file |
| 1> file | Puts standard output to file |
| 2> file | Puts standard error to file |
| exec fd> /some/filename | Assigns the file descriptor fd to /some/filename for output |
| exec fd < /some/filename | Assigns the file descriptor fd to /some/filename for input |
| read | Reads from the file descriptor fd and stores into variable var1 |
| cmd >& fd | Executes cmd and sends output to the file descriptor fd |
| exec fd | Closes the file descriptor fd |
User-Defined File Descriptors
You can use a file descriptor to assign a numeric value to a file instead of using the file name. The exec command is one of the built-in commands in a shell. Use this command to assign a file descriptor to a file. The syntax is:
# exec fd> filename # exec fd< filename The file descriptor assigned to a file is valid in the current shell only.
File Descriptors in the Bourne Shell
In the following example, the syntax for reading and writing using file descriptors uses the Bourne syntax for the read and echo statements. This syntax works with both shells.
- Copy the /etc/hosts file to the /tmp/hosts2 file.
- Use grep to read the /tmp/hosts2 file, strip out the comment lines, and send the output to the /tmp/hosts3 file.
- Assign file descriptor 3 to the /tmp/hosts3 file for input, and then each read statement issued to file descriptor 3 reads a record from the file. The statement used to associate file descriptor 3 to the input file named /tmp/hosts3 is: “exec 3 < /tmp/hosts3”
- Assign file descriptor 4 to the /tmp/hostsfinal file for output. If the output file does not exist, it is created. If it does exist, its size truncates to 0 bytes. The statement that associates file descriptor 4 to the output file is: “exec 4> /tmp/hostsfinal”
- Read the /tmp/hosts3 file and output two fields to the /tmp/hostsfinal file. Reading from the input file is accomplished by specifying the file descriptor number to the & argument of the print statement. This causes the output to be written as a line in the output file. The output file is written sequentially.
- Close the input file when it is no longer needed. This is good practice. The OS closes the file automatically when the process terminates if the program fails to close it
- When all writes to the output file are complete, close the file.
$ cat readex.sh #!/bin/sh # Script name: readex.sh ##### Step 1 - Copy /etc/host cp /etc/hosts /tmp/hosts2 ##### Step 2 - Strip out comment lines grep -v ’^#’ /tmp/hosts2 > /tmp/hosts3 ##### Step 3 - fd 3 is input file /tmp/hosts3 exec 3 /tmp/hostsfinal ##### The following 4 statements accomplish STEP 5 read & 4 # Write to fd 4 (do not write aliases) ##### END OF STEP 5 statements exec 3 $ more /tmp/hosts2 # # Internet host table # 127.0.0.1 localhost 192.9.200.111 ultrabear loghost 192.9.200.121 ladybear $ more /tmp/hosts3 127.0.0.1 localhost 192.9.200.111 ultrabear loghost 192.9.200.121 ladybear $ more /tmp/hostsfinal localhost 127.0.0.1 ultrabear 192.9.200.111 Korn Shell File Descriptors
The following example shows how the previous example would be written in the Korn shell. In the Korn shell, the read statement uses a -ufd syntax rather than the
The print statement uses a -ufd syntax to direct the output to file descriptor 4 in the Korn shell.
$ cat readex.ksh #!/bin/ksh # Script name: readex.ksh ##### Step 1 - Copy /etc/host cp /etc/hosts /tmp/hosts2 ##### Step 2 - Strip out comment lines grep -v ’^#’ /tmp/hosts2 > /tmp/hosts3 ##### Step 3 - fd 3 is an input file /tmp/hosts3 exec 3 /tmp/hostsfinal ##### The following 4 statements accomplish STEP 5 read -u3 addr1 name1 alias # read from fd 3 read -u3 addr2 name2 alias # read from fd 3 print -u4 $name1 $addr1 # write to fd 4 (do not write aliases) print -u4 $name2 $addr2 # write to fd 4 (do not write aliases) ##### END OF STEP 5 statements exec 3 $ more /tmp/hosts2 # # Internet host table # 127.0.0.1 localhost 192.9.200.111 ultrabear loghost 192.9.200.121 ladybear $ more /tmp/hosts3 127.0.0.1 localhost 192.9.200.111 ultrabear loghost 192.9.200.121 ladybear $ more /tmp/hostsfinal localhost 127.0.0.1 ultrabear 192.9.200.111 read(2) - Linux man page
On files that support seeking, the read operation commences at the current file offset, and the file offset is incremented by the number of bytes read. If the current file offset is at or past the end of file, no bytes are read, and read() returns zero.
If count is zero, read() may detect the errors described below. In the absence of any errors, or if read() does not check for errors, a read() with a count of 0 returns zero and has no other effects.
If count is greater than SSIZE_MAX, the result is unspecified.
Return Value
On success, the number of bytes read is returned (zero indicates end of file), and the file position is advanced by this number. It is not an error if this number is smaller than the number of bytes requested; this may happen for example because fewer bytes are actually available right now (maybe because we were close to end-of-file, or because we are reading from a pipe, or from a terminal), or because read() was interrupted by a signal. On error, -1 is returned, and errno is set appropriately. In this case it is left unspecified whether the file position (if any) changes.
Errors
EAGAIN The file descriptor fd refers to a file other than a socket and has been marked nonblocking (O_NONBLOCK), and the read would block. EAGAIN or EWOULDBLOCK The file descriptor fd refers to a socket and has been marked nonblocking (O_NONBLOCK), and the read would block. POSIX.1-2001 allows either error to be returned for this case, and does not require these constants to have the same value, so a portable application should check for both possibilities. EBADF fd is not a valid file descriptor or is not open for reading. EFAULT buf is outside your accessible address space. EINTR The call was interrupted by a signal before any data was read; see signal(7). EINVAL fd is attached to an object which is unsuitable for reading; or the file was opened with the O_DIRECT flag, and either the address specified in buf, the value specified in count, or the current file offset is not suitably aligned. EINVAL fd was created via a call to timerfd_create(2) and the wrong size buffer was given to read(); see timerfd_create(2) for further information. EIO I/O error. This will happen for example when the process is in a background process group, tries to read from its controlling terminal, and either it is ignoring or blocking SIGTTIN or its process group is orphaned. It may also occur when there is a low-level I/O error while reading from a disk or tape. EISDIR fd refers to a directory.
Other errors may occur, depending on the object connected to fd. POSIX allows a read() that is interrupted after reading some data to return -1 (with errno set to EINTR) or to return the number of bytes already read.
Conforming to
Notes
On NFS file systems, reading small amounts of data will only update the timestamp the first time, subsequent calls may not do so. This is caused by client side attribute caching, because most if not all NFS clients leave st_atime (last file access time) updates to the server and client side reads satisfied from the client's cache will not cause st_atime updates on the server as there are no server side reads. UNIX semantics can be obtained by disabling client side attribute caching, but in most situations this will substantially increase server load and decrease performance.
See Also
close(2), fcntl(2), ioctl(2), lseek(2), open(2), pread(2), readdir(2), readlink(2), readv(2), select(2), write(2), fread(3)
Read file descriptor linux
NAME
read - read from a file descriptor
SYNOPSIS
#include unistd.h> ssize_t read(int fd, void *buf, size_t count);
DESCRIPTION
read() attempts to read up to count bytes from file descriptor fd into the buffer starting at buf. On files that support seeking, the read operation commences at the current file offset, and the file offset is incremented by the number of bytes read. If the current file offset is at or past the end of file, no bytes are read, and read() returns zero. If count is zero, read() may detect the errors described below. In the absence of any errors, or if read() does not check for errors, a read() with a count of 0 returns zero and has no other effects. If count is greater than SSIZE_MAX, the result is unspecified.
RETURN VALUE
On success, the number of bytes read is returned (zero indicates end of file), and the file position is advanced by this number. It is not an error if this number is smaller than the number of bytes requested; this may happen for example because fewer bytes are actually available right now (maybe because we were close to end-of-file, or because we are reading from a pipe, or from a terminal), or because read() was interrupted by a signal. On error, -1 is returned, and errno is set appropriately. In this case it is left unspecified whether the file position (if any) changes.
ERRORS
EAGAIN The file descriptor fd refers to a file other than a socket and has been marked nonblocking (O_NONBLOCK), and the read would block. EAGAIN or EWOULDBLOCK The file descriptor fd refers to a socket and has been marked nonblocking (O_NONBLOCK), and the read would block. POSIX.1-2001 allows either error to be returned for this case, and does not require these constants to have the same value, so a portable application should check for both possibilities. EBADF fd is not a valid file descriptor or is not open for reading. EFAULT buf is outside your accessible address space. EINTR The call was interrupted by a signal before any data was read; see signal(7). EINVAL fd is attached to an object which is unsuitable for reading; or the file was opened with the O_DIRECT flag, and either the address specified in buf, the value specified in count, or the current file offset is not suitably aligned. EINVAL fd was created via a call to timerfd_create(2) and the wrong size buffer was given to read(); see timerfd_create(2) for further information. EIO I/O error. This will happen for example when the process is in a background process group, tries to read from its controlling terminal, and either it is ignoring or blocking SIGTTIN or its process group is orphaned. It may also occur when there is a low-level I/O error while reading from a disk or tape. EISDIR fd refers to a directory. Other errors may occur, depending on the object connected to fd. POSIX allows a read() that is interrupted after reading some data to return -1 (with errno set to EINTR) or to return the number of bytes already read.
CONFORMING TO
NOTES
On NFS filesystems, reading small amounts of data will update the timestamp only the first time, subsequent calls may not do so. This is caused by client side attribute caching, because most if not all NFS clients leave st_atime (last file access time) updates to the server and client side reads satisfied from the client's cache will not cause st_atime updates on the server as there are no server side reads. UNIX semantics can be obtained by disabling client side attribute caching, but in most situations this will substantially increase server load and decrease performance.
SEE ALSO
close(2), fcntl(2), ioctl(2), lseek(2), open(2), pread(2), readdir(2), readlink(2), readv(2), select(2), write(2), fread(3)
COLOPHON
© 2019 Canonical Ltd. Ubuntu and Canonical are registered trademarks of Canonical Ltd.