Confused about stdin, stdout and stderr?
I am rather confused with the purpose of these three files. If my understanding is correct, stdin is the file in which a program writes into its requests to run a task in the process, stdout is the file into which the kernel writes its output and the process requesting it accesses the information from, and stderr is the file into which all the exceptions are entered. On opening these files to check whether these actually do occur, I found nothing seem to suggest so! What I would want to know is what exactly is the purpose of these files, absolutely dumbed down answer with very little tech jargon!
11 Answers 11
Standard input — this is the file handle that your process reads to get information from you.
Standard output — your process writes conventional output to this file handle.
Standard error — your process writes diagnostic output to this file handle.
That’s about as dumbed-down as I can make it 🙂
Of course, that’s mostly by convention. There’s nothing stopping you from writing your diagnostic information to standard output if you wish. You can even close the three file handles totally and open your own files for I/O.
When your process starts, it should already have these handles open and it can just read from and/or write to them.
By default, they’re probably connected to your terminal device (e.g., /dev/tty ) but shells will allow you to set up connections between these handles and specific files and/or devices (or even pipelines to other processes) before your process starts (some of the manipulations possible are rather clever).
my_prog errorfile | grep XYZ - create a process for my_prog .
- open inputfile as your standard input (file handle 0).
- open errorfile as your standard error (file handle 2).
- create another process for grep .
- attach the standard output of my_prog to the standard input of grep .
When I open these files in /dev folder, how come I never get to see the output of a process running?
It’s because they’re not normal files. While UNIX presents everything as a file in a file system somewhere, that doesn’t make it so at the lowest levels. Most files in the /dev hierarchy are either character or block devices, effectively a device driver. They don’t have a size but they do have a major and minor device number.
When you open them, you’re connected to the device driver rather than a physical file, and the device driver is smart enough to know that separate processes should be handled separately.
The same is true for the Linux /proc filesystem. Those aren’t real files, just tightly controlled gateways to kernel information.
Thats for your response. While I can understand the purpose of the files from what you describe, I would like to move a level more. when I open these files in /dev folder, how come I never get to see the output of a process running. Say I execute top on the terminal, is it not supposed to output its results onto the stdout file periodically, hence when it is being updated I should be able to see an instance of the output being printed onto this file. But this is not so.. So are these files not the same (the ones in the /dev directory).
Because those aren’t technically files. They’re device nodes, indicating a specific device to write to. UNIX may present everything to you as a file abstraction but that doesn’t make it so at the deepest levels.
@Shouvik, note that /dev/stdin is a symlink to /proc/self/fd/0 — the first file descriptor that the currently running program has open. So, what is pointed to by /dev/stdin will change from program to program, because /proc/self/ always points to the ‘currently running program’. (Whichever program is doing the open call.) /dev/stdin and friends were put there to make setuid shell scripts safer, and let you pass the filename /dev/stdin to programs that only work with files, but you want to control more interactively. (Someday this will be a useful trick for you to know. 🙂
@CarlosW.Mercado, a file is a physical manifestation of the data. For example, the bits stored on the hard disk. A file handle is (usually) a small token used to refer to that file, once you have opened it.
Note that stderr is intended to be used for diagnostic messages in general. Actually, only the normal output should be written to stdout . I sometimes see people splitting the log in stdout and stderr . This is not the intended way of useage. The whole log should go into stderr altogether.
It would be more correct to say that stdin , stdout , and stderr are «I/O streams» rather than files. As you’ve noticed, these entities do not live in the filesystem. But the Unix philosophy, as far as I/O is concerned, is «everything is a file». In practice, that really means that you can use the same library functions and interfaces ( printf , scanf , read , write , select , etc.) without worrying about whether the I/O stream is connected to a keyboard, a disk file, a socket, a pipe, or some other I/O abstraction.
Most programs need to read input, write output, and log errors, so stdin , stdout , and stderr are predefined for you, as a programming convenience. This is only a convention, and is not enforced by the operating system.
Thanks for your inputs. Would you happen to know how I could intercept the output data stream of a process and output it into a file of my own?
As a complement of the answers above, here is a sum up about Redirections:
EDIT: This graphic is not entirely correct.
The first example does not use stdin at all, it’s passing «hello» as an argument to the echo command.
The graphic also says 2>&1 has the same effect as &> however
ls Documents ABC > dirlist 2>&1 #does not give the same output as ls Documents ABC > dirlist &> This is because &> requires a file to redirect to, and 2>&1 is simply sending stderr into stdout
Your comment combining with the accepted answer makes perfect sense and clearly explains things! Thanks!
&> requires file to redirect in to, but 2>&1 doesn’t &> is for logging, but 2>&1 can be used for logging and terminal STDOUT STDERR view at the same time 2>&1 can be used just to view the STDOUT and STDERR (in the terminal’s command_prompt) of your program (depending on your program’s ability to handle errors)
Actually, that first one is just plain wrong. In no way does «hello» interact with the standard input of echo . In fact, if you replaced echo with a program that read standard input, it would just sit there waiting for your terminal input. In this case, hello is provided as an argument (through argc/argv ). And the comment about 2>&1 and &> having the same effect is accurate if you see the effect as is stated: «merges stderr with stdout». They both do that but in a slightly different way. The equivalence would be > somefile 2>&1 and &> somefile .
I’m afraid your understanding is completely backwards. 🙂
Think of «standard in», «standard out», and «standard error» from the program’s perspective, not from the kernel’s perspective.
When a program needs to print output, it normally prints to «standard out». A program typically prints output to standard out with printf , which prints ONLY to standard out.
When a program needs to print error information (not necessarily exceptions, those are a programming-language construct, imposed at a much higher level), it normally prints to «standard error». It normally does so with fprintf , which accepts a file stream to use when printing. The file stream could be any file opened for writing: standard out, standard error, or any other file that has been opened with fopen or fdopen .
«standard in» is used when the file needs to read input, using fread or fgets , or getchar .
Any of these files can be easily redirected from the shell, like this:
cat /etc/passwd > /tmp/out # redirect cat's standard out to /tmp/foo cat /nonexistant 2> /tmp/err # redirect cat's standard error to /tmp/error cat < /etc/passwd # redirect cat's standard input to /etc/passwd There are two important caveats: First, "standard in", "standard out", and "standard error" are just a convention. They are a very strong convention, but it's all just an agreement that it is very nice to be able to run programs like this: grep echo /etc/services | awk '' | sort and have the standard outputs of each program hooked into the standard input of the next program in the pipeline.
Second, I've given the standard ISO C functions for working with file streams ( FILE * objects) -- at the kernel level, it is all file descriptors ( int references to the file table) and much lower-level operations like read and write , which do not do the happy buffering of the ISO C functions. I figured to keep it simple and use the easier functions, but I thought all the same you should know the alternatives. 🙂
Understanding stdin, stderr and stdout in Linux
There is a decent chance that if you have used Linux operating systems then you might have encountered the three famous data streams known as stdin, stderr and stdout. All these are different in their functions and have their own uses but one thing common between all three of them is that they are data streams that bash creates.
Let’s understand more about what data streams actually mean and how they are beneficial. In terms of computing, a data stream is something that gives us the ability to transfer data from a source to an outflow and vice versa. The source and the outflow are the two end points of the data stream. It might be interesting for you to know that whatever command you are running in your Linux terminal, it will either be at one of these end points.
Now we know a little about the data streams, let’s know more about the three famous data streams.
- stdin − It stands for standard input, and is used for taking text as an input.
- stdout − It stands for standard output, and is used to text output of any command you type in the terminal, and then that output is stored in the stdout stream.
- stderr − It stands for standard error. It is invoked whenever a command faces an error, then that error message gets stored in this data stream.
It should be noted that in Linux all these streams are treated as if they were files. Also, linux assigns unique values to each of these data streams.
Now let’s consider a few examples of these three data streams.
The example shown below depicts a typical stdin stream.
Command
Output
mmukul@192 Docs-Linux % read This is to stdin
In the above command, we are providing an input to the stream and the read tool is getting the input from stdin.
Now, an example of stdout is shown below −
Command
Output
immukul@192 Downloads % ls -ltr total 1085456 drwxr-xr-x@ 13 immukul staff 416 Dec 7 2019 source-code-pro-release -rw-r--r--@ 1 immukul staff 350337 Dec 22 2019 messi.jpg -rw-r--r--@ 1 immukul staff 5953321 Dec 22 2019 927225.png -rw-r--r--@ 1 immukul staff 601852 Dec 22 2019 238870.jpg . . .
We know that we make use of the ls command with the -ltr flag to list all the files in a certain sorted fashion, where the lastly updated file is shown at the bottom. The list is sent to the stdout data stream and the terminal then simply prints it out.
Now, an example of stderr is shown below −
Command
The above command is invalid as I don’t have any directory named printit and it will generate an error message that will be sent to stderr and then the terminal will print it.
Output
immukul@192 Downloads % ls -ltr printit ls: printit: No such file or directory