Linux output to buffer

How to redirect output to a file and stdout

In bash, calling foo would display any output from that command on the stdout. Calling foo > output would redirect any output from that command to the file specified (in this case ‘output’). Is there a way to redirect output to a file and have it display on stdout?

If someone just ended up here looking for capturing error output to file, take a look at — unix.stackexchange.com/questions/132511/…

A note on terminology: when you execute foo > output the data is written to stdout and stdout is the file named output . That is, writing to the file is writing to stdout. You are asking if it is possible to write both to stdout and to the terminal.

@WilliamPursell I’m not sure your clarification improves things 🙂 How about this: OP is asking if it’s possible to direct the called program’s stdout to both a file and the calling program’s stdout (the latter being the stdout that the called program would inherit if nothing special were done; i.e. the terminal, if the calling program is an interactive bash session). And maybe they also want to direct the called program’s stderr similarly («any output from that command» might be reasonably interpreted to mean including stderr).

If we have multiple commands that want to pipe outputs, use ( ) . For example (echo hello; echo world) | tee output.txt

11 Answers 11

The command you want is named tee :

For example, if you only care about stdout:

If you want to include stderr, do:

program [arguments. ] 2>&1 | tee outfile 

2>&1 redirects channel 2 (stderr/standard error) into channel 1 (stdout/standard output), such that both is written as stdout. It is also directed to the given output file as of the tee command.

Furthermore, if you want to append to the log file, use tee -a as:

program [arguments. ] 2>&1 | tee -a outfile 

If OP wants «all output» to be redirected, you’ll also need to grab stderr: «ls -lR / 2>&1 | tee output.file»

@evanmcdonnal The answer is not wrong, it just may not be specific enough, or complete depending on your requirements. There certainly are conditions where you might not want stderr as part of the output being saved to a file. When I answered this 5 years ago I assumed that the OP only wanted stdout, since he mentioned stdout in the subject of the post.

Ah sorry, I might have been a little confused. When I tried it I just got no output, perhaps it was all going to stderr.

Use -a argument on tee to append content to output.file , instead of overwriting it: ls -lR / | tee -a output.file

If you’re using $? afterwards it will return the status code of tee , which is probably not what you want. Instead, you can use $ .

$ program [arguments. ] 2>&1 | tee outfile 

2>&1 dumps the stderr and stdout streams. tee outfile takes the stream it gets and writes it to the screen and to the file «outfile».

This is probably what most people are looking for. The likely situation is some program or script is working hard for a long time and producing a lot of output. The user wants to check it periodically for progress, but also wants the output written to a file.

The problem (especially when mixing stdout and stderr streams) is that there is reliance on the streams being flushed by the program. If, for example, all the writes to stdout are not flushed, but all the writes to stderr are flushed, then they’ll end up out of chronological order in the output file and on the screen.

It’s also bad if the program only outputs 1 or 2 lines every few minutes to report progress. In such a case, if the output was not flushed by the program, the user wouldn’t even see any output on the screen for hours, because none of it would get pushed through the pipe for hours.

Update: The program unbuffer , part of the expect package, will solve the buffering problem. This will cause stdout and stderr to write to the screen and file immediately and keep them in sync when being combined and redirected to tee . E.g.:

$ unbuffer program [arguments. ] 2>&1 | tee outfile 

Источник

Redirect Output of a Process to a File and Standard Streams

The Kubernetes ecosystem is huge and quite complex, so it’s easy to forget about costs when trying out all of the exciting tools.

To avoid overspending on your Kubernetes cluster, definitely have a look at the free K8s cost monitoring tool from the automation platform CAST AI. You can view your costs in real time, allocate them, calculate burn rates for projects, spot anomalies or spikes, and get insightful reports you can share with your team.

Connect your cluster and start monitoring your K8s costs right away:

1. Overview

In this tutorial, we’ll explore few common strategies to redirect the output of a process to a file and standard streams such as stdout and stderr simultaneously.

2. The tee Command

The tee command is one of the most popular Linux commands that we can use for redirecting a process’s output.

2.1. Redirect stdout

Let’s take a simple use case of redirecting the output of the ls command to stdout and a temporary file /tmp/out.log:

$ ls -C | tee /tmp/out.log bin dev home lib32 libx32 mnt proc run srv tmp var boot etc lib lib64 media opt root sbin sys usr 

We can verify that the contents of the file are the same as the output generated from the executed command:

$ cat /tmp/out.log bin dev home lib32 libx32 mnt proc run srv tmp var boot etc lib lib64 media opt root sbin sys usr 

Another important thing to note is that the default behavior of the tee command is to overwrite the contents of the file. However, if required, we can choose the -a option to append the new content after the existing content of a file.

2.2. Redirect stdout and stderr to the Same File

We need to understand that internally, the tee command is acting as a T-splitter for the incoming stdin so that data can be redirected to the stdout and one or more files. Let’s use this understanding to redirect stderr of a process to stdout and a file:

$ (ls -C; cmd_with_err) 2>&1 | tee /tmp/out.log bin dev home lib32 libx32 mnt proc run srv tmp var boot etc lib lib64 media opt root sbin sys usr bash: cmd_with_err: command not found

We can notice that cmd_with_err is an unknown command, so it generates an error message. To make this available to the tee command, we redirect the stderr file descriptor (fd=2) to the stdout file descriptor (fd=1).

Alternatively, we can also use |& as a shorthand notation for 2>&1| to get the same result:

$ (ls -C; cmd_with_err) |& tee /tmp/out.log bin dev home lib32 libx32 mnt proc run srv tmp var boot etc lib lib64 media opt root sbin sys usr bash: cmd_with_err: command not found

Now, let’s verify the content of the /tmp/out.log file:

$ cat /tmp/out.log bin dev home lib32 libx32 mnt proc run srv tmp var boot etc lib lib64 media opt root sbin sys usr bash: cmd_with_err: command not found

2.3. Redirect stdout and stderr to Separate Files

In some scenarios, we might need to redirect the stdout and stderr of a process to separate files. We can do this by using process substitution while invoking the tee command. Before that, let’s take a look at a template code snippet that will enable the tee command to listen to a specific file descriptor and write back to the same file descriptor stream and a file:

We must note that fd is just a placeholder for a file descriptor, and the actual value will be 1 for stdout, 2 for stderr, and 0 for stdin.

Now, let’s use this understanding to redirect the stdout and stderr output of the process to /tmp/out.log and /tmp/err.log, respectively:

$ ((ls -C; cmd_with_err) 1> >(tee /tmp/out.log)) 2> >(tee /tmp/err.log 2>&2) bin dev home lib32 libx32 mnt proc run srv tmp var boot etc lib lib64 media opt root sbin sys usr bash: cmd_with_err: command not found

We can verify that /tmp/out.log contains the valid stdout message, whereas /tmp/err.log contains the error message from stderr:

$ cat /tmp/out.log bin dev home lib32 libx32 mnt proc run srv tmp var boot etc lib lib64 media opt root sbin sys usr $ cat /tmp/err.log bash: cmd_with_err: command not found

3. Redirection Delays

In a few scenarios, invocation of the tee command to redirect the process’s output to a file and stdout can introduce delay. In this section, we’ll explore such a scenario and learn to mitigate it.

3.1. Scenario

Let’s see a simple python script that prints the current time every one second:

$ cat time.py #!/usr/bin/python from datetime import datetime import time import sys from sys import stdout while True: sys.stdout.write(datetime.today().strftime("%H:%M:%S %p\n")) time.sleep(1)

If we execute this script, we’ll observe a delay of one second between any two consecutive timestamps written on the stdout:

$ ./time.py 6:49:48 PM 6:49:49 PM 6:49:50 PM 6:49:51 PM 6:49:52 PM 6:49:53 PM

3.2. Delayed Redirection

Now, let’s use the tee command to redirect the output of this process to stdout and the time.out file:

Unlike earlier, we’ll notice that there’s no output written on stdout for a long time, after which a huge chunk of output will be dumped on stdout in a single go.

The delay is introduced due to Linux’s stdio buffering policy in the glibc, a system library used by python internally. The buffering policy causes the writes to stdout to pass through a 4096–byte buffer, thereby reducing the number of I/O calls required to write on the stream.

For interactive applications, such delays in redirection are not acceptable. So, let’s find ways to mitigate the redirection delay issue.

3.3. Mitigation

As the root cause of the issue is associated with the delay in flushing of data to stdout, one way to solve this issue is by ensuring timely flushing of data to stream in the application code:

$ cat time.py #!/usr/bin/python from datetime import datetime import time import sys from sys import stdout while True: sys.stdout.write(datetime.today().strftime("%H:%M:%S %p\n")) sys.stdout.flush() time.sleep(1)

Let’s verify that the delay is indeed gone:

$ ./time.py | tee time.out 19:29:12 PM 19:29:13 PM

In this scenario, we had direct access to the application code, so we were able to modify it.

But, in many cases, the program could be an executable binary, and we might not have access to modify it. In such a scenario, we can use the unbuffer command in Linux to solve the delay caused by buffered writes to stdout.

Let’s remove the sys.stdout.flush() method call from our script and re-execute the redirection command using the unbuffer command:

$ unbuffer ./time.py | tee time.out 19:34:22 PM 19:34:23 PM

We can observe that there are no unexpected delays in the stdout writes now.

4. Conclusion

In this article, we explored several use cases of redirecting a program’s output to streams such as stdout and stderr simultaneously. Additionally, we learned few strategies to solve the issue of delayed redirection caused due to buffering.

Источник

How to send command output to GNU Screen’s copy mode buffer

Is there a way to send the output of a command to a GNU Screen’s copy mode buffer. I’d like to type something like this:

$ echo 'this is an example' | screen_send_to_copy_mode 

or, maybe C — a Some key and then, after that, when I pressed C — ] I would get this is an example as an output. Is it even possible?

2 Answers 2

You could do something like:

screen_send_to_copy_mode() ( tmp=$(mktemp) || exit cat > "$tmp" && screen -X readbuf "$tmp" ret=$? (sleep 2; rm -f -- "$tmp")& exit "$ret" ) echo 'this is an example' | screen_send_to_copy_mode 

You could redirect the output of the commands to the screen exchange-file.

As the ubication of this file is distribution dependent you should first assign your own file,

C — a : bufferfile your-file

Redirect the output of the commands to your-file and then to update the paste buffer and paste it,

To restore the screen’s default exchange-file,

You must log in to answer this question.

Related

Hot Network Questions

Subscribe to RSS

To subscribe to this RSS feed, copy and paste this URL into your RSS reader.

Site design / logo © 2023 Stack Exchange Inc; user contributions licensed under CC BY-SA . rev 2023.7.14.43533

Linux is a registered trademark of Linus Torvalds. UNIX is a registered trademark of The Open Group.
This site is not affiliated with Linus Torvalds or The Open Group in any way.

By clicking “Accept all cookies”, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy.

Источник