- Example of using named pipes in Linux shell (Bash)
- Destroying a named pipe
- An introduction to pipes and named pipes in Linux
- What are Pipes in Linux? How does Pipe Redirection Works?
- Pipes in Linux: The general idea
- Keep in mind: Pipe redirects stdout to stdin but not as command argument
- Types of pipes in Linux
- Read the full story
Example of using named pipes in Linux shell (Bash)
One of the best examples of a practical use of a named pipe.
Another useful behavior is using netcat as a proxy. Both ports and hosts can be redirected. Look at this example:
nc -l 12345 | nc www.google.com 80 Port 12345 represents the request.
This starts a nc server on port 12345 and all the connections get redirected to google.com:80 . If a web browser makes a request to nc , the request will be sent to google but the response will not be sent to the web browser. That is because pipes are unidirectional. This can be worked around with a named pipe to redirect the input and output.
mkfifo backpipe nc -l 12345 0backpipe While this serves as a useful example, if you’re building proxies of this nature you’d generally be better off using socat, which is a Unix command-line tool that essentially implements a very fully featured domain-specific language for building proxies and (to a more limited degree) servers. This particular example would be done more reliably and efficiently with socat STDIO TCP:www.google.com:80 .
mkfifo named_pipe echo "Hi" > named_pipe & cat named_pipe The first command creates the pipe.
The second command writes to the pipe (blocking). The & puts this into the background so you can continue to type commands in the same shell. It will exit when the FIFO is emptied by the next command.
The last command reads from the pipe.
It’s customary for «#» to refer to a root prompt (ie, a prompt in a root shell). There’s nothing here that would require running in a root shell.
The echo will block so this won’t run if executed in the same shell unless second line is place in the background with an ending & .
Open two different shells, and leave them side by side. In both, go to the /tmp/ directory:
mkfifo myPipe echo "IPC_example_between_two_shells">myPipe while read line; do echo "What has been passed through the pipe is $"; done First shell won't give you any prompt back until you execute the second part of the code in the second shell. It's because the fifo read and write is blocking.
You can also have a look at the FIFO type by doing a ls -al myPipe and see the details of this specific type of file.
Next step would be to embark the code in a script!
Creating a named pipe
On Unix-likes named pipe (FIFO) is a special type of file with no content. The mkfifo command creates the pipe on a file system (assigns a name to it), but doesn't open it. You need to open and close it separately like any other file.
Using a named pipe
Named pipes are useful when you need to pipe from/to multiple processes or if you can't connect two processes with an anonymous pipe. They can be used in multiple ways:
- In parallel with another process:
$ echo 'Hello pipe!' > pipe_name & # runs writer in a background $ cat pipe_name Hello pipe! $ # open the pipe on auxiliary FD #5 in both ways (otherwise it will block), $ # then open descriptors for writing and reading and close the auxiliary FD $ exec 5<>pipe_name 3>pipe_name 4&- $ $ echo 'Hello pipe!' >&3 # write into the pipe through FD #3 . $ exec 3>&- # close the FD when you're done $ # (otherwise reading will block) $ cat $ handler() < >cat > exec 3 trap - USR1 # unregister signal handler (see below) > unset -f handler writer # undefine the functions > > $ $ exec 4<>pipe_name 3&- $ trap handler USR1 # register handler for signal USR1 $ $ writer() < >if ; then > kill -USR1 $PPID # send the signal USR1 to a specified process > echo 'Hello pipe!' > pipe_name > fi > > $ export -f writer # pass the function to child shells $ $ bash -c writer & # can actually be run sequentially as well $ Hello pipe! Destroying a named pipe
The pipe itself (and its content) gets destroyed when all descriptors to it are closed. What's left is just a name.
To make the pipe anonymous and unavailable under the given name (can be done when the pipe is still open) you could use the rm console command (it's the opposite of mkfifo command):
An introduction to pipes and named pipes in Linux
In Linux, the pipe command lets you sends the output of one command to another. Piping, as the term suggests, can redirect the standard output, input, or error of one process to another for further processing.
The syntax for the pipe or unnamed pipe command is the | character between any two commands:
Command-1 | Command-2 | …| Command-N
Here, the pipe cannot be accessed via another session; it is created temporarily to accommodate the execution of Command-1 and redirect the standard output. It is deleted after successful execution.
In the example above, contents.txt contains a list of all files in a particular directory—specifically, the output of the ls -al command. We first grep the filenames with the "file" keyword from contents.txt by piping (as shown), so the output of the cat command is provided as the input for the grep command. Next, we add piping to execute the awk command, which displays the 9 th column from the filtered output from the grep command. We can also count the number of rows in contents.txt using the wc -l command.
A named pipe can last until as long as the system is up and running or until it is deleted. It is a special file that follows the FIFO (first in, first out) mechanism. It can be used just like a normal file; i.e., you can write to it, read from it, and open or close it. To create a named pipe, the command is:
This creates a named pipe file that can be used even over multiple shell sessions.
Another way to create a FIFO named pipe is to use this command:
To redirect a standard output of any command to another process, use the > symbol. To redirect a standard input of any command, use the < symbol.
As shown above, the output of the ls -al command is redirected to contents.txt and inserted in the file. Similarly, the input for the tail command is provided as contents.txt via the < symbol.
Here, we have created a named pipe, my-named-pipe , and redirected the output of the ls -al command into the named pipe. We can the open a new shell session and cat the contents of the named pipe, which shows the output of the ls -al command, as previously supplied. Notice the size of the named pipe is zero and it has a designation of "p".
So, next time you're working with commands at the Linux terminal and find yourself moving data between commands, hopefully a pipe will make the process quick and easy.
What are Pipes in Linux? How does Pipe Redirection Works?
There are two kinds of pipes in Linux: named and unnamed. Here's a detailed look at pipe redirection.
You might have used a syntax like cmd0 | cmd1 | cmd2 in your terminal quite too many times.
You probably also know that it is pipe redirection which is used for redirecting output of one command as input to the next command.
But do you know what goes underneath? How pipe redirection actually works?
Not to worry because today I'll be demystifying Unix pipes so that the next time you go on a date with those fancy vertical bars, you'll know exactly what's happening.
Note: I have used the term Unix at places because the concept of pipes (like so many other things in Linux) originates from Unix.
Pipes in Linux: The general idea
Here's what you'll be seeing everywhere regarding “what are Unix pipes?”:
- Unix pipes are an IPC (Inter Process Communication) mechanism, that forwards the output of one program to the input of another program.
Now, this is the general explanation that everyone gives. I want to go in deeper. Let's rephrase the previous line in a more technical way by taking away the abstractions:
- Unix pipes are an IPC (Inter Process Communication) mechanism that takes the stdout of a program and forwards that to the stdin of another program via a buffer.
Much better. Taking away the abstraction made it much cleaner, and more accurate. You can look at the following diagram to understand how pipe works:
One of the simplest example of the pipe command is to pass some command output to grep command for searching a specific string.
For example, you can search for files with name containing txt like this:
Keep in mind: Pipe redirects stdout to stdin but not as command argument
One very important thing to understand is that pipe transfers the stdout of a command to stdin of another but not as a command argument. I'll explain it with example.
If you use cat command without any arguments, it'll default to reading from stdin . Here's an example:
$ cat Subscribe to Linux Handbook for more articles like this ^D Subscribe to Linux Handbook for more articles like this Here I used cat without passing any files, so it defaulted to stdin . Next I wrote a line and then used Ctrl+d to let it know that I'm done writing (Ctrl+d implies EOF or End Of File). Once I was done writing, cat read from stdin , and wrote that very line to stdout .
Now consider the following command:
The second command is NOT equivalent to cat hey . Here, the stdout , "hey", is taken to a buffer, and transferred to the stdin of cat . As there were no command line arguments, cat defaulted to stdin for read, and there was something already in the stdin, so cat command took it, and printed to stdout .
In fact, I created a file named hey and put some content in it. You can see the difference clearly in the image below:
Types of pipes in Linux
There are two kinds of pipes in Linux:
Read the full story
The rest of the article is available to LHB members only. You can sign up now for FREE to read the rest of this article along with access to all members-only posts. You also get subscribed to our fortnightly Linux newsletter.