Linux errno broken pipe

What causes the Broken Pipe Error?

I know that broken pipe error is thrown when the socket on the peer side is closed. But, in my test I have noted that an immediate ‘send’ call on this side when the peer side is closed doesn’t always lead to a broken pipe error. E.g.: After closing the socket on peer side (I have tried clean closing by calling close and also abnormal closing by killing the peer), if I try to send 40 bytes, then I don’t get a broken pipe, but, if I try to send 40000 bytes then it immediately gives broken pipe error. What exactly causes broken pipe and can it’s behavior be predicted?

5 Answers 5

It can take time for the network close to be observed — the total time is nominally about 2 minutes (yes, minutes!) after a close before the packets destined for the port are all assumed to be dead. The error condition is detected at some point. With a small write, you are inside the MTU of the system, so the message is queued for sending. With a big write, you are bigger than the MTU and the system spots the problem quicker. If you ignore the SIGPIPE signal, then the functions will return EPIPE error on a broken pipe — at some point when the broken-ness of the connection is detected.

@varevarao: I don’t think that queueing transmissions and sending at specific intervals is a workaround. Queuing transmissions until there’s more than the MTU to send might be a workaround if your application can live with the delays.

The current state of a socket is determined by ‘keep-alive’ activity. In your case, this is possible that when you are issuing the send call, the keep-alive activity tells that the socket is active and so the send call will write the required data (40 bytes) in to the buffer and returns without giving any error.

When you are sending a bigger chunk, the send call goes in to blocking state.

The send man page also confirms this:

When the message does not fit into the send buffer of the socket, send() normally blocks, unless the socket has been placed in non-blocking I/O mode. In non-blocking mode it would return EAGAIN in this case

So, while blocking for the free available buffer, if the caller is notified (by keep-alive mechanism) that the other end is no more present, the send call will fail.

Predicting the exact scenario is difficult with the mentioned info, but I believe, this should be the reason for you problem.

Источник

“[Errno 32] Broken pipe” in Python

At age 30, Python has been considered a mature language. The programming language is hugely popular among data scientist and AI engineers thanks to its simplicity and easy syntax. Despite that, its vague errors usually makes new users pull their hair out to debug.

In this article, we will discuss about [Errno 32] Broken pipe – a popular error message you often see when interacting with the file system. By the end of the article, you will understand why it happens and how to avoid it as well as how to fix your code. You might also be interested in finding out more about other common error messages of Python, such as locale.Error: unsupported locale setting.

What causes “[Errno 32] Broken pipe” in Python?

“Broken pipe” is essentially an IOError error (short for input/output error), which happened at the Linux system level. It usually occurs when reading and writing files, or in other words, doing file input/output or network input/output (via sockets).

The corresponding Linux system error is EPIPE, excerpted from GNU libc error codes:

Macro: int EPIPE

“Broken pipe.” There is no process reading from the other end of a pipe. Every library function that returns this error code also generates a SIGPIPE signal; this signal terminates the program if not handled or blocked. Thus, your program will never actually see EPIPE unless it has handled or blocked SIGPIPE .

From what we’ve just read, we know that [Errno 32] Broken pipe is caused by the system sending SIGPIPE signal, which is an inter-process communication mechanism of Linux.

For example, SIGINT is another signal used internally by Linux system. In Linux, Ctrl C will send a SIGINT signal to end the process, or we can use the kill command to achieve the same effect.

Python does not ignore SIGPIPE by default. Instead, it translate the signal into an exception and raises IOError: [Errno 32] Broken pipe every time it receives a SIGPIPE.

[Errno 32] Broken pipe when pipe outputs in Linux terminal

If you encounter [Errno 32] Broken pipe when trying to pipe output of a Python script to another program such as the below example, read on.

This pipeline syntax will create a process that sends data upstream, and a process that reads data downstream. When the downstream does not need to read upstream data, it will send a SIGPIPE signal to the upstream process.

When downstream no longer needs to read upstream data? For example, the head command in the example only needs to read enough lines to tell the upstream that I no longer need to read it, and it will send the SIGPIPE signal to the upstream process.

When the upstream process is a Python program, an error such as IOError: [Errno 32] Broken pipe will occur.

Avoid [Errno 32] Broken pipe by ignoring SIGPIPE

If you don’t care too much about properly catching SIGPIPE and just need to get things running quickly, add the code snippet below to the top of your Python program.

from signal import signal, SIGPIPE, SIG_DFL #Ignore SIG_PIPE and don't throw exceptions on it. (http://docs.python.org/library/signal.html) signal(SIGPIPE,SIG_DFL) 

What the code does is redirecting SIGPIPE signals to the default SIG_DFL, which the system usually ignore.

But beware, the Python manual on signal library warn against this type of handling SIGPIPE

Do not set SIGPIPE ’s disposition to SIG_DFL in order to avoid BrokenPipeError . Doing that would cause your program to exit unexpectedly also whenever any socket connection is interrupted while your program is still writing to it.

Properly catch IOError to avoid [Errno 32] Broken pipe

Since [Errno 32] Broken pipe is actually a IOError, you can place a try/catch block to catch it like the code snippet below :

import sys, errno try: ### IO operation ### except IOError as e: if e.errno == errno.EPIPE: ### Handle the error ###

Possible solution for [Errno 32] Broken pipe in multi-process program.

In programs that uses worker processes to speed up processing and make use of multi-core CPUs, you can try reducing the number of the worker processes to see whether the error disappear or not.

Conclusion

Properly fixing [Errno 32] Broken pipe requires time and a close look at your code. Most of the time, you can safely ignore the problem if the Python program is relatively simple. Some other times, the solution involves in reducing worker processes in multi-process programs. But we hope that this article offer useful information and help you solve your problem.

Leave a Comment Cancel reply

Hey! I’m Daan. I work as a SysAdmin in the Netherlands. Whenever I find free time, I write about IT, Linux, Open Source and hardware on this site.

VS Code Tutorials

• Automatic code indentation
• Comment out multiple lines
• Quickly duplicate a line in VSCode
• Create a comment block in VSCode
• Show hidden files in VSCode
• Quickly find a file in VSCode
• How to delete the whole line in VSCode
• Collapse code blocks in VSCode
• Enable/disable word wrap in VSCode
• Bind terminal commands to VSCode keyboard shortcuts
• VSCode Command Palette
• VSCode Format On Save
• VSCode «go to definition» guide

Источник

socket.error: [Errno 32] Broken pipe

I wrote a client-server python program where the client sends a list to the server, the server receives the array, deletes the first two elements of the list and sends it back to the client. There is no problem with the server receiving the list. But when the server wants to send back the edited list, it is showing error: socket.error: [Errno 32] Broken pipe . The client.py and the server.py are running from different machines with different ip. I’m posting the code for the client.py and server.py below: Client.py

import socket, pickle HOST = '192.168.30.218' PORT = 50010 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect((HOST, PORT)) arr = ['CS','UserMgmt','AddUser','Arnab','Password'] data_string = pickle.dumps(arr) s.send(data_string) data = s.recv(4096) data_arr1 = pickle.loads(data) s.close() print 'Received', repr(data_arr1) print data_arr1; 

import socket, pickle; HOST = '127.0.0.1'; PORT = 50010; s= socket.socket(socket.AF_INET, socket.SOCK_STREAM); s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1); s.bind(('',PORT)); s.listen(1); conn, addr = s.accept(); print 'Connected by' , addr; data_addr = list(); while 1: data = conn.recv(4096); if not data: break; data_addr = pickle.loads(data); print 'Received Data', repr(data_addr); print data_addr; data_addr.pop(0); data_addr.pop(0); print data_addr; data_string1 = pickle.dumps(data_addr); s.send(data_string1); break; conn.close(); socket.shutdown(); socket.close(); 

Traceback (most recent call last): File "server.py", line 22, in s.send(data_string1); socket.error: [Errno 32] Broken pipe 

How do I fix this problem so that the client can receive the edited list from the server without any error ? Thank You in advance.

Источник

How to prevent errno 32 broken pipe?

Currently I am using an app built in python. When I run it in personal computer, it works without problems. However, when I move it into a production server. It keeps showing me the error attached as below:. I’ve done some research and I got the reason that the end user browser stops the connection while the server is still busy sending data. I wonder why did it happen and what is the root cause that prevents it from running properly in production server, while it works on my personal computer. Any advice is appreciated

 Exception happened during processing of request from ('127.0.0.1', 34226) Traceback (most recent call last): File "/usr/lib/python2.7/SocketServer.py", line 284, in _handle_request_noblock self.process_request(request, client_address) File "/usr/lib/python2.7/SocketServer.py", line 310, in process_request self.finish_request(request, client_address) File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/lib/python2.7/SocketServer.py", line 641, in __init__ self.finish() File "/usr/lib/python2.7/SocketServer.py", line 694, in finish self.wfile.flush() File "/usr/lib/python2.7/socket.py", line 303, in flush self._sock.sendall(view[write_offset:write_offset+buffer_size]) error: [Errno 32] Broken pipe 

6 Answers 6

Your server process has received a SIGPIPE writing to a socket. This usually happens when you write to a socket fully closed on the other (client) side. This might be happening when a client program doesn’t wait till all the data from the server is received and simply closes a socket (using close function).

In a C program you would normally try setting to ignore SIGPIPE signal or setting a dummy signal handler for it. In this case a simple error will be returned when writing to a closed socket. In your case a python seems to throw an exception that can be handled as a premature disconnect of the client.

The broken pipe error usually occurs if your request is blocked or takes too long and after request-side timeout, it’ll close the connection and then, when the respond-side (server) tries to write to the socket, it will throw a pipe broken error.

yes e.g. I wanted to upload too much data on Google Sheets with 1 request. After halving the dataset, it worked.

It depends on how you tested it, and possibly on differences in the TCP stack implementation of the personal computer and the server.

For example, if your sendall always completes immediately (or very quickly) on the personal computer, the connection may simply never have broken during sending. This is very likely if your browser is running on the same machine (since there is no real network latency).

In general, you just need to handle the case where a client disconnects before you’re finished, by handling the exception.

Remember that TCP communications are asynchronous, but this is much more obvious on physically remote connections than on local ones, so conditions like this can be hard to reproduce on a local workstation. Specifically, loopback connections on a single machine are often almost synchronous.

Источник