Linux time wait connections

Understanding TCP Connection with Examples

TCP is defined as a connection-oriented and reliable protocol. We will check out how TCP connection works in this article.

How TCP Connection Works?

TCP allows for the transmission of information in both directions. This means that computer systems that communicate over TCP can send and receive data at the same time, similar to a telephone conversation.

The protocol uses segments (packets) as the basic units of data transmission. In addition to the payload, segments can also contain control information and are limited to 1,500 bytes.

The TCP software in the network protocol stack of the operating system is responsible for establishing and terminating the end-to-end connections as well as transferring data.

Establishing a TCP session would begin with a three-way handshake, followed by data transfer, and then a four-way closure. The four-way closure where both sender and receiver agree on closing the session is termed as graceful closure.

After the 4-way closure, the server will allow some minutes of time (default), during which any pending packets on the network are to be processed, this is the TIME_WAIT state. After the TIME_WAIT state completes, all the resources allocated for this connection are released.

Check TCP Connection Status in Linux

To display listeners and connections on Linux we can use the netstat or ss command. While older Linux boxes only support netstat, newer Linux distributions use netstat and ss in parallel. However, with the introduction of ss, netstat is marked as deprecated. Check this post to learn more about TCP sockets.

# Display all the connections
netstat -anpl
# Display only TCP sockets / connections.
netstat -t
ss -t
# Display only UDP sockets / connections.
netstat -u
ss -u

TCP Connection States

• SYN_SENT Indicates that the sender has initiated the active open process with the receiver.
• SYN_RECEIVED Indicates that the receiver has received a SYN segment from the sender.
• ESTABLISHED Indicates that the receiver has received a SYN segment from the sender, the sequence numbers are synchronized, and a connection is established.
• LISTEN Indicates a state of readiness to accept connection requests.
• FIN_WAIT_1 Indicates that an active close process has been initiated. This state forms the first state in the three-step connection termination process.
• TIMED_WAIT Indicates that this side is waiting for acknowledgement from another side after it has initiated an active close process. The wait period is timed by a timer mechanism on the sender’s machine.
• CLOSE_WAIT Indicates that a FIN segment has arrived from another side to begin the process of terminating the connection.
• FIN_WAIT_2 Indicates that the acknowledgement for the FIN segment sent to another side has arrived. This state forms the second state in the three-step connection termination process.
• LAST_ACK Indicates that user input for terminating a connection is obtained and that a FIN segment can now be sent to complete the connection termination process. This state is the last state in the three-step connection termination process.
• CLOSED Indicates that the acknowledgement for the last FIN segment has arrived and that the connection is terminated.

Examples of SYN_SENT TCP Connection

We can use this Python code to connect port 180 on google.com. This will not work as this port is not open. At the same time, we can open a new terminal to check the state of TCP connection. After some time, the connection will be failed.

If we see SYN_SENT connections, this usually means that there is a firewall problem between sender and receiver. The TCP 3-way handshake can not be completed.

# python -c ‘import socket;client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM);client_socket.connect((“google.com”, 180))’

# netstat -anpl|grep :180
tcp 0 1 10.254.222.37:40896 172.217.25.238:180 SYN_SENT 17998/python

Examples of TIME_WAIT TCP Connection

We will try to connect port 80 on google.com with the following command. This time the connection will work. The state of TCP connection will change from SYN_SENT to ESTABLISHED and become to TIME_WAIT in the end.

TIME_WAIT indicates that the local endpoint (this side) has closed the connection. The connection is being kept around so that any delayed packets can be matched to the connection and handled appropriately

This post shares more about the time_wait tcp connection in Linux.

# python -c ‘import socket;client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM);client_socket.connect((“google.com”, 80))’

# netstat -anpl|grep :80
tcp 0 0 10.254.222.37:32912 172.217.25.238:80 TIME_WAIT –

Trace TCP Connection in Linux

In Linux RHEL 8, we can use tcpstates command to trace all the TCP connections.

# /usr/share/bcc/tools/tcpstates -T
TIME SKADDR C-PID C-COMM LADDR LPORT RADDR RPORT OLDSTATE -> NEWSTATE MS
17:14:23 ffff8caa72f86300 17205 yum 192.168.122.166 0 10.4.204.72 443 CLOSE -> SYN_SENT 0.000
17:14:23 ffff8caa72f86300 0 swapper/2 192.168.122.166 0 10.4.204.72 443 SYN_SENT -> ESTABLISHED 252.507

David is a Cloud & DevOps Enthusiast. He has years of experience as a Linux engineer. He had working experience in AMD, EMC. He likes Linux, Python, bash, and more. He is a technical blogger and a Software Engineer. He enjoys sharing his learning and contributing to open-source.

howtouselinux.com is dedicated to providing comprehensive information on using Linux.

We hope you find our site helpful and informative.

Источник

others-How to solve too many time_wait network connections in linux server?

In this post, I would demonstrate how to solve too many time_wait network connections problem in linux server?

2. The problem and solution

2.1 The problem

netstat -n | awk '/^tcp/ <++S[$NF]>END ' TIME_WAIT 4356 CLOSE_WAIT 75 FIN_WAIT1 15 FIN_WAIT2 72 ESTABLISHED 429 CLOSING 1 LAST_ACK 3 

2.2 The solution

2.2.1 What is TIME_WAIT status of TCP connection

This problem is caused by the incorrectly set network configurations, The TIME_WAIT state of TCP, also known as the 2MSL waiting state: When one end of TCP initiates an active shutdown (receives a FIN request), after the last ACK response is sent, that is, after the third handshake is completed, the fourth handshake ACK is sent After the package, it enters the TIME_WAIT state

2.2.2 How to solve too many TIME_WAIT problem?

Change your network configuration file ** /etc/sysctl.conf**, add following configurations:

net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_tw_recycle = 1 net.ipv4.tcp_fin_timeout = 60 net.ipv4.ip_local_port_range = 10000 65000 net.ipv4.tcp_max_syn_backlog = 8192 net.ipv4.tcp_max_tw_buckets = 10000 

When a TCP connection is closed, a delay of 2*MSL in TIME-WAIT state ties up the socket pair for 4 minutes (see Section 3.5 of [Postel81]. Applications built upon TCP that close one connection and open a new one (e.g., an FTP data transfer connection using Stream mode) must choose a new socket pair each time

According to Linux documentation, you should use the TCP_TW_REUSE flag to allow reusing sockets in TIME_WAIT state for new connections.

It seems to be a good option when dealing with a web server that have to handle many short TCP connections left in a TIME_WAIT state.

As described here, The TCP_TW_RECYCLE could cause some problems when using load balancers.

warning: When recycle is enabled, the server can’t distinguish new incoming connections from different clients behind the same NAT device.

After apply the changes, we got this:

[[email protected] etc]# /opt/bin/print_tcp_by_status.sh TIME_WAIT 2268 CLOSE_WAIT 21 FIN_WAIT1 8 FIN_WAIT2 70 ESTABLISHED 461 SYN_RECV 2 LAST_ACK 38 [[email protected] etc]# /opt/bin/print_tcp_by_status.sh TIME_WAIT 2270 CLOSE_WAIT 22 FIN_WAIT1 8 FIN_WAIT2 69 ESTABLISHED 462 LAST_ACK 39 [[email protected] etc]# /opt/bin/print_tcp_by_status.sh TIME_WAIT 2268 CLOSE_WAIT 24 FIN_WAIT1 6 FIN_WAIT2 84 ESTABLISHED 451 SYN_RECV 1 LAST_ACK 38 [[email protected] etc]# /opt/bin/print_tcp_by_status.sh TIME_WAIT 60 CLOSE_WAIT 6 FIN_WAIT1 17 FIN_WAIT2 33 ESTABLISHED 426 SYN_RECV 1 LAST_ACK 5 

3. Summary

In this post, I demonstrated how to solve the too-many-time_wait-problems in linux servers, the key point is to change your network settings to allow reuse/recycle time_wait connections. That’s it, thanks for your reading.

Источник

Huge amount of TIME_WAIT connections says netstat

root@wherever:# netstat Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 localhost:60930 localhost:sunrpc TIME_WAIT tcp 0 0 localhost:60934 localhost:sunrpc TIME_WAIT tcp 0 0 localhost:60941 localhost:sunrpc TIME_WAIT tcp 0 0 localhost:60947 localhost:sunrpc TIME_WAIT tcp 0 0 localhost:60962 localhost:sunrpc TIME_WAIT tcp 0 0 localhost:60969 localhost:sunrpc TIME_WAIT tcp 0 0 localhost:60998 localhost:sunrpc TIME_WAIT tcp 0 0 localhost:60802 localhost:sunrpc TIME_WAIT tcp 0 0 localhost:60823 localhost:sunrpc TIME_WAIT tcp 0 0 localhost:60876 localhost:sunrpc TIME_WAIT tcp 0 0 localhost:60886 localhost:sunrpc TIME_WAIT tcp 0 0 localhost:60898 localhost:sunrpc TIME_WAIT tcp 0 0 localhost:60897 localhost:sunrpc TIME_WAIT tcp 0 0 localhost:60905 localhost:sunrpc TIME_WAIT tcp 0 0 localhost:60918 localhost:sunrpc TIME_WAIT tcp 0 0 localhost:60921 localhost:sunrpc TIME_WAIT tcp 0 0 localhost:60673 localhost:sunrpc TIME_WAIT tcp 0 0 localhost:60680 localhost:sunrpc TIME_WAIT [etc. ] root@wherever:# netstat | grep 'TIME_WAIT' |wc -l 1942 

That number is changing rapidly. I do have a pretty tight iptables config so I have no idea what can cause this. any ideas? Thanks, Tamas Edit: Output of ‘netstat -anp’:

Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 127.0.0.1:60968 127.0.0.1:111 TIME_WAIT - tcp 0 0 127.0.0.1:60972 127.0.0.1:111 TIME_WAIT - tcp 0 0 127.0.0.1:60976 127.0.0.1:111 TIME_WAIT - tcp 0 0 127.0.0.1:60981 127.0.0.1:111 TIME_WAIT - tcp 0 0 127.0.0.1:60980 127.0.0.1:111 TIME_WAIT - tcp 0 0 127.0.0.1:60983 127.0.0.1:111 TIME_WAIT - tcp 0 0 127.0.0.1:60999 127.0.0.1:111 TIME_WAIT - tcp 0 0 127.0.0.1:60809 127.0.0.1:111 TIME_WAIT - tcp 0 0 127.0.0.1:60834 127.0.0.1:111 TIME_WAIT - tcp 0 0 127.0.0.1:60872 127.0.0.1:111 TIME_WAIT - tcp 0 0 127.0.0.1:60896 127.0.0.1:111 TIME_WAIT - tcp 0 0 127.0.0.1:60919 127.0.0.1:111 TIME_WAIT - tcp 0 0 127.0.0.1:60710 127.0.0.1:111 TIME_WAIT - tcp 0 0 127.0.0.1:60745 127.0.0.1:111 TIME_WAIT - tcp 0 0 127.0.0.1:60765 127.0.0.1:111 TIME_WAIT - tcp 0 0 127.0.0.1:60772 127.0.0.1:111 TIME_WAIT - tcp 0 0 127.0.0.1:60558 127.0.0.1:111 TIME_WAIT - tcp 0 0 127.0.0.1:60564 127.0.0.1:111 TIME_WAIT - tcp 0 0 127.0.0.1:60600 127.0.0.1:111 TIME_WAIT - tcp 0 0 127.0.0.1:60624 127.0.0.1:111 TIME_WAIT - 

Источник