- How to terminate dead connections from the command line without restarting server
- 10 Answers 10
- Here is the contents of that link
- Close established TCP connection on Linux
- 2 Answers 2
- Рабочий метод разрыва конкретного активного соединения из командной строки linux (drop/kill/cut/close ESTABLISHED connection)
How to terminate dead connections from the command line without restarting server
Is there a way to terminate these connections from the Linux command line without restarting the server? After searching, I found a solution called tcpkill , but it will not work for me as it permanently blocks the IP.
10 Answers 10
On linux kernel >= 4.9 you can use the ss command from iproute2 with key -K
ss -K dst 192.168.1.214 dport = 49029
the kernel have to be compiled with CONFIG_INET_DIAG_DESTROY option enabled.
For linux, this is really the best way and pretty much the only way if you have idle connections ( tcpkill could not work). However, I’ll admit that I’ve not inspected killcx but it feels like a lot of security software would prevent that from working unless you modify your iptables to allow these spoofed packets through.
Thanks! Worked like a charm with sudo ss -K . on Ubuntu Bionic 18.04 LTS. I had a tmux process that was stuck at a small screen size because of a remote, but dead, but not timed-out connection. All fixed!
To «kill» a socket, you must send a TCP reset packet. To send it (and be accepted by the other side), you must know the actual TCP sequence number.
1) The already mentioned tcpkill method learns the SEQ number by passively sniffing on the network and waiting for valid packets of this connection to arrive. Then it uses the learned SEQ number to send RSET packets to both sides. However if the connection is idle/hanged and no data flows, it won’t do anything and will wait forever.
2) Another method uses perl script called killcx (link to Sourceforge). This actively sends spoofed SYN packets and learns the SEQ number from the answer. It then sends RSET packets the same way as tcpkill .
Alternatively approach (based on what you want to achieve) is to use gdb debugger to attach to a process owning this socket/connection and issue close() syscall on its behalf — as detailed in this answer.
If you want to deal only with hanged connections (the other side is dead), there are various timeouts (TCP keepalive for example), which should automatically close such connections if configured properly on the system.
@AlexanderGonchiy for live connections it would prevent the process to respond to packets, so it would cause the connection to timeout. For idle connections nothing would happen. I’m not sure if kernel would send anything to the network on closing the fd.
tcpkill might do it for you. In Ubuntu it is in the dsniff package.
$ sudo tcpkill -i wlan0 host 192.168.1.214
(or some other tcpdump like expression for what connection to kill).
This works only if the connection is transmitting anything. It will not work for hanged/idle TCP connections (see my answer for details)
Do — as root netstat -tunp|grep 49029 . The last column of the output should show you the PID and program name of the process responsible for that connection.
If you are lucky there is a single process for just that connection.
If you are unlucky it gets more complicated (the PID is responsible for more than just that one connection). What kind of service is this?
Why do you want to terminate that session?
You may try to use iptables REJECT with —reject-with tcp-reset , which would send RST to remote when it matches a packet.
tcpkill cannot close a dead (hanged) connection. It is based libpcap , it construct a packet to sent FIN packet. If the connection is already dead, it cannot get the right sequence number.
The only way is to close the process, so makes everywhere is NOT SPOF.
DROP packets for incoming/outgoing IP address using:
sudo iptables -A INPUT -s 50.17.54.5 -j DROP sudo iptables -A OUTPUT -d 50.17.54.5 -j DROP
Undo the change when finished and delete the rules you added using:
sudo iptables -D INPUT -s 50.17.54.5 -j DROP sudo iptables -D OUTPUT -d 50.17.54.5 -j DROP
Adding to Marki555 answer which is a great list of options on how to do this.
I found another option — which is long — but also works for idle connections. It is to use a kernel debugger to get the TCP sequence number and then send FIN packets (I believe) using hping3. From https://blog.habets.se/2017/03/Killing-idle-TCP.html
Here is the contents of that link
Killing idle TCP connections
Mar 15, 2017, Categories: network,linux
Let’s say you have some TCP connections to your local system that you want to kill. You could kill the process that handles the connection, but that may also kill other connections, so that’s not great. You could also put in a firewall rule that will cause the connection to be reset. But that won’t work on a connection that’s idle (also if one side is initiator then using this method the other side would not tear down its side of the connection). There’s tcpkill, but it needs to sniff the network to find the TCP sequence numbers, and again that won’t work for an idle connection.
Ideally for these long-running connections TCP keepalive would be enabled. But sometimes it’s not. (e.g. it’s not on by default for gRPC TCP connections, and they certainly can be long-running and idle).
You could also do this by attaching a debugger and calling shutdown(2) on the sockets, but having the daemon calling unexpected syscalls thus getting into an unexpected state doesn’t really make for a stable system. Also attaching a debugger hangs the daemon while you’re attached to it.
This post documents how to do this on a Debian system.
If a client connects to a dual-stack hostname it’ll (usually, see RFC3484) first try IPv6, and then IPv4 if that fails.
If a server comes up after the client tries IPv6 then it’ll fall back to IPv4, even though IPv6 would have worked at that time too.
I want to kick the IPv4 clients over to IPv6, since restarting the server (or even rebooting the server) doesn’t change anything about the race, and I don’t want to restart the clients because they’re doing long-running compute work that I don’t want to lose state on.
With IPv6 I can differentiate hosts behind NAT, for example.
Take the date from uname -a and add a week or so, and open the Debian archive for that day. E.g. http://snapshot.debian.org/archive/debian/2017031500T000000Z/pool/main/l/linux/. Download the -dbg version of the kernel you’re running. E.g.: linux-image-3.16.0-4-amd64-dbg_3.16.39-1+deb8u2_amd64.deb 351181890 2017-03-10 03:37:13
mkdir tmpkernel cd tmpkernel dpkg -x ../linux-image….deb . cp ./usr/lib/debug/lib/modules/*/vmlinux .
$ ss -e -t dst 10.0.64.123 State Recv-Q Send-Q Local Address:Port Peer Address:Port ESTAB 0 0 ::ffff:192.0.2.1:22 ::ffff:10.0.64.123:30201 uid:1003 ino:68386802 sk:ffff88000caa2800
sudo apt-get install crash sudo crash -e emacs ./vmlinux
crash> struct tcp_sock.rcv_nxt,snd_una ffff88000caa2800 rcv_nxt = 2691239595 snd_una = 3825672049
hping3 -s 22 -c 1 -M 3825672049 -L 2691239595 -F -A -p 30201 10.0.64.123 hping3 -s 30201 -c 1 -L 2691239595 -M 3825672049 -F -A -p 22 -a 10.0.64.123 192.0.2.1
netstat -napW | grep 10.0.64.123 If possible you may want to check the remote end too. But if it’s the client that will eventually send traffic then it’ll be cleanly disconnected at that point.
Close established TCP connection on Linux
I am not able to find an answer to a simple thing I will try to achive: once a tcp connection is established to my linux server, let’s say ssh / tcp 22 or x11 / tcp 6000 display -> how do I close this connection without killing the process (sshd / x11 display server). I saw also some suggestoin to use iptables, but it does not work for me, the connection is still visible in netstat -an. would be good if someone can point me to the right direction. what I tried so far
tcpkill: kills the process, not good for me iptables: does not close the established connection, but prevent further connections.
Attach to the process with a debugger, then call shutdown() followed by close() , or just close() , on the appropriate file descriptor. Then hope the process can handle that.
2 Answers 2
Ok, I found at least one solution (killcx) which is working. Maybe we will be able to find an easier solution. Also, i saw the comment from «zb» — thanks — which might also work, but I was not able to find a working syntax, since this tool seems to be really useful but complex. So here is an example how to work with the 1. solution which is working for me:
netstat -anp | grep 22 output: tcp 0 0 192.168.0.82:22 192.168.0.77:33597 VERBUNDEN 25258/0 iptables -A INPUT -j DROP -s 192.168.0.77 (to prevent reconnect) perl killcx.pl 192.168.0.77:33597 (to kill the tcp connection)
killcx can be found here: http://killcx.sourceforge.net/ it «steals» the connection from the foreign host (192.168.0.77) and close it. So that solution is working fine, but to complex to setup quickly if you are under stress. Here are the required packages:
apt-get install libnetpacket-perl libnet-pcap-perl libnet-rawip-perl wget http://killcx.sourceforge.net/killcx.txt -O killcx.pl
however, would be good to have an easier solution.
Рабочий метод разрыва конкретного активного соединения из командной строки linux (drop/kill/cut/close ESTABLISHED connection)
Иногда бывает необходимо принудительно разорвать активное соединение. Самый распространенный способ:
Проблема в том, что один воркер может одновременно обслуживать несколько соединений, поэтому правильнее будет убить соединение а не воркер. Для этого на многих форумах рекомендуют использовать tcpkilll, cutter или awk+hping3. Однако, хоть эти утилиты и находятся в официальных репозиториях, мне не удалось заставить их разрывать соединения.
После продолжительных поисков был обнаружен perl скрипт killcx, разрывающий соединение по удаленному хосту и порту.
killcx 94.133.119.242:4403
[PARENT] checking connection with [94.133.119.242:4403]
[PARENT] found connection with [78.220.184.126:80] (ESTABLISHED)
[PARENT] forking child
[CHILD] interface not defined, will use [eth0]
[CHILD] setting up filter to sniff ACK on [eth0] for 5 seconds
[CHILD] hooked ACK from [77.220.184.126:80]
[CHILD] found AckNum [3091573605] and SeqNum [3105164779]
[CHILD] sending spoofed RST to [78.220.184.126:80] with SeqNum [3091573605]
[CHILD] sending RST to remote host as well with SeqNum [3105164779]
[CHILD] all done, sending USR1 signal to parent [13723] and exiting
[PARENT] received child signal, checking results.
=> success : connection has been closed !
Для его работы требуется целый набор библиотек.
apt-get install \
libpcap0.8 \
libpcap-dev \
libnet-pcap-perl \
libyaml-perl \
libyaml-dev \
libyaml-0-1 \
-y
cpan -i \
Net::RawIP \
NetPacket::Ethernet