Linux kernel networking: a general introduction
Linux’s networking is originated from the BSD’s socket just like most of the Unix-like operating system, this is called TCP/IP protocol. The TCP/IP protocol stack contains four layer in concept. The top-most is application layer , then trasport layer , next IP layer and finally the data link layer. Linux networking protocol stack is very complicated, this article will just talk about the general architecture. The following articles will contain more details though I don’t how much there will be.
As we know, there are lots of protocols in the kernel and also there are lots of physical network card in the world. The linux need to abstract the common code and also the special code for every protocol and device. So the function pointer is in everywhere of network subsystem, and actually in everywhere of Linux kernel. Follwing pic shows the Linux core network architecture.
+--------------------------+ | system call interface | +--------------------------+ +------------------------------+ | protocol agnostic interface | +------------------------------+ +------------------------------+ | network protocols | +------+------+-------+--------+ | | | | | | inet | dccp | sctp | packet | | | | | | +------+------+-------+--------+ +------------------------------+ | device agnostic interface | +------------------------------+ +------------------------------+ | device drivers | +------+------+-------+--------+ | | | | | |e1000 | virtio vmxnet| . | | | | | | +------+------+-------+--------+ System call inteface
Easy to understand, all the Unix-like operating system have the same system call interface. The socket, bind, listen, accept, connect and some other system call are all available in all operating system. Also the socket is abstracted as a file descriptor and the usespace interact with the kernel with this fd. The
protocol agnostic interface
This is the struct ‘sock’, as the struct ‘socket’ connect with the VFS(fd) for the userspace, the ‘sock’ connects with the following protocols.
network protocols
Here defines a lot of network protocols, for example the IPV4 protocol stacks, the ipx, irda and the other directory in linux/net directory. And for every network protocol stack, there are a ‘family’ for example the ipv4 is ‘inet_family_ops’. In the initialization, the kernel will add some protocols in the family, for example TCP/UDP.
device agnostic interface
This layer connects the protocols to the various network devices. This contains the common interface for example the device driver can register the network card device using ‘register_netdevice’, also it can send packet using ‘dev_queue_xmit’. They are all not related with a specific protocol and specific network device.
device driver
This layer is the physical networkcard device that does the finally send/receiver packet work. There are lots of network device driver in linux/drivers/net directory.
The next articles will discuss this general picture in more details. Stay hungry, stay foolish.
reference
Anatomy of the Linux networking stack
Demystifying ifconfig and network interfaces in Linux
This post explains ifconfig output of common developer’s box, paying special attention to parameters poorly explained in official documentation. It also slightly touches Linux network interfaces.
ifconfig
ifconfig is a command line tool for UNIX-like systems that allows for diagnosing and configuring network interfaces. At boot time, it sets up network interfaces such as Loopback and Ethernet. Most of the time, however, ifconfig is used for network diagnostics.
Before diving into details of its output, let’s first make clear what is an interface.
network interface
A network interface is a software interface to networking hardware. Linux kernel distinguishes between two types of network interfaces: physical and virtual.
Physical network interface represents an actual network hardware device such as network interface controller (NIC). In practice, you’ll often find eth0 interface, which represents Ethernet network card.
Virtual network interface doesn’t represent any hardware device and is usually linked to one. There are different kinds of virtual interfaces: Loopback, bridges, VLANs, tunnel interfaces and so on. With proliferation of software defined networks, virtual interfaces become wildly used.
Demystifying ifconfig output
Let’s have a look at ifconfig output for a developer’s box with installed Ubuntu and Docker.
$ ifconfig docker0 Link encap:Ethernet HWaddr 02:42:2d:66:fc:f1 inet addr:172.17.0.1 Bcast:0.0.0.0 Mask:255.255.0.0 inet6 addr: fe80::42:2dff:fe66:fcf1/64 Scope:Link UP BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:2 errors:0 dropped:0 overruns:0 frame:0 TX packets:3 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:152 (152.0 B) TX bytes:258 (258.0 B) eth0 Link encap:Ethernet HWaddr 08:00:27:31:65:b5 inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0 inet6 addr: fe80::3db9:eaaa:e0ae:6e09/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1089467 errors:0 dropped:0 overruns:0 frame:0 TX packets:508121 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:903808796 (903.8 MB) TX bytes:31099448 (31.0 MB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:9643 errors:0 dropped:0 overruns:0 frame:0 TX packets:9643 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1 RX bytes:719527 (719.5 KB) TX bytes:719527 (719.5 KB) interfaces
There are 3 network interfaces on the box:
eth0 is a physical interface representing Ethernet network card. It’s used for communication with other computers on the network and on the Internet.
lo is a special virtual network interface called loopback device. Loopback is used mainly for diagnostics and troubleshooting, and to connect to services running on local host.
docker0 is a virtual bridge interface created by Docker. This bridge creates a separate network for docker containers and allows them to communicate with each other.
interface details
Let’s look closely at details of ifconfig output:
Link encap shows how packets are encapsulated for transmission. Most interfaces wrap packets in Ethernet frames.
HWaddr is hardware address of the ethernet interface (also known as MAC address).
inet addr is IPv4 address assigned to the interface.
Bcast is broadcast address for the interface.
Mask is network mask for the interface.
inet6 addr is IPv6 address assigned to the interface.
Scope is scope of IPv6 address. It can be link-local or global. Link-local address is used in local area network and is not routable. Global address is routable.
UP indicates that kernel modules related to the interface have been loaded and interface is activated.
BROADCAST indicates that interface is configured to handle broadcast packets, which is required for obtaining IP address via DHCP.
RUNNING indicates that interface is ready to accept data.
MULTICAST indicates that interface supports multicasting.
MTU is maximum transmission unit. IP datagrams larger than MTU bytes will be fragmented into multiple Ethernet frames.
Metric determines the cost of using the interface. Interfaces with lower cost have higher priority.
interface stats
RX packets is a total number of packets received.
RX errors shows a total number of packets received with error. This includes too-long-frames errors, ring-buffer overflow errors, CRC errors, frame alignment errors, fifo overruns, and missed packets.
RX dropped is a number of dropped packets due to unintended VLAN tags or receiving IPv6 frames when interface is not configured for IPv6.
RX overruns is a number of received packets that experienced fifo overruns, caused by rate at which a buffer gets full and kernel isn’t able to empty it.
RX frame is a number of misaligned frames, i.e. frames with length not divisible by 8.
TX packets is total number of packets transmitted.
TX errors , TX dropped and TX overruns are similar to RX equivalents.
TX carriers is a number of packets that experienced loss of carriers. This usually happens when link is flapping.
TX collisions is a number of transmitted packets that experienced Ethernet collisions.
TX txqueuelen is length of transmission queue.
RX bytes is a total number of bytes received over interface.
TX bytes is a total number of bytes transmitted over interface.
Summary
Despite being superseded by ip command, ifconfig is still commonly used and provides lots of useful details about network interfaces, both physical and virtual.