What is the optimal size of a UDP packet for maximum throughput?
Some of the receivers of the data are embedded systems for which TCP/IP stack is not implemented.
I know that this place is filled with people who are very adament about using what’s available. But I hope to have better answers than just focusing on MTU alone.
At the customer site, the network load is unpredictable, and can be very high, resulting many losses. But each time we test, we get different results. We can’t repeat the traffic patterns at our lab setup. And, there are limits on how much testing we can do at the customer’s.
You might want to look at doing some network impairment. We found a cheap and reasonably good product called the mini-maxwell .
are you requiring the far end to acknowledge receipt of the packets? that’ll have more effect on latency than MTU issues.
7 Answers 7
The best way to find the ideal datagram size is to do exactly what TCP itself does to find the ideal packet size: Path MTU discovery.
TCP also has a widely used option where both sides tell the other what their MSS (basically, MTU minus headers) is.
@sep61: if it did not, there would be no reason for TCP to use PMTUD. Discovering the PMTU has a cost, but the implementors of TCP felt the benefits justfied the costs.
Downside to PMTUD is when overzealous and underinformed sysadmins break it by disabling ALL ICMP on their firewalls.
Path MTU Discovery is an IP functionnality, as far as I remember. So it should be also available to UDP.
Alternative answer: be careful to not reinvent the wheel.
TCP is the product of decades of networking experience. There is a reson for every or almost every thing it does. It has several algorithms most people do not think about often (congestion control, retransmission, buffer management, dealing with reordered packets, and so on).
If you start reimplementing all the TCP algorithms, you risk ending up with an (paraphasing Greenspun’s Tenth Rule) «ad hoc, informally-specified, bug-ridden, slow implementation of TCP».
If you have not done so yet, it could be a good idea to look at some recent alternatives to TCP/UDP, like SCTP or DCCP. They were designed for niches where neither TCP nor UDP was a good match, precisely to allow people to use an already «debugged» protocol instead of reinventing the wheel for every new application.
This is more a comment on the question than an answer to the question. Sometimes, you’re just required to do this, for instance out of historical reason: an application has been using UDP for decades, and the scope of usage is changing, and you need to adapt the UDP communication to something new. You don’t want to lose the decades of experience your team put into the application protocol, so it might look like reimplementing the wheel to outsiders, whereas it is just making an already existing wheel smarter.
@CesarB «Some of the receivers of the data are embedded systems for which TCP/IP stack is not implemented.» there is no hammer available. Assume that if someone asks questions like that, they have carefully considered the alternatives and only the proposed solutions are available. It is utterly disrespectful to disregard the constraints of the question.
Well, I’ve got a non-MTU answer for you. Using a connected UDP socket should speed things up for you. There are two reasons to call connect on your UDP socket. The first is efficiency. When you call sendto on an unconnected UDP socket what happens is that the kernel temporarily connects the socket, sends the data and then disconnects it. I read about a study indicating that this takes up nearly 30% of processing time when sending. The other reason to call connect is so that you can get ICMP error messages. On an unconnected UDP socket the kernel doesn’t know what application to deliver ICMP errors to and so they just get discarded.
The easiest workaround to find mtu in c# is to send udp packets with dontfragment flag set to true. if it throws an exception, try reduce the packet size. do this until there is no exception thrown. you can start with 1500 packet size.
lttlrck. youre incorrect. udp and ping may have different header size. much so with windows vs linux. so, the most foolproof method to find max packet size in udp environment is by sending ‘dontfragment packets’ in UDP environment.
Whether it is UDP/TCP/SCTP makes no difference at all to the MTU. Using ping with the ‘no fragment’ (-f on Windows) is a well known way to discover the MTU.
IP header is >= 20 bytes but mostly 20 and UDP header is 8 bytes. This leaves you 1500 — 28 = 1472 bytes for you data. PATH MTU discovery finds the smallest possible MTU on the way to destination. But this does not necessarily mean that, when you use the smallest MTU, you will get the best possible performance. I think the best way is to do a benchmark. Or maybe you should not care about the smallest MTU on the way at all. A network device may very well use a small MTU and also transfer packets very fast. And its value may very well change in the future. So you can not discover this and save it somewhere to use later on, you have to do it periodically. If I were you, I would set the MTU to something like 1440 and benchmark the application.
Another thing to consider is that some network devices don’t handle fragmentation very well. We’ve seen many routers that drop fragmented UDP packets or packets that are too big. The suggestion by CesarB to use Path MTU is a good one.
Maximum throughput is not driven only by the packet size (though this contributes of course). Minimizing latency and maximizing throughput are often at odds with one other. In TCP you have the Nagle algorithm which is designed (in part) to increase overall throughput. However, some protocols (e.g., telnet) often disable Nagle (i.e., set the No Delay bit) in order to improve latency.
Do you have some real time constraints for the data? Streaming audio is different than pushing non-realtime data (e.g., logging information) as the former benefits more from low latency while the latter benefits from increased throughput and perhaps reliability. Are there reliability requirements? If you can’t miss packets and have to have a protocol to request retransmission, this will reduce overall throughput.
There are a myriad of other factors that go into this and (as was suggested in another response) at some point you get a bad implementation of TCP. That being said, if you want to achieve low latency and can tolerate loss using UDP with an overall packet size set to the PATH MTU (be sure to set the payload size to account for headers) is likely the optimal solution (esp. if you can ensure that UDP can get from one end to the other.
Udp packet size linux
NAME
udp - User Datagram Protocol for IPv4
SYNOPSIS
#include #include netinet/in.h> #include netinet/udp.h> udp_socket = socket(AF_INET, SOCK_DGRAM, 0);
DESCRIPTION
This is an implementation of the User Datagram Protocol described in RFC 768. It implements a connectionless, unreliable datagram packet service. Packets may be reordered or duplicated before they arrive. UDP generates and checks checksums to catch transmission errors. When a UDP socket is created, its local and remote addresses are unspecified. Datagrams can be sent immediately using sendto(2) or sendmsg(2) with a valid destination address as an argument. When connect(2) is called on the socket, the default destination address is set and datagrams can now be sent using send(2) or write(2) without specifying a destination address. It is still possible to send to other destinations by passing an address to sendto(2) or sendmsg(2). In order to receive packets, the socket can be bound to a local address first by using bind(2). Otherwise the socket layer will automatically assign a free local port out of the range defined by /proc/sys/net/ipv4/ip_local_port_range and bind the socket to INADDR_ANY. All receive operations return only one packet. When the packet is smaller than the passed buffer, only that much data is returned; when it is bigger, the packet is truncated and the MSG_TRUNC flag is set. MSG_WAITALL is not supported. IP options may be sent or received using the socket options described in ip(7). They are processed by the kernel only when the appropriate /proc parameter is enabled (but still passed to the user even when it is turned off). See ip(7). When the MSG_DONTROUTE flag is set on sending, the destination address must refer to a local interface address and the packet is sent only to that interface. By default, Linux UDP does path MTU (Maximum Transmission Unit) discovery. This means the kernel will keep track of the MTU to a specific target IP address and return EMSGSIZE when a UDP packet write exceeds it. When this happens, the application should decrease the packet size. Path MTU discovery can be also turned off using the IP_MTU_DISCOVER socket option or the /proc/sys/net/ipv4/ip_no_pmtu_disc file; see ip(7) for details. When turned off, UDP will fragment outgoing UDP packets that exceed the interface MTU. However, disabling it is not recommended for performance and reliability reasons. Address format UDP uses the IPv4 sockaddr_in address format described in ip(7). Error handling All fatal errors will be passed to the user as an error return even when the socket is not connected. This includes asynchronous errors received from the network. You may get an error for an earlier packet that was sent on the same socket. This behavior differs from many other BSD socket implementations which don't pass any errors unless the socket is connected. Linux's behavior is mandated by RFC 1122. For compatibility with legacy code, in Linux 2.0 and 2.2 it was possible to set the SO_BSDCOMPAT SOL_SOCKET option to receive remote errors only when the socket has been connected (except for EPROTO and EMSGSIZE). Locally generated errors are always passed. Support for this socket option was removed in later kernels; see socket(7) for further information. When the IP_RECVERR option is enabled, all errors are stored in the socket error queue, and can be received by recvmsg(2) with the MSG_ERRQUEUE flag set. /proc interfaces System-wide UDP parameter settings can be accessed by files in the directory /proc/sys/net/ipv4/. udp_mem (since Linux 2.6.25) This is a vector of three integers governing the number of pages allowed for queueing by all UDP sockets. min Below this number of pages, UDP is not bothered about its memory appetite. When the amount of memory allocated by UDP exceeds this number, UDP starts to moderate memory usage. pressure This value was introduced to follow the format of tcp_mem (see tcp(7)). max Number of pages allowed for queueing by all UDP sockets. Defaults values for these three items are calculated at boot time from the amount of available memory. udp_rmem_min (integer; default value: PAGE_SIZE; since Linux 2.6.25) Minimal size, in bytes, of receive buffers used by UDP sockets in moderation. Each UDP socket is able to use the size for receiving data, even if total pages of UDP sockets exceed udp_mem pressure. udp_wmem_min (integer; default value: PAGE_SIZE; since Linux 2.6.25) Minimal size, in bytes, of send buffer used by UDP sockets in moderation. Each UDP socket is able to use the size for sending data, even if total pages of UDP sockets exceed udp_mem pressure. Socket options To set or get a UDP socket option, call getsockopt(2) to read or setsockopt(2) to write the option with the option level argument set to IPPROTO_UDP. Unless otherwise noted, optval is a pointer to an int. UDP_CORK (since Linux 2.5.44) If this option is enabled, then all data output on this socket is accumulated into a single datagram that is transmitted when the option is disabled. This option should not be used in code intended to be portable. Ioctls These ioctls can be accessed using ioctl(2). The correct syntax is: int value; error = ioctl(udp_socket, ioctl_type, &value); FIONREAD (SIOCINQ) Gets a pointer to an integer as argument. Returns the size of the next pending datagram in the integer in bytes, or 0 when no datagram is pending. Warning: Using FIONREAD, it is impossible to distinguish the case where no datagram is pending from the case where the next pending datagram contains zero bytes of data. It is safer to use select(2), poll(2), or epoll(7) to distinguish these cases. TIOCOUTQ (SIOCOUTQ) Returns the number of data bytes in the local send queue. Only supported with Linux 2.4 and above. In addition all ioctls documented in ip(7) and socket(7) are supported.
ERRORS
All errors documented for socket(7) or ip(7) may be returned by a send or receive on a UDP socket. ECONNREFUSED No receiver was associated with the destination address. This might be caused by a previous packet sent over the socket.
VERSIONS
IP_RECVERR is a new feature in Linux 2.2.
SEE ALSO
ip(7), raw(7), socket(7), udplite(7) RFC 768 for the User Datagram Protocol. RFC 1122 for the host requirements. RFC 1191 for a description of path MTU discovery.