Improved handling of over-load conditions in a networked system through efficient port-space usage
Original Publication Date: 2003-Jun-09
Included in the Prior Art Database: 2003-Jun-09
In the TCP/IP protocol, communication takes place by establishing sockets between the communicating machines. Sockets are associated with the source and destination IP addresses and port numbers which uniquely identify them. Typically, servers will bind to a particular port they are assigned i.e. server ports are typically well known. Clients and other services use temporary, operating system-assigned ports called ephemeral ports. Ephemeral ports are temporary ports assigned by a machine's IP stack, and are assigned from a designated range of ports for this purpose. When the connection terminates, the ephemeral port is available for reuse, although most IP stacks won't reuse that port number until the entire pool of ephemeral ports have been used. So, if the client program reconnects, it will be assigned a different ephemeral port number for its side of the new connection. An important ramification of having an ephemeral port range is that it limits the maximum number of connections from one machine to a specific service on a remote machine. Although the user typically has options to extend this range, the range is restricted by the fact that a port number can be 16-bits wide and thus a maximum of 65535. Most operating systems like Windows have a port range of 4000. A port range of 4000 may seem large, but it is actually small for 21st century computing demands while considering that a TCP connection must expire through the TIME_WAIT state before it is really completed. For example, even if both sides of a connection properly close their ends of the connection, due to TCP's error control, each side must wait until the TIME_WAIT state is expired before the connection's resources can really be disposed. The TIME_WAIT state is twice the MSL (maximum segment lifetime), which, depending on the IP stack, is usually configured to be 240 seconds. That means that a system could have only 4000 connections in a 240-second window, and in practice this can be exhausted. Thus on busy servers (say a proxy server, terminal server or any multi-application server) or on wrongly configured machines (where the ephemeral port space range is not adequate for the number of incoming/outgoing connections) the system could easily run out of ephemeral ports to assign. If TCP/IP runs out of the ephemeral port space then new connections will not be setup (Application will receive a Out of Ports Error - error EADDRINUSE or EGAIN or EADDRNOTAVAIL error depending on the platform). Other than extending the available port range there are no known solutions. But since this range is still restricted by the number of bits used to communicate the port numbers in the TCP header, there are limitations to this approach.