Browse Prior Art Database

Affinitize network processes dynamically (tcp soft affinity) to improve multiprocessor network scalability

IP.com Disclosure Number: IPCOM000012675D
Original Publication Date: 2003-May-20
Included in the Prior Art Database: 2003-May-20
Document File: 2 page(s) / 42K

Publishing Venue

IBM

Abstract

The memory access latency is one of the bottleneck in today's computer system. The memory access latency does not improve in the pace of the processor speed improvement. This article describes a method called tcp_soft_affinity which improves the network scalability in symmetrical multiprocessor systems (SMP) by reducing the memory access latency in network processing. TCP_SOFT_AFFINITY makes the "network process context" processing and the "network interrupt context" processing to occur on the same processor, avoiding loading the network data on two different processors thereby reducing the memory access latency and improving the SMP network scalability.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 51% of the total text.

Page 1 of 2

  Affinitize network processes dynamically (tcp soft affinity) to improve multiprocessor network scalability

The scalability of the symmetrical multiprocessor (SMP) system depends on many factors in general. In this article, we look at the network I/O scalability on these SMP systems. The ideas explored for network scalability are also applicable to the whole I/O subsystem (network, disk etc.,) scalability. For the I/O subsystem, mostly the processing of the data is done in different processing execution. That is, when a user application issues a request to get data (either through network or from local disk), the request is processed in different contexts such as process and interrupt contexts. In SMP systems these different context processing can happen on different processors. Our network experiments showed that most of the time these two processing contexts happened in two different processors and that led to poor scalability.

When the process context and the interrupt context for the same network I/O request get split between processors, then the same data is being touched in two different processors. Let us take the example of receiving network data, when a user application requests data that does not reside locally, then the request is sent out through network and the process waits for the data to arrive. When the data is received, the interrupt context or bottom half processing is done in a processor where the interrupt was routed by the interrupt management. The interrupt management (or the interrupt router) does not have a clue who needs this data and where or on which processor the waiting process is going to run next. The interrupt router routes the data to a processor that it deems appropriate. The data is brought into the L2 cache for that processor to process the request. Then the bottom half processing wakes the waiting thread/process. If this woken process executes on another processor then the data needs to be reloaded into the L2 cache for the processor where the woken thread/process executes. This affects the network scalability as the same data needs to be loaded into two different L2 caches of the SMP systems. Even worse if the data is modified, then it needs to be synchronized before it gets copied to the second processor's L2 cache. This leads to increase memory latency. Let us see why can't we let the interrupt management to tell where to route the data. In case of complex applications, it is not easy to recognize the...