Browse Prior Art Database

THREAD SWITCHING VERSUS ADAPTER SWITCHING ON PCIe* SLOTS

IP.com Disclosure Number: IPCOM000247205D
Publication Date: 2016-Aug-16
Document File: 4 page(s) / 284K

Publishing Venue

The IP.com Prior Art Database

Abstract

Disclosed is a method to dynamically switch workloads to aid performance or for power savings by either migrating the workload between threads or altering the physical topology of the peripheral component interconnect express (PCIe*) slot(s) to create workload affinity to a CPU.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 44% of the total text.

Page 01 of 4

THREAD SWITCHING VERSUS ADAPTER SWITCHING ON PCIe *


*

SLOTS

SLOTS

Disclosed is a method to dynamically switch workloads to aid performance and for power savings by either migrating the workload between threads or altering the physical topology of the peripheral component interconnect express (PCIe*) slot(s) to create workload affinity to a CPU. In server systems, the input/output ( IO) interface is usually done through PCIe over the past few years. PCIe offers maximum bandwidth and compatibility with different protocols and has proven to be an accepted solution. With increasing speed and expanding memory, the IO bridge may be incorporated within the processor to provide faster access to the IO world. With higher speed PCIe buses, the utilization of the bus is split using a switch. Functions that can multiplex the same bus from bandwidth perspective are grouped and presented in a switch so that they can be shared. In a system, an adapter behind a socket can have affinity towards the other processor and, hence, data transfer would be done through the PCIe bus (IO) to the other chip memory through inter-processor communication bus. Figure 1 depicts a block diagram of a 2S system showing different PCIe slots in the hosts for different adapters.

Figure 1

    
Apart from networking, due to acceleration and non-volatile memory (NVM) express (NVMe**), the requirement of data movement from both the CPUs to an adapter is unavoidable. A slot behind the DCM2 hosts a card. DCM1 has to go via inter-chip bus. Consider the following situation:

Data from DCM1 to its own slot


1.

Data from DCM2 to the slot in DCM1 goes over the X bus


2.

Work Load (WL) thread moves to the other DCM


3.

If free thread not available, data occupies X bus


a.

Current Solutions in the market: Single device multiplexed to two different CPUs through

1


Page 02 of 4

PCIe fabric switch.

Disadvantages:
Involves active switch.

Introduces latency.

Wastes pin out of the processor and switch.

PCIe fabric and switch are active logic.

Thread balancing and adapter balancing is not an option in current solutions.

Disclosed method:
In server systems, if the passive multiplexer (mux) allows switching between two CPUs, then


A.

the workload assigned thread can be moved between CPUs or adapter affinity can be switched using the following approach:

Monitoring.


1.

Monitor the inter-chip bus traffic and the address ranges to see the affinity of an


a.

adapter to a different CPU.

Analyze the buffer and IO affinity to the workload thread.


b.

Decision making.


2.

Analyze the power and thermal take-down due to the card affinity to the next physical


a.

chip.
If the local chip that has an affined adapter, has threads and memory to handle


1.

without degrading the existing performance, move the workload to the other chip. If the local chip cannot handle any more threads or the workload has an affinity


2.

with the thread employed with adjacent chip, make the decision to move the affinity to the next chip by swit...