Multi-Queue Adapter-Directed VM-VM Data Transfer Through a Switch in a Multi-Root Bus
Original Publication Date: 2009-Jul-16
Included in the Prior Art Database: 2009-Jul-16
Disclosed is a method for improving networking performance between virtual machines in a computer system by transferring data directly from one VM to another through a PCI (or other) bus, via a memory mover function, as directed by a Single Root I/O Virtualization (SR-IOV) or multi-queue capable adapter, by using a new PCIe transaction type with single source and single destination address.
As virtualization becomes more prevalent, the situation often arises where two partitions or virtual machines (VMs) on a system connected by an I/O bus fabric need to communicate via some network protocol such as TCP/IP, Fibre Channel, etc. The current default behavior is for the commands and data to go from the initiator all the way out to the network switch and then be routed back to the target - like getting to your next-door neighbor's house by driving around the block. This adds latency to the command/data flow, increases utilization of the links, and adds unnecessary cost and power for the network switches. This problem exists for all forms of networking, such as TCP/IP/Ethernet, Fibre Channel, Infiniband*, etc.
One attempt to solve this problem is to route the commands and data to an external switch through a single Single Root I/O Virtualization (SR-IOV) capable network adapter port. While this approach reduces the number of adapters needed, it still requires the commands and data to flow all the way out to the external network in order to be routed back to the target partition.
Another attempt to solve this problem is to add switching function to the SR-IOV adapter itself. This approach eliminates the need for the data to flow all the way out to an external switch but still requires the data to be framed in the source virtual function (VF), routed to the switch, and then unframed in the target VF. This is an inefficient use of adapter resources, consumes a...