Direct Attachment of Storage Controllers to Servers over an RDMA-capable Network
Publication Date: 2011-May-23
The IP.com Prior Art Database
Disclosed are mechanisms for directly attaching storage controllers to servers using a Remote Direct Memory Access (RDMA)-capable network such as InfiniBand* (IB) or Internet Wide Area RDMA Protocol (iWARP). Storage controllers typically protect data by appending protection bytes, such as the Small Computer System Interface (SCSI) End-to-End protection bytes defined by T11, to each SCSI block. This metadata is generated as the data enters the storage controller when written by the server, may be checked at various stages within the controller, and is checked before being sent to the server when the data is read. This invention describes mechanisms for generating and checking this metadata in RDMA-capable adapters such as a Host Channel Adapter (HCA), and can be used with industry-standard protocols such as SCSI RDMA Protocol (SRP) and iSCSI Enabled for RDMA (iSER).
Page 01 of 3
Direct Attachment of Storage Controllers to Servers over an RDMA -capable Network
The following descriptions use the example of an IB HCA using SRP protocols, although these mechanisms are also applicable to iWARP RDMA enabled Network Interface Controllers (RNICs) and the iSER protocol. Typical SCSI block sizes of 512 bytes are also used in this example.
An illustration of the SCSI Write operation is given in Figure 1. The SCSI Command Descriptor Block (CDB) is transferred from the server to the storage controller using an SRP command that is sent using standard QP send/receive mechanisms. When the storage controller software decodes the SCSI CDB and determines that a SCSI write is to be performed, it posts an RDMA Read command to its local send queue to fetch the SCSI write data from the server. Any initialization information needed to generate the protection bytes, such as the starting Logical Block Address (LBA) for the first 512-byte block of data, is passed to the HCA in the RDMA Read Work Queue Entry (WQE). The location of the data in the server is specified by the virtual address, R
_Key and length
that is conveyed in the SRP command, and this is also provided in the RDMA Read WQE. The QP that is used for this RDMA Read is enabled for metadata operations by setting the appropriate bit in a QP control register. When the RDMA Read is issued, the "Generate metadata" bit is set in the WQE so that the metadata will be generated by the HCA as it moves the RDMA Read Response data to storage controller memory (hereafter referred to as storage controller cache).
Figure 1: SCSI Write from Server to Storage Controller
(This page contains 00 pictures or other non-text object)
Page 02 of 3
An illustration of the SCSI Read operation is given in Figure 2. As in the SCSI Write case, the SCSI CDB is transferred from the server to the storage controller using an SRP command that is sent using standard QP send/receive mechanis...