Browse Prior Art Database

Detecting Non-Responding (Hung) Ports in Infiniband Networks

IP.com Disclosure Number: IPCOM000239818D
Publication Date: 2014-Dec-03
Document File: 5 page(s) / 57K

Publishing Venue

The IP.com Prior Art Database

Abstract

Disclosed is a method to detect non-responding (i.e. hung) ports in InfiniBand networks. When an SA (Subnet Administration) agent of a switch or HCA fails, the method uses that as a metric for proactive component phase-out or reboot in the subnet manager (SM) process. Additionally, a method of detecting and proactive phase-out of non-transmitting ports is provided.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 25% of the total text.

Page 01 of 5

Detecting Non-Responding (Hung) Ports in Infiniband Networks

InfiniBand (IB) standard requires a presence of a software entity called "subnet manager" (SM) on one of the nodes in the IB network. The InfiniBand switch firmware only manages the physical link negotiation and neither assigns the InfiniBand addresses nor configures the switch routing tables. The assignments of addresses and routing table configuration are fulfilled by the subnet manager. Without the presence of the subnet manager, the nodes on the network cannot exchange data packets.

The existing available implementation of the subnet manager is OpenSM. It suffers from a number of architectural and standard limitations. The configuration is serialized by full fabric discovery, which leads to long convergence times. OpenSM does not provide handling for "stuck" or misbehaving entities (e.g., switches, Host Channel Adaptors (HCAs), etc.) and cannot handle fast link transitions (e.g., fast boots or link blinks). Additionally, OpenSM election implementation is not race free and allows for a temporary (supposedly short) presence of multiple masters, during which switch routing tables may become inconsistent. Because of this, a prolonged incorrect fabric configuration in which some nodes are not able to communicate is possible with OpenSM, if non-responding ports are present in the fabric.

Because sweeps cannot complete in time, the routing table configuration is delayed. This prevents the timely activation of ports with present physical links. Applications might not be able to reach ports with present physical links, either because the ports have not been configured since the last link appearance or because the ports are in an active state but not present in the routing tables. The last may happen because multiple masters are present and the presence of multiple masters is being resolved.

New OpenSM master instances wipe existing multicast routing tables. Each endpoint is notified about the presence of new SM and is expected to rejoin to multicast groups it wants to continue to use. Thus, if new OpenSM is elected and hung ports are present, then the muticast traffic may be lost until OpenSM is able to complete the full discovery. This may affect critical applications, for example Internet Protocol (IP) over IB and Ethernet over IB, which are widely used and

depend on functional multicast.

These limitations have been solved in XIV implementation of the Subnet Manager standard; XIV solutions can be considered extensions to the standard.

1


Page 02 of 5

Timely detection of non-responding ports is important for a number of reasons. A non-responding port has the following characteristics:

• The connected component has functional physical link(s)

• The connected component remains in the same logical state as before the failure
• The hung port either has issues responding to Subnet Management Protocol (SMP) Management Datagram (MAD) queries or configuration requests, or it ha...