Browse Prior Art Database

A Method and System for Dynamically Reconfiguring a Network Based on Traffic Demand of a Big Data Application

IP.com Disclosure Number: IPCOM000234595D
Publication Date: 2014-Jan-21
Document File: 4 page(s) / 72K

Publishing Venue

The IP.com Prior Art Database

Abstract

A method and system is disclosed for dynamically reconfiguring a network depending on network traffic demand of a big data application. Dynamically reconfiguring the network improves network performance in executing the big data application.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 33% of the total text.

Page 01 of 4

A Method and System for Dynamically Reconfiguring a Network Based on Traffic Demand of a Big Data Application

Disclosed is a method and system for dynamically reconfiguring a network based on network traffic demand of a big data application (e.g., using MapReduce, Hadoop, etc.). The method and system reconfigures the network by physically constructing a new network topology and by reconfiguring routing over the new network topology. Further, the method and system uses a Software Defined Networking (SDN) controller for constructing the new network topology by using the SDN controller to configure an optical circuit switch to augment the existing conventional Ethernet network . Configuring the optical circuit switch enables changing physical connectivity between Top Of Rack (TOR) switches by adding new, very high bandwidth paths between racks. The method also reconfigures routing by implementing OpenFlow protocol on the TOR switches, for dynamically routing traffic generated by the big data application to destination racks using either the optical circuit-switched, or conventional packet-switched paths.

The system as shown in figure below includes a cross-layer network control plane

wherein the TOR switches are connected to two aggregation networks, a multi-root tree

with Ethernet switches and the optical circuit switch.

Figure

The system also includes the optical circuit switch connected to each ToR switch in the system via multiple optical uplinks. Further the optical circuit switch can be implemented as a Micro Electro Mechanical Systems (MEMS) based Optical Circuit Switch. Additionally, all switches in the network are controlled by the SDN controller. The system also includes an application scheduler interfaced with the SDN controller . The application scheduler is used by the big data application in managing incoming job requests. The method and system involves using the application scheduler for collecting traffic demands from different big data application jobs . The application scheduler subsequently reports traffic demands to the SDN controller , and instructs the SDN controller to configure the network based on the traffic demand . The method and

1


Page 02 of 4

system also involves using the SDN controller to provide a configuration interface for receiving the traffic demands reported by the application scheduler and the commands issued by the application scheduler. Further, the SDN controller receives traffic demands from the application scheduler in a standard format such as , a traffic demand matrix. The traffic demand matrix describes the volume and policy requirements for traffic exchanged between multiple source racks and multiple destination racks .

The method and system also enables interfacing the SDN controller to a broader coordination framework such as Mesos, for managing multiple applications.

The method and system further enables using the SDN controller as a query interface for providing network information...