Distribute Job Orchestra in Multi-Cluster environment
Publication Date: 2016-Feb-29
The IP.com Prior Art Database
This invention is for distribution job orchestra in Multi-Cluster environment and presented two solutions to improve resource utilization: to share resources between different clusters and to redirect jobs to other clusters which have free slots.
Page 01 of 7
Distribute Job Orchestra in Multi
Distribute Job Orchestra in Multi-
In the current Financial Area and Telecom Industries, customer have multiple clusters that may be distributed globally around the world, that means the clusters are distributed in different geographical locations. Currently, a client would typically submit Job to the default cluster and would not be able to use resources in other clusters, even if those resources are idle. So customer would like to get higher global resource utilization and make use of both local and remote resources transparently. But there is no good sharing solution across the clusters, and the Job on each cluster is different, some of them are heavy, but others may be free or the utilization is low. In this invention, we support a solution to participate the workload between all of these clusters, so that Job can be run between each cluster as special rules, the tasks can be done earlier and the resource utilization of clusters will be more efficient. It also has more important reference value for Customer that requires distribute Job placement in Multi-Cluster environment. And also, sharing resources between different clusters will be considered.
So, in this invention, we support two ways to improve the cluster usage rate: by sharing resources among the multiple clusters and redirect jobs to remote clusters in Multi-Clusters environment.
The administrator of the MCJO management cluster, who monitors resource usage across clusters. MCJO management system The machines where the MCJO core components are running.
--Cluster environment Cluster environment
Multi-Cluster Job Orchestra
A central component of MCJO framework.
MCJO management cluster The cluster where the MCJO core components are running. MCJO administrator
An existing work cluster in the customer's environment, on which they can deploy MCJO proxy or submit tasks through MCJO SDK.
A program deployed in the silo cluster that collects information from the silo cluster and reports them to MCJO Server.
A set of API which can redirect client to submit tasks to multi silo clusters.
The client will use MCJO SDK combining original cluster SDK to submit tasks to multi silo clusters
A global console through which user can view the information of all silo clusters that have MCJO proxy deployed.
The work cluster instance where client usually submit workload. To the client this cluster is its local cluster; and to the cluster, the client is local client.
Client intends to connect to its local cluster but the connection may be redirected to other clusters that are remote clusters to the client. The client is remote client to those clusters.
A functional job ("Job") is a set of sessions (1 or more sessions). A functional job cannot span more than 1 cluster
A box can be assimilated to a virtual consumer (at the MCJO level) and can sp...