virtual task for resource usage optimization
Publication Date: 2018-Aug-10
The IP.com Prior Art Database
virtual task for resource usage optimization 1. Background: Some investment banks run the risk analysis program in a data center, such a program
will run thousands of jobs, each job with tens to millions of tasks, the resource usage for
tasks are similar in the same job and may be different in different jobs. There is a task
scheduler to assign tasks to hosts in the data center, how to get the results of important
tasks on time and finish all tasks in a shorter time is a problem.
2. Known solutions: No known solution for this request.
One workaround is to run more tasks on a host to make the resource fully used, the
1: some task may fail because of lack of memory when there are many tasks on a host.
2: some high priority tasks cannot be finished on time because they have to compete for
resources with low priority tasks on the same host.
3: cannot handle the case that task resource usage is changed during runtime.
3. Solution details: Enhance the task scheduler, it will:
Combine several tasks as a virtual task and schedule the virtual tasks to hosts. The
resource request of a virtual task is defined/limited by the high priority task in it.
For a low priority task, after it is scheduled with a virtual task, it can be scheduled with
another virtual task again when there is idle resource, or its priority is increased, such a
task can be finished in a shorter time.
Add a sub task scheduler on each host in the cluster, it will monitor the virtual task