Dismiss
InnovationQ will be updated on Sunday, Oct. 22, from 10am ET - noon. You may experience brief service interruptions during that time.
Browse Prior Art Database

Method for dynamically setting batch job memory resource requirements based on historical usage and user suggestion

IP.com Disclosure Number: IPCOM000179647D
Original Publication Date: 2009-Feb-19
Included in the Prior Art Database: 2009-Feb-19
Document File: 4 page(s) / 31K

Publishing Venue

IBM

Abstract

A method for optimizing batch job resource requirements based on historical usage data and user suggestions.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 54% of the total text.

Page 1 of 4

Method for dynamically setting batch job memory resource requirements based on historical usage and user suggestion

Disclosed is a system for dynamically setting batch job memory resource requirements based on historical usage and user suggestion. This is a unique approach because it handles the common problem where users are not familiar with what memory resources are required by their batch jobs. By adding this kind of intelligence to the job scheduler it results in overall better utilization of the batch systems by configuring jobs with more accurate resource requirements.

Below is a list of fields that are required for each batch job submitted.

Required fields:

Userid - The userid of the person submitting the job to the batch system. It can either be a numerical uid or an alphanumeric user name. It must be unique.

Executable

         The execute identifies the name of the program to be executed. Arguments

       - The arguments to be supplied to the executable. Model

      A description field related to the job being run. It is user defined, but should be somewhat unique to whatever process is being executed. It will be used to find similarities between jobs. Usage Score

          Score that determines how well previous similar run had their memory requirement specified. This is defined by the job scheduler not the submitting user.

Optional job Fields:

User Memory Requirement Suggestion

-

will require during execution.

Definitions:

Batch system

          A batch system is a job management system that allows users to run more jobs in less time by matching the jobs' processing needs with the available resources. (From the IBM LoadLevelerâ„¢ manual Chapter One: What is LoadLeveler pg 3)

Batch jobs

        Jobs that are submitted to and dispatched to a system for execution by the batch system.

Memory

-

       When memory is mentioned in the document it is referring to RAM (random access memory).

Memory Requirement

                The amount of memory resource that will be required on the system where the batch job is dispatched. Keep in mind that this is the peak memory usage by the job on the system throughout the entire time that the job is assigned to the system.

Usage Score


A score will be assigned in the history database based on how accurate the memory requirement was compared to the actual memory usage. The score will range from 0 to a maximum value defined by the cluster administrator. How the score is calculated is based on how sensitive the batch system is to being over or under the memory requirements. If the available system in the batch cluster can handle jobs not accurately defining requirement the sensitivity level should be low. If it greatly affects the batch system (normally caused by having limited resources on batch nodes) when jobs inaccurately define memory requirements the sensitivity level should be high. Based on this sensitivity level the score will be calculated base...