Browse Prior Art Database

Middleware directives for host side tasks in heterogeneous systems

IP.com Disclosure Number: IPCOM000237148D
Publication Date: 2014-Jun-05
Document File: 4 page(s) / 127K

Publishing Venue

The IP.com Prior Art Database

Abstract

Given accelerators from multiple vendors and a middleware that handles them, it becomes increasingly difficult for the user to efficiently manage the whole flow of runtime setup, I/O and execution. Through predefined compiler directives the user informs the middleware that it would synchronize task execution with certain stages in the runtime setup or processing pipeline. The problem of scheduling and synchronization is to be handled by the middleware. This simplifies both the development logic and code offers a good overall optimization of the program from the runtime's perspective, given that the user supplies proper directives.

This text was extracted from a Microsoft Word document.
At least one non-text object (such as an image or picture) has been suppressed.
This is the abbreviated version, containing approximately 51% of the total text.

Middleware directives for host side tasks in heterogeneous systems

Given accelerators from multiple vendors and a middleware that handles them, it becomes increasingly difficult for the user to efficiently manage the whole flow of runtime setup, I/O and execution. Through predefined compiler directives the user informs the middleware that it would synchronize task execution with certain stages in the runtime setup or processing pipeline. The problem of scheduling and synchronization is to be handled by the middleware. This simplifies both the development logic and code offers a good overall optimization of the program from the runtime's perspective, given that the user supplies proper directives.

A framework which handles accelerators from multiple vendors may pose problems with regards to latency. Such a framework from the user's point of view will have "host tasks" and "target tasks". Target side tasks represent the actual useful work (executed by accelerators) while host side tasks are about accelerator management (initialization, memory transfers etc). Usually the host side tasks are executed serial leaving the user to introduce any specific optimizations. Such a framework is OpenCL and the idea will be exemplified for it along with code samples for C/C++. The idea described in this document is not specific for OpenCL, hence may be applied for any similar framework.

In OpenCL delay introduced by the runtime (host side tasks) can have a significant impact over the overall execution time. Examples that would require special user handling are: dynamic kernel compilation, command queue creation, enqueue read/write target memory (first write, last read), release/cleanup runtime.

The host side tasks always encompass the same steps to be executed prior to any target side execution. Through predefined compiler directives the user informs the middleware that it would synchronize task execution with certain stages in the runtime setup (host side). By supplying directives the user does not have to parallelize/optimize the host side path of execution and leaves the middleware to handle scheduling and reduce latency. The middleware would dispatch task creation and synchronization to another API such as OpenMP or POSIX Threads. If there is no implementation of such an API from a specific vendor that implements the middleware, the directives would simply be ignored.

Target tasks for which the user will apply directives to the framework are lightweight ones, generally oriented towards I/O or pre-processing/post-processing of data.

The general syntax for a pragma directive should be:

#pragma $vendor_$middleware $wait_action

The following should be supported by pragma directives:

-          wait_$action(platform, args) – wait for an object in a platform to be initialized

o    Example: #pragma fsl wait_build(FSL, prog1), #pragma fsl wait_init(FSL, queue1)

-          wait_$action_all(platform) – wait for all objects in a certain category to be initialized

o    Example: #pragma...