Browse Prior Art Database

System and method for event log mining from desktop screen recordings of multiple agents

IP.com Disclosure Number: IPCOM000254236D
Publication Date: 2018-Jun-13
Document File: 2 page(s) / 79K

Publishing Venue

The IP.com Prior Art Database

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 77% of the total text.

System and method for event log mining from desktop screen recordings of multiple agents

We describe a system that aims to mine event logs from screen recordings of desktop of BPO practitioners who are using multitude of tools to execute various steps in business process.

The described system does the following tasks to achieve the above objective of mining event logs:

1. Extract tool specific event frames along with user profile and event time stamps from screen recordings using event detection, OCR and supervised classification methods.

2. Annotate event frames with user profile and case profile details using OCR, data from ERP DBs and HR DBs.

3. Generate Event Labels using process models and pre-labelled form data.

Figure 1 provides the system overview:

In the following section we describes the various steps to achieve the tasks the system need to perform to mine the event logs from multi-modal data.

1. Tool Specific Event Frame Generator:

1. Video segments of process steps are derived from process demo videos (screen- cast to procedural segment videos)

2. Build tool classifier model using pre-labelled form frames

3. Mark event start and event ends in recorded videos using frame differencing methods

4. Label tool classes for each event and filter event frames matching pre-defined relevant form images

5. Extract user-id, time stamp and set of tool specific relevant event frames from the recordings using OCR and clustering methods.

2. Event Profile Annotator:

1. Event profi...