Browse Prior Art Database

OCR Screen Reader

IP.com Disclosure Number: IPCOM000015806D
Original Publication Date: 2002-Jul-20
Included in the Prior Art Database: 2003-Jun-21
Document File: 6 page(s) / 258K

Publishing Venue

IBM

Abstract

A program is disclosed that screen reader which is critical assistive technology for visually impaired people, can read not only ordinary GUI screen on local computer but also completely imaged GUI screen sent from remote computer by GUI remote control tool. Most of these GUI remote control tools sends every information on the GUI screen as image. The former screen reader obtains text information by monitoring text drawing API on the screen, then read it by speech synthesis technology. So if GUI remote control tool draws remote GUI as image, screen reader can not obtain any information from there.

This text was extracted from a PDF file.
At least one non-text object (such as an image or picture) has been suppressed.
This is the abbreviated version, containing approximately 43% of the total text.

Page 1 of 6

OCR Screen Reader

A program is disclosed that screen reader which is critical assistive technology for visually impaired people, can read not only ordinary GUI screen on local computer but also completely imaged GUI screen sent from remote computer by GUI remote control tool. Most of these GUI remote control tools sends every information on the GUI screen as image. The former screen reader obtains text information by monitoring text drawing API on the screen, then read it by speech synthesis technology. So if GUI remote control tool draws remote GUI as image, screen reader can not obtain any information from there.

OCR Screen Reader is based on former screen reader technology. However, when the updated region is draw as image, OCR Screen Reader obtains screen copy of the updated region as 256 scale RGB image by screen to memory BitBlt operation, then store it to Full color image [1] (Fig. 1).

Fig. 1

After that, OCR Screen Reader converts Full color image [1] to Gray scale image [2] by following rule for all of the constituent pixels (Fig. 1).

I = (R x 0.3 + G x 0.6 + B x 0.1) /64

R: Red luminance (0 to 255) of each pixel in Full color image [1] G: Green luminance (0 to 255) of each pixel in Full color image [1] B: Blue luminance (0 to 255) of each pixel in Full color image [1] I: luminance (0 to 255) of each pixels in Gray scale image [2]

There are abstracted GUI object images (ex. button, icon, and listbox) which has already converted to 4 gray scale as same as Gray scale image [2]. OCR Screen Reader searches these GUI object images on Gray scale image [2] by pattern matching (Fig. 2).

     Full color image [1] 256 scale for red, green, and blue (24 bit/pixel)

Gray scale image [2] 4 gray scale

(2 bit/pixel)

1

[This page contains 3 pictures or other non-text objects]

Page 2 of 6

Fig. 2

If it finds these GUI objects on Gray scale image [2], it records the found location in Off screen model database [3] (Fig. 3). Off screen model database [3] is a screen object management database which has following data for each records;

1) Type of the GUI object
2) Base X position on this region
3) Base Y position on this region
4) width of the GUI object
5) Height of the GUI object

After these found GUI objects are recorded in Off screen model database [3], these GUI objects are erased from Gray scale image [2]. This process creates Object removed image [4] from Gray scale image
[2] (Fig. 3).

Help icon image

Gray scale image [2]

Exit icon image

Listbox image

Button image

Focused Button image

checkbox image

2

[This page contains 21 pictures or other non-text objects]

Page 3 of 6

Fig. 3Gray scale image [2] Fig. 3

Then Object removed image [4] is converted to Monochrome image [5] by following rule for all of the constituent pixels (Fig. 4).

I2 = I4 /2 (truncate under decimal point)

I4: luminance (0 to 3) of each pixels in Object removed image [4] I2: luminance (0 or 1) of each pixels in Monochrome image [5]

Vertical lines and horizontal lines which are long...