Browse Prior Art Database

An Input Method integrated Virtual Display and Realistic Input Disclosure Number: IPCOM000247572D
Publication Date: 2016-Sep-18
Document File: 8 page(s) / 96K

Publishing Venue

The Prior Art Database


This disclosure defines a "Virtual Display and Realistic Input" method integrated with projection, Image Recognition, Optical Character Recognition (OCR), automatic form filling method and Augmented Reality (AR) technologies. Projection and AR technologies are used to show the digital document (e.g. form, table, questionnaires, etc). After the user writes text at the region of the showed document, Image Recognition and OCR technologies are used to recognize and digitize the written text. And then digitized text is filled out the original document automatically after mapping the written text with the input fields of digital document.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 33% of the total text.

Page 01 of 8

An Input Method integrated Virtual Display and Realistic Input

Mobile devices are using more widely. Many businesses or applications require users to fill out registration forms, when users need to participate in activities or use certain features of mobile devices applications. However, because of portability and mobility into account, the mobile devices screen cannot be larger, which leads to filling out the form inconveniently.

Firstly, the mobile devices screen is too small to fill out a form easily if the content is very large. Moreover, it is not easy to navigate. Secondly, It is not easy to locate the relevant fields to fill.

Another scene, when the users use the mobile device to read the article or read a book, because of the mobile devices screen size limitations, it is not convenient to read, but also not easy to turn the page. In addition, when you add notes, the notepad and articles applications had to be switched.

Hence, what is needed is a method that display and fill out forms without the above described problems.

The method in this disclosure enables the user to fill out a digital document (e.g. form, table, questionnaires, etc) in a realistic scene. To implement the input, this method shows the digital document as a virtual scene (for example, to project a form on a paper), andthe user writes text in a realistic scene (for example, to write text on the paper on which the projected form is showed). After that, the method captures the overlapped virtual scene and realistic scene, using Image Recognition and OCR (Optical Character Recognition) technologies, to turn the text of realistic scene into digital text, and then merges the digital text with the content of virtual scene (the digital document)to finish data input. Wherein,

To show the virtual scene, the digital document content would be presented with the following ways (but not limited to):

- projection technology, such as projectors or other devices with the projection function (e.g. smart phones and wearable devices). - augmented reality (AR) technology, such as the AR glasses.

Additionally, the user can input text in a realistic scene with the following ways (but not limited to):

- writing with a pen on any written materials, such as papers, whiteboards and so on. - inputting visible text or content on any other electronic devices.

To fill out the digital document with the text inputted by the user in a realistic scene, the following ways or technologies would be used for different purposes:

- taking a picture or capture the overlapped virtual scene and realistic scene to an image, such as using a camera.


Page 02 of 8

- mapping the written text items with corresponding locations (e.g. input fields, options) in the digital document, and the mapping ways will be introduced in next part.
- digitizing all text entries with OCR technology.
- automatically inputting the digitized texts to the corresponding locations in the digital document.

- Furthermore, Image...