Browse Prior Art Database

USING MACHINE LEARNING FOR REAL TIME RE-LIGHTING AND COLOR ADJUSTMENT IN VIDEO CONFERENCING

IP.com Disclosure Number: IPCOM000249666D
Publication Date: 2017-Mar-15
Document File: 9 page(s) / 2M

Publishing Venue

The IP.com Prior Art Database

Related People

Monica Shen Knotts: AUTHOR [+5]

Abstract

People seldom look their best on video streams in unforgiving conditions such as 1080p and fluorescent lighting. This issue is about more than vanity - studies show that an executive who looks ill has a weakened presence, thereby compromising his or her effectiveness in a meeting. With machine learning, an automated system may re-light and color correct the video stream in real-time to improve the video conferencing experience.

This text was extracted from a PDF file.
This is the abbreviated version, containing approximately 28% of the total text.

Copyright 2017 Cisco Systems, Inc. 1

USING MACHINE LEARNING FOR REAL TIME RE-LIGHTING AND COLOR ADJUSTMENT IN VIDEO CONFERENCING

AUTHORS: Monica Shen Knotts John Apostolopoulos

Rob Liston Dan Tan

Xiaoqing Zhu

CISCO SYSTEMS, INC.

ABSTRACT

People seldom look their best on video streams in unforgiving conditions such as

1080p and fluorescent lighting. This issue is about more than vanity - studies show that an

executive who looks ill has a weakened presence, thereby compromising his or her

effectiveness in a meeting. With machine learning, an automated system may re-light and

color correct the video stream in real-time to improve the video conferencing experience.

DETAILED DESCRIPTION

A combination of in-room physical devices can enhance a high-quality video

experience during a video conference session. Carefully placed room lighting on both

ceiling and walls or on the video conferencing unit may eliminate shadows, and the right

shade of color on the walls, table, and backdrop may reflect warm tones onto the faces of

the meeting participants. This issue is about more than avoiding unflattering under-eye

shadows. Without these color and lighting interventions, people look ill on camera,

diminishing their presence and appearance of confidence to other video conference meeting

participants.

Observers note that distinct wall colors, such as the green used in many corporate

conference rooms, reflect poorly on most people's skin, making them appear ill and less

capable. This may continue to be a common experience, given current color trends (e.g.,

the Pantone Color of the Year 2017 is Greenery PANTONE 15-0343). In addition, most

companies cannot manage the economic cost of physically altering each conference room

with lighting and other equipment.

Copyright 2017 Cisco Systems, Inc. 2

With advances in machine learning, visual aspects of video conferencing

experiences may be digitally improved. More particularly, video content may be

manipulated in real time to improve the appearance of meeting participants. Techniques

described herein observe meeting participants and add/compare to standards of quality

video images (e.g., a learning database). Room-based compensation is performed by re-

lighting and color-adjusting human faces and visible skin of meeting participants. This can

improve a person’s appearance by, for example, creating a natural, healthy appearance in

skin tone, removing under-eye shadows caused by poor placement of overhead lighting,

removing reflection from head surfaces (e.g., thinning hair) caused by poor placement of

overhead lighting, softening facial lines, etc.

The system may learn and improve compensation actions. For example, the system

may learn how to optimize for a particular room (e.g., wall colors, physical lighting

installation, etc.), while adapting to changes in the room (e.g., added whiteboard, changing

color composition, etc.). The system may also improve by applying prior-learned

information from other conference rooms...