Smart glasses for the hard of hearing are changing theater in London

When my Aunt Nicki visits me in London, we avoid musical theater and the cinema.

Aunt Nicki is hard of hearing. Although there are many enhanced listening devices available to help her, such as an Assistive Living amplifier or a closed captioning screen that sits in a cup holder, she tells me they don't work well enough.

When I recently tried on the new "smart caption glasses" at the Royal National Theatre, I had her on my mind. The theater is testing a pilot program for the technology for all performances of Hadestown and War Horse through October. It plans to make the glasses available for all of its performances during the 2019 season.

The headset, manufactured by Epson in partnership with the Royal National Theatre, is heavier than normal reading glasses but much lighter than bulky virtual reality headsets like the Oculus Rift. The lenses look like a typical pair of glasses, but they fit inside a large gray casing that sits on the sides of your head. When you look through the glasses, closed captions scroll across the bottom of the augmented reality lenses.

Audience members test out the smart caption glasses.

A handheld keypad is attached to the glasses via cable to allow each user to customize the color, size and position of the closed captions. Changing the positioning of the text is key for user comfort.

The biggest challenge is finding the sweet spot of balancing the captions in the foreground with the theater performance in the background.

Here's how it works: The script from an opening-night production is fed into speech software, which follows the performance and adjusts the captions accordingly. This process is refined using sound, video and lighting cues.

"If a performer jumps a few lines, the system will react," said Jonathan Suffolk, the technical director for the Royal National Theatre. "It will take a second or two but the system will react and it will recognize where it is."

Suffolk said some hard of hearing users wear it so they can read lips at the same time as reading the captions.

The glasses are free for attendees and available at every performance. It's a major improvement on existing open captioning, which unlike closed captioning are always in view. This typically includes large screens displaying captions.

Text sits at the bottom of the glasses.

"The response has been overwhelmingly positive for deaf theater-goers who would like to come into the theater anytime they want," Suffolk said of its recent trial. "It's really fundamental that organizations like the National Theatre -- in its really privileged position -- does this type of work for the entire sector."

Richard France -- who works for a group called Deafinitely Theatre, which creates performances for both the deaf and hearing communities -- has worked with many different technologies but calls the smart glasses a potential game changer.

"I'm used to going to the theater with captions in it, but it's much more enjoyable with the glasses because I can follow the movement of the actors on the stage, and they feel is more inclusive," France said.

The technology has been successfully used in US movie theaters, but the Royal National Theatre's challenges are greater because their performances are live.

Advocates for the deaf caution there's no single device or system that will work for all members of the deaf community, but say the availability of this type of device at every Royal National Theatre performance is as much a game changer as the actual technology.

Sarina Roffé, the former president of the National Cued Speech Association, believes it's a solid step toward making live performances more inclusive.

"This new technology allows the deaf and hard of hearing to have complete access," Roffé said. "It's really amazing how far we've advanced and how it has positively affected the quality of life for people who are deaf and hard of hearing."

So perhaps the next time Aunt Nicki comes to London, we'll be able to catch a show together.

Originally published on CNN.com by Samual Burke on November 21, 2018