Technology has a tremendous promise–for these people around the globe, they will be able to talk and describe themselves perfectly in several languages while looking almost in person. You don’t need to speak that language to appear like you’re fluent in it—to anyone, anywhere. Building a lifelike hologram is a feat in itself, but Microsoft has moved up, incorporated today’s technology, and here they are with a model that can not only create your hologram but also make it communicate with your language.
In a revolutionary breakthrough, Microsoft has not only created an impressive full-bodied hologram but also makes that hologram talk in another language.
Microsoft demonstrated the technologies at the Microsoft Inspire conference in Las Vegas. Microsoft has created a life-size hologram of executive Julia White, which looks and sounds just like her and makes her speak another language with her own voice. This demo was made possible by a fusion of two existing technologies—mixed reality and neural text-to-talk—and foreshadows a future in which technology drastically eliminates existing obstacles to human contact.
Microsoft has previously seen holograms, but the localization element of the technology is a first for the company. Actually, it continues to be a research project as of now, but with Microsoft’s massive capital, it could eventually become a commercial reality. Microsoft has accomplished this accomplishment by integrating two existing technologies: neural text-to-talk and augmented reality. Microsoft has demonstrated hologram technologies in the past. But the modern translation tech bodes well for breaking down walls in the way people connect and interact.
Team Effort
As for how Microsoft pulled off this remarkable demo, the first move was to build White’s seemingly similar, life-size hologram. To do so, she visited one of the company’s Virtual Reality Capture Studios, where she filmed her presentation in English.
That would be enough to encourage someone who wears one of Microsoft’s HoloLens devices to feel like White was standing in front of them, giving her speech. Next, Microsoft used the neural text-to-talk platform of Azure AI to catch White’s personalized speech signature. Since merging the two innovations, White’s hologram can speak in any language with its own speech tones and inflections.
To make it look like White was speaking Japanese, Microsoft used its neural text-to-speech technology to construct White’s “personalized voice signature.” Once it had that, the hologram could say something, in any language, and the audio would sound like White’s voice.
The Future Is Here
Of course, an overseas audience could also benefit from a show like White’s, even though Microsoft did not convert it into a hologram or use AI to mimic the voice—they could watch it on their television or on their computer screen, and listen as a human interpreter to repeat White’s words in their native tongue.
But this technology removes so many of those barriers between the speaker and the audience.
Imagine a world leader giving a message, and people across the globe feeling that a leader was in a room with them and sharing a local language. Or a world-class lecturer delivering a lecture that anybody might attend and understand—without leaving their homes and without studying the language of their teacher.
And then there’s the exciting possibility of what might come next. Mixed reality contact lenses that do away with the need for a headset altogether? AI that can translate speech in the native speaker’s voice in real-time? As White put it at the end of her demo, “All of these technologies exists today. The future is here.”
Even, Microsoft’s presentation is definitely exciting and points to the company’s goals for Azure, HoloLens, and beyond. HoloLens 2 might be aimed at companies for the time being, but Microsoft is working to develop apps and services that can scale to anywhere augmented reality may end up going.
