Last year, genealogy service MyHeritage went viral after introducing a new “deepfake” feature that allowed users to animate the faces of loved ones in still photos. TikTok users posted videos reacting to the technology, called “Deep Nostalgia”, as they brought back relatives they had never met or those whose loss they still mourned. To date, over 100 million photos have been animated with this feature. Now comes the next iteration. Now, MyHeritage and technology partner D-ID are expanding “Deep Nostalgia” with the launch of “LiveStory,” a feature that not only brings people in photos to life with motion, but makes them talk.
MyHeritage has licensed the new technology from D-ID, a Tel Aviv-based startup that is working to create patented video reconstruction technology powered by AI and deep learning techniques.
D-ID’s technologies, available to developers through APIs, have been used by a range of licensees, in media, education, marketing and more. Warner Bros., for example, worked with D-ID to let users customize a movie trailer with moving photos and for a Harry Potter exhibit. Mondelēz International, advertising agency Publicis and Digitas Vietnam partnered with D-ID for local festival marketing efforts. Indian short video app Josh has incorporated facial animation technology as a creative tool. Non-profit organizations and governments have also used the technology in various public awareness campaigns.
With the launch of LiveStory, MyHeritage brings D-ID’s latest AI technology directly to consumers.
To use this feature, consumers can create a free MyHeritage account, which allows them to try the technology for free multiple times. Beyond that, they’ll need to upgrade to a paid subscription plan for unlimited LiveStory usage.
The technology works to create an animated video narrative of an ancestor’s life, so they can tell their own story. This is powered by D-ID’s patent-pending talking portrait technology, which generates a narrated video based on an uploaded photo, then combines it with a synthetic voice generator. The narration of the story comes from text input by the user.
To match lips to words, D-ID trained a neural network on a database of videos of people speaking. Its technology is capable of working with any language, according to the company. MyHeritage’s implementation, however, supports 31 languages, including dozens of dialects, with both male and female voice options.
“The technology is so good you don’t need a video of the driver,” notes D-ID co-founder and CEO Gil Perry – meaning the technology doesn’t require video of the driver’s movements. ‘a real person which are then mapped to a still image. “You just type in text and the photo, then you get the person to talk,” he says. The technology isn’t perfect, Perry admits, comparing it instead to “really good lip-syncing.”
Once the LiveStory is created, users can watch it, share it with friends, or post it on social media. They can also personalize the story further by editing the text, choosing a different voice, or even uploading their own audio recording.
Picture credits: DID
In the longer term, D-ID envisions how this technology could be used in a metaverse environment, where digital avatars of people could be animated with AI – and not just faces, but full-body movements in worlds. 3D. Perry imagines how users could upload their own childhood photos, those of family members, or even historical figures, then animate them in the metaverse and have conversations.
“Our children will want to chat with Albert Einstein, learn, hear him and ask him questions,” he says. “And he will answer us.” (And when universal translation is available, it can even respond in the user’s language.)
This technology, of course, is several years away, but it will be built on concepts being developed today – like Deep Nostalgia and LiveStory – when and if they eventually come to fruition.
In the meantime, there’s MyHeritage and, soon, D-ID’s own consumer app that will help demonstrate its technology in a different way. The application will arrive in the “coming weeks”, says D-ID.
The MyHeritage LiveStory feature was announced today at the RootsTech Family History Technology Conference. It will work on desktop, mobile web, and in the MyHeritage mobile app.
“LiveStory takes storytelling to the next level,” said Gilad Japhet, Founder and CEO of MyHeritage, in a statement regarding the launch. “With this latest viral feature, MyHeritage continues to lead the world of online family history in both vision and innovation. Our use of AI to breathe new life into historic photos is unique and helps millions of people to cultivate a renewed emotional connection with their deceased ancestors and loved ones. Genealogy is about telling and preserving our family stories. We continue to show the world how fun and compelling genealogy can be.
D-ID, which was also co-founded by Sella Blondheim and Eliran Kuta, is a 32-person team and plans to double in hiring positions in the US, UK, Singapore and Israel.