This startup can make a digital twin of anyone

How many times have you wished you had a clone of yourself? Now, an Israeli AI video company called Hour One believes it has cracked the code to exactly that — at least where video is concerned.

With just a few minutes in a recording studio, Hour One can produce a digital twin of anyone, capturing their likeness, expressions, and voice. 

It might sound eerie at first, but a new collaboration between the company and creative technologist Taryn Southern offers a glimpse into a future where it’s a regular part of our lives. It’s a vision they believe holds significant opportunities for those willing to take the plunge into the uncanny.

Taryn Southern…But AI

The initiative, “Is This For Real,” is a forthcoming news series hosted by Southern — or rather, by her digital clone. Carrying the emerging technology theme, the show, which premieres this summer, will be a weekly examination of the latest in science and technology.

Like many new developments in media over the past year, Southern said that the production emerged from the constraints imposed by the COVID-19 pandemic.

“When the pandemic hit, production all over the world shut down,” Southern said in a statement. “People were looking for alternate ways to make content and I was curious about what could be produced with AI-generated video.”

Southern is no stranger to incorporating emerging technologies into her practice. She was an early adopter of VR video, and in 2018, she released “I AM AI,” an album co-produced with AI. But her show with Hour One leans the heaviest into the machine end of her ongoing human-machine collaborations. 

“Experimenting with AI video production has been similar to working with AI music,” Southern said. “It provokes important conversations around the future of identity and trust, and will undoubtedly change the future of production.”

But Why?

Hour One isn’t offering much information about its “proprietary AI-driven process” to create a digital clone, but a behind-the-scenes video posted to Southern’s YouTube account shows how she created hers: by heading to a studio to record seven minutes of footage and reading through a “couple scripts and singing a song.” This footage was then uploaded to the cloud, and AI Taryn was “born.” 

New content can now be created by uploading a written script, from which Hour One’s engine will produce a video of the AI clone speaking the text — to which custom graphics and backgrounds can then be added.

But even for its relative ease, the process is still much more involved than simply recording a video yourself. What exactly do Southern and Hour One think will be valuable about having an AI clone, beyond a rousing dinner party conversation?

“When you shift from using cameras to code, entirely new things are possible,” said Hour One Head of Business Natalie Monbiot, in a statement. “As her code-based self, AI Taryn can take on all kinds of new skills, such as the ability to speak any language.”

AI clones can deliver content in any language, bypassing the need for subtitles to connect with audiences anywhere in the world.

AI clones are natural polyglots, able to deliver content in any language, bypassing the need for subtitles to connect with audiences anywhere in the world. Theoretically, these clones could also function as “experts” in realms that their human counterparts know nothing about — it all just comes down to who is writing the script. 

Hour One believes this will reduce the time spent in production — potentially from days to minutes — and creators can generate content regardless of how talent happens to look. 

In her behind-the-scenes video, Southern jokes that she can now create new video content “without the real Taryn having to shower or leave her bed.” It’s also a format that those suffering from serious disease might use to continue producing content while otherwise unable.

Neither Deepfake Nor Novelty

Hour One uses generative adversarial networks, or GANs, to produce these clones. If that term sounds familiar, it’s because GANs are the class of machine learning used to create deepfakes, a form of synthetic media that substitutes faces and/or voices on existing content. 

Importantly, though, what Hour One is doing is not deepfakery. All clones are “original content” created for specific, consenting parties. The company also claims that all videos will have an “Altered Visuals” disclosure of some kind, “to respect the user’s right to know that the video has been computer-generated.”

Despite being a relatively plausible reproduction of Southern, her clone is still identifiably a clone. “Is This For Real” will function more as a proof-of-concept that explores the utility of this process, particularly as the technology improves and as audiences adapt to an increasing presence of synthetic content in their media diets.

For now, the company sees clear benefits for those who spend a lot of time in front of the camera and need to produce a steady, high volume of content, citing the 24-hour news cycle as an example. While the technology might not yet “fool” viewers, a sample set on Hour One’s website shows that, in many cases, they’ll do the job well enough — at a fraction of the cost and effort.

One day, this could provide creators like Southern a new revenue stream through licensing across a variety of content and formats they may not have had the time or availability to produce before. 

For now, the clearest use case of this breed of synthetic media is in domains less reliant on creativity or performance, but that still benefit from presenter-led video content. In areas like HR, eLearning, eCommerce, real estate, and digital health, AI clones might well start popping up in places where we currently encounter chatbots, tour videos, or corporate training content. 

The big question is whether audiences are actually ready to accept synthetic hosts. The uncanny valley has proven to be a notoriously difficult effect to overcome, but the collaborators behind “Is This For Real” are betting that this new wave of AI-driven video might finally mean synthetic clones are here for real.

Those who want to explore becoming an “Hour One character” can check it out on the company’s official application portal.

We’d love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected].