The terrifying deepfake tool that lets you put words in people’s mouths

As technology develops, it’s getting harder and harder to decipher fiction from reality. From 2017 crowning ‘fake news’ the word of the year, and AI-generated revenge porn victimising countless women, just two weeks ago, Facebook’s video policy was put to the test as a viral Mark Zuckerberg deepfake did the rounds. Our art, culture, and politics are being reformulated by a rapidly developing technology.

With ever-advancing software always at risk of being co-opted for more sinister things, from propaganda to sex crimes, you’d be forgiven for your skepticism when it comes to new (and reportedly exciting) developments in tech. Such apprehension will undoubtedly rear its head when you hear about the latest deepfake tool causing a stir.

Created by a group of scientists from Stanford University, a new research paper – titled Text-based Editing of Talking-head video – demonstrates how to use a tool that quite literally lets you put words in people’s mouths. The innovative algorithm is intended for use in television and film, allowing video editors to “modify talking head videos as if they were editing text – copying, pasting, or adding and deleting words”.

In an explainer video, we see several examples of people saying sentences before part of it is edited – one flawlessly swaps ‘where fore art thou’ for ‘why are you’ in the sentence: ‘O Romeo, Romeo! Where fore art thou Romeo?’ (move over, Shakespeare) – and you see the original person convincingly saying the altered sentence.

So far, so creepy – but also so clever. When utilised for its original purpose, the tool is revolutionary when it comes to video editing, enabling the user to quickly fix a misspoken word by editing the transcript – the application then assembles the intended speech using words, or portions of words, spoken elsewhere in the video. The software can even be used to erase filler words in interviews.

Despite its potentially positive effects, the tool obviously raises some ethical concerns. “Unfortunately, technologies like this will always attract bad actors,” software creator Ohad Fried said, “but the struggle is worth it given the many creative video editing and content applications this enables.”

“Deepfakes are a hot topic as of late,” Haerin Shin, co-lecturer of the Ethics of Artificial Intelligence course at Vanderbilt University, tells Dazed, “so it’s amazing that the software allows people to easily manipulate these videos – it’s not just putting words in people’s mouths, but making the mouths work the words. But the very notion that it’s possible to make people do and say things the way others want them to is deeply alarming.”

With this consistently-evolving tech always open to misuse, should that stop us establishing it in the first place? “Given the history of the digitally networked society, these kind of technologies are bound to emerge,” Shin continues. “There’s no way of controlling it, so it’s (now about) how we implement equally effective measures to prevent their misuse.”

Addressing ethical concerns in their research paper, the Talking-head creators suggest forgery detection, watermarking, and verification systems as options available to prevent (or curtail) misuse. Though when I question the actual security of a watermark, Shin accepts that it’s not – wait for it – a watertight method. “If people are out there to misuse the software,” she explains, “(and they have the skillset) to erase a watermark, they’re going to do it. It’s impossible to prevent completely, but (a watermark) will stop a larger part of the population from attempting to misuse with further ease.”

Shin believes that though we might initially fear widespread corruption of the technology, it’s naive to believe that the general public could easily utilise the deepfake tool, referencing “an illusion of ease” that so often comes with these viral tech developments. 

“There’s no way of controlling it, so it’s (now about) how we implement equally effective measures to prevent their misuse” – Haerin Shin, Vanderbilt University

“We think we know how the mechanism works, but do people actually know how the software operates?” Shin questions. “The surface level ease and accessibility of the interface is deeply deceiving, and that in itself is alarming because it exacerbates a sense of disenchantment (with technology).”

As fake news permeates every inch of our society, it’s understandable that internet users are disillusioned. “It’s necessary to be cynical,” Shin asserts, “so you’re not taken in by the inundation of information.” But when it comes to technology as believable as the Talking-head tool, you have to question whether it’s feasible that people will practice cynicism all the time. With deepfakes getting more and more intelligent, soon it might be impossible to distinguish between fact and fabrication (even if you do exercise vigilance). 

Cansu Canca, the founder and director of AI Ethics Lab, thinks the biggest problem with this deepfake development is its impact on decision-making. “We base our decisions on what we know,” she tells Dazed, “and our knowledge is dependent on the information we receive. Deepfakes manipulate information, and therefore manipulate our decision-making at its early stage. (This) matters immensely when we think of its social repercussions, such as democratic systems or public health.” 

However, Shin is optimistic: “I think until we reach a point where autonomous systems could autonomously self-replicate, self-produce, and self-correct, there will always be a way to control it.” 

So although the Talking-head tool seems terrifying on the surface, it’s unlikely your regular Joe with a MacBook would actually be able to work the software at all, let alone for evil. Plus, if we keep developments like this in the public eye then knowledge of new AI tools, and awareness of potential misuse would spread. This combined with the aforementioned safeguarding measures means we can be fairly confident doctored footage masquerading as fact won’t slip through the cracks. Not sure if that makes it any less creepy though TBH.