Online Trust is Challenged by Deepfakes

"On the internet nobody knows you're a dog."
Peter Steiner wrote this in 1993.

Last year I amended it:
"With AI you can be anyone on the internet."

Generation of deepfakes is now embedded in applications and easily available to anyone. It is built into social media apps like TikTok, Snapchat, Facebook, Instagram, YouTube and more.

We tend to discuss this in terms of misinformation or bullying. That is serious enough, but another huge challenge is identity.

Criminals are using deepfakes to hack systems and to hack people.

We built digital identity systems assuming a person was presenting themselves. 

As I mentioned in my previous post, only a limited number of eID systems actually verify that the user is the rightful owner of the eID. When we add biometric checks, we must now assume that what is presented may be generated.

That is a significant shift, which I will write more about.

I am curious how others see this shift. Are we underestimating it?

Hacking systems with AI

Deepfakes are increasingly being used during digital onboarding. The objective is to create an account in someone else's identity, or create a synthetic identity. The challenge with remote onboarding is that it takes place in an uncontrolled environment, which criminals may manipulate.

Solutions for remote onboarding have implemented many security features to increase confidence that what is captured by the camera is real. These include mechanisms to detect and prevent someone inserting a fake video stream into the application, as well as capturing images in different ways to assess whether there is a real user present at that moment.

But is this sufficient? 

Deepfakes are now so convincing that distinguishing real from generated content is becoming difficult, both for humans and for automated checks. I would claim that liveness detection and injection prevention help, but they are not sufficient on their own. 

A complete onboarding solution should therefore include injection detection, liveness checks and deepfake detection. In practice, however, implementing all three reliably is not trivial.

Deepfakes are becoming more sophisticated every day, and detecting them is becoming increasingly difficult.

In addition, ID documents may also be faked and should be checked. Reading the NFC chip is a good defence against fake documents. However, there are still many older passports in circulation, and not all parts of the world have fully adopted NFC in passports. In addition, there are still many phones that cannot read NFC.

This is also something that must be considered.

We have improved the technical controls, but criminals are increasingly shifting their focus to hacking people using deepfakes. More on that next.

For those working with onboarding and identity verification, I would be interested to hear how you see deepfake attacks are affecting onboarding.

Hacking People with AI

As we tighten security and make it more difficult to hack systems, criminals move to hacking people. This is often referred to as social engineering, but I feel that the term hacking people is more appropriate. Criminals exploit weaknesses in the human brain to make victims do something they should not.

The problem is that these "weaknesses" are actually an important part of human survival mechanisms in a brain that has been developed over several million years. Daniel Kahneman, in "Thinking, Fast and Slow", talks about System 1 and System 2 of the brain.

System 1 is always on and constantly looking for dangers or rewards. If a tiger jumps out of the bushes, System 1 tells you to act immediately and get away. There is no mechanism that makes the brain question whether the visual signal is real.

System 2 kicks in much later. That is when a lot of us fire up Excel as a decision making tool to buy a new pair of shoes. 

For most of us, a tiger jumping out of the bushes is no longer a real threat. But when "the bank" calls and tells you that someone hacked your account and that you must act immediately, System 1 kicks in. Or when "your child" calls and says they are in trouble.

With deepfakes, criminals can now be anyone they want.

There have already been fraud cases targeting companies, where the "CEO" and "board members" have instructed the CFO to initiate a money transfer to a new partner.

How do we address this problem? 

It is difficult to secure the human operating system the way we secure computers. Education is important. Stop, Think, Check is a good rule. Very few things are so urgent that you must act immediately. Let your System 2 kick in, and then decide.

But it is difficult to educate System 1.

What if we had a deepfake airbag? 

An airbag is invisible and silent, but activates when the car detects a collision. A deepfake airbag could work in the same way. It would run silent in the background, observing images, video and audio on your device, and alerting the user when deepfakes are detected.

There are of course challenges to overcome, one of them is privacy. How do we implement such an airbag without exposing everything we do? I do not have the answer to that. Maybe Multi Party Computation (MPC) could be part of the solution.

What would a practical deepfake airbag actually look like?