Deepfake technology: Why the sound you hear and the video you see may not be true

 Deepfake technology: Why the sound you hear and the video you see may not be true

With the rapid development of digital technology, the nature of information production, distribution, and consumption has changed radically. Although the widespread access and use of the Internet and social media has expanded freedom of expression, it has also created new risks of misinformation and misuse of technology.

Among these risks, deepfake technology has recently emerged as the most serious challenge.

With the rapid development of artificial intelligence (AI) technology, new types of cyber risks have begun to emerge around the world. Deepfake is also a means of serious digital crime that has been spreading rapidly in recent times. Since generative AI technology makes synthetic media faster, cheaper, and more reliable, deepfake content has emerged as a global risk from computer-generated imagery (CGI) research.

The claim that what humans see with this technology is true is gradually weakening in the digital age. With the rapid development of AI, a technology called deepfake is making videos, audio, and images so real that not only ordinary people but also experts are starting to be fooled. This type of technology produces fake but very real-looking digital content, as if saying things that were never said, doing things that were not done, or showing scenes that were not seen. Deepfake seems to have a profound impact on everything from personal life to national security.

In January 2024, phone calls made in the US state of New Hampshire to mislead voters by imitating President Joe Biden's voice were confirmed to be deepfake. For which the person responsible was fined $6 million. Similarly, the fact that millions of deepfake phone calls and content were used in the Lok Sabha elections in India has been made public. These incidents make it clear that the political misuse of deepfake technology is not only a possibility but also a real and present-day threat.

Deepfake videos linked to political leadership and public figures have started appearing in Nepal as well. In this context, a deep study of the impact that deepfake technology can have on Nepali society, democracy, and information systems is necessary. Early studies on deepfakes focused on the technical aspects. Chesney and Citron (2019) described deepfakes as a key weapon in the ‘new information war’. According to them, deepfakes undermine the credibility of audiovisual evidence.

Europol (2023) has identified deepfakes as a serious challenge to criminal investigation and law enforcement. The study concluded that deepfakes increase the amount of false evidence and complicate the investigation process. Similarly, the Artificial Intelligence Act implemented by the European Union in 2024 regulates deepfakes by placing them in a risk-based classification. This has accelerated the global debate towards policy solutions.

However, studies, research and public debate on deepfakes in Nepal are still limited. With the rapid development of technology, deepfakes are increasing the risk of political abuse, cybercrime, character assassination, and misinformation in various countries around the world, but there does not seem to be an in-depth study of their potential impacts and challenges in Nepal.

Although the recently released National Artificial Intelligence (AI) Policy, 2082, initially attempted to address emerging technology-related risks including deepfakes, a clear legal system, regulatory mechanisms, and implementation strategy still appear insufficient. According to experts, timely policy clarity, public awareness, and the development of a technology-friendly legal framework are necessary to prevent the misuse of deepfakes.

How is deepfakes created?

To create deepfakes, a large amount of photo, video, and voice data of the person concerned is collected. Based on that material, the AI ​​model learns facial expressions, eye movements, lip movements, speaking style, and tone of voice. Then, by imitating all those features, a new video or audio is created, which looks and sounds real.

There are many ways to create deepfakes. In particular, machine learning, deep learning, and artificial intelligence technologies are used in unique ways. These include facial recognition algorithms, autoencoders, and artificial neural networks such as generative adversarial networks. The use of such technologies has been found to make deepfakes more robust.

Key technologies and architecture

Generative adversarial networks are the basic technology for creating modern, high-quality deepfakes. In this technology, two AI models, or two neural networks, work in competition with each other. One creates fake content, while the other tries to distinguish it from real or fake. Through this process, the fake content gradually becomes stronger and more realistic.

In a generative adversarial network, two neural networks, namely a generator and a discriminator, compete with each other. In which the generator creates synthetic images or video frames, the discriminator analyzes the output to distinguish between real and fake data. That is, if the generator produces fake images or videos, the discriminator tries to distinguish whether the content is real or fake. Discriminator The generator gradually creates better and more realistic content to fool the network. As a result of this continuous competition, the deepfake content produced starts to look more and more like the real thing, which is difficult for ordinary people to distinguish.

Autoencoder

Another basic structure of deepfake technology is the autoencoder. It is used to create and swap facial outlines. This method uses an autoencoder. The autoencoder is divided into two parts; Encoder and Decoder. The encoder analyzes a person's face image and converts its main features, such as facial expressions, head movements, and lighting, into concise data. In this process, the real identity of the face is removed. The decoder uses those features to reconstruct the face. In a deepfake system, separate decoders are usually used for two different people. This allows the facial expressions and movements of one person to be transferred to the face of another person. This process is called face swapping. Technically, the encoder network analyzes the source material and extracts the necessary data from it, and this data is sent to the decoder network. The decoder combines it with the target video to produce a new output. Although the final result looks exactly like the real thing, it is actually fake content.

Image: How does a deepfake work?

Recently, technologies that use only a video or photo to create a deepfake have also been developed. In addition, it has become possible for ordinary users to easily create deepfake videos or audio through various websites and mobile apps, which has further increased the risk of its misuse.

The Opportunity Created by Deepfakes

Although this technology is often cited in the context of spreading confusion, fraud and misinformation, deepfakes also have significant benefits when used ethically, transparently and in compliance with the law. Its use, with proper regulation and awareness, can help bring about positive changes in education, health, cybersecurity, communications and government services.

The use of deepfakes technology can provide interactive learning to students by presenting great figures from history in digital form. Since complex topics can be taught through virtual teachers, simulation-based exercises in technical and vocational training help to provide real-world experience to make learning more effective.

In the film and television industry, deepfakes help to safely create risky scenes, convert languages ​​and improve lip sync. It helps to recreate old scenes or dub in different languages. Similarly, in healthcare, deepfakes can be helpful in areas such as reconstructing voices for people who cannot speak due to accidents or diseases, and improving facial expressions.

Since Nepal is a multi-ethnic, multilingual, multi-religious, multi-cultural and geographically diverse country, deepfake technology can play a major role in making information access equal for all. Dubbing video content in the local language and presenting it clearly makes it easier and more effective to reach rural and disadvantaged communities. This technology can also be helpful for people with disabilities.

The use of deepfake using virtual characters to effectively disseminate government services, public awareness, public information, disaster-related information and public awareness messages helps to make the message simple, clear and attractive. One of the positive uses of deepfake technology in the field of cybersecurity and research is deepfake detection. The development of such systems helps to control cybercrime and increase media literacy by using synthetic data for research and training.

The Challenge of Deepfake

Deepfake technology is a modern technology based on AI. It is capable of creating fake videos, audios and images that look very real. It is being used in the entertainment, film and creative sectors.

But this technology also poses serious challenges to society, security and public trust. In particular, the risk of cybercrime and misuse of social media is increasing rapidly.

It is difficult for the general public to distinguish fake video or audio content created using deepfake technology from real content. Since such content has the potential to spread rapidly through social media, the risk of spreading misinformation, rumors and confusion is increasing day by day. This can have a far-reaching negative impact on social harmony.

Fake content created by imitating a person's face or voice can cause serious problems such as character assassination, defamation and blackmail. Victims of such incidents have also been seen to suffer mental stress, social exclusion and serious damage to self-esteem.

Similarly, incidents of financial fraud through fake phone calls, messages, or videos using deepfake audio or video are also increasing. The trend of asking for money by imitating the voices of relatives, bank officials/employees, or responsible persons of organizations has emerged as a new cyber challenge. It has made it easier for cybercrimes such as phishing, social engineering attacks, identity theft, and misinformation to spread.

In many countries, clear legal systems related to deepfakes are still in the development stage. The issue of who should be held responsible for its misuse and what punishment should be given has become complicated. This has challenged legal and moral values. In Nepal, the existing Electronic Transactions Act, 2064, does not cover all dimensions and misuse of deepfakes, so a special law on deepfakes or an amendment to the existing law is necessary. In addition, the lack of technical capacity development for the investigation and prosecution process, production of skilled human resources, and technology and equipment to identify deepfakes have made it difficult to investigate, identify, control, and take action.

Due to the lack of digital literacy among the general public, there is a tendency to easily believe deepfakes and share them on social media without verifying the facts. Such behavior is contributing to the rapid spread of misinformation and confusion.

Deepfake Control and Solutions

As the misuse of deepfake technology increases, various technical solutions are considered necessary to identify and control it. For this, legal provisions alone are not sufficient without technical measures. Special types of AI-based software can be used to identify fake videos or audio.

Such tools provide an indication of whether the content is fake or not by analyzing unnatural facial movements, eye blinks, lip and voice coordination, pixel asymmetry, and errors in sound, etc. Similarly, the technology of placing digital watermarks on video, images, or audio content also helps to verify its source and authenticity to some extent.

Recently, technologies such as content credentials are being developed. Which help to securely show information about when, who, and how such content was created. Platform-based automatic monitoring can also be done. For example, algorithms that automatically identify deepfake content can be used on social media platforms such as Facebook, YouTube, and TikTok.

Such systems help identify suspicious content and immediately remove or warn. To prevent fraud caused by deepfake audio, it is considered beneficial to adopt multi-factor authentication rather than relying solely on voice-based authentication. This helps prevent financial crimes committed using fake voices.

Developing a national level cyber forensic capacity, state-of-the-art cyber forensic labs, skilled technical personnel, and regular training are required for deepfake investigation and investigation. This helps in making evidence collection and legal action effective in real incidents. In addition, incorporating security and ethical standards from the beginning into AI systems that develop deepfake technology is also considered an important technical measure. This helps in minimizing the possibility of misuse.

As the misuse of deepfake technology increases, a clear legal system and effective policy framework are needed to control it. With the development of technology, the control of cybercrime is becoming challenging due to the lack of updating of laws and policies. In addition, a clear definition in the law is needed on what deepfake is and in what cases its use is illegal.

Deepfake content created with the intention of defamation, fraud, blackmail, spreading misinformation, or interfering in the electoral process should be defined as a separate crime. The current cyber laws, the Electronic Transactions Act, and the Privacy Act cannot cover all types of deepfake abuse. Therefore, it seems necessary to make the existing laws technology-friendly and include provisions related to deepfake.

The role of social media platforms is important in preventing the spread of deepfake content. Therefore, it seems necessary to legally assign such platforms the responsibility of quickly removing suspicious content, collaborating with government agencies, and informing users. Coordination between technology, law, and policy is necessary to control this kind of development spread in the digital space.

Designating responsible agencies by clearly coordinating between the National AI Policy, National Cyber ​​Security Policy, and Digital Media Policy (which does not exist in Nepal) is a key aspect of the policy solution. Finally, it is equally necessary to increase public awareness and capacity. Regular training and public awareness programs are necessary for government agencies, the justice system, the police, and media personnel to make legal and policy solutions effective. It plays an effective role in supporting the proper application and implementation of the law.

How to recognize deepfakes and how to avoid their dangers?

Do not immediately trust suspicious videos or audio.

Confirm the source if you receive emotional or emergency messages.

Do not give money or confidential information based on voice alone.

Share personal photos, videos and voices in limited quantities on social media

Tighten privacy settings.

Use two-factor authentication, multi-factor authentication (mobile number, OTP, fingerprint, etc.) to protect your digital accounts on social media.

Use different strong passwords (creating passwords by combining letters, numbers and special symbols) for different social media accounts (e.g. Facebook, Twitter, TikTok, Instagram) and change them regularly. Change your password regularly.

Use a digital watermark when sharing photos or videos online.

Update your software regularly.

If you are a victim of a deepfake, consult with experts in cybersecurity and data privacy law.

File a complaint with the relevant agency if necessary.