Generative AI and Four Types of Deception

Since autumn of 2022, generative AI has taken the world by storm. With millions of regular users, billions of requests and corresponding results, tools employing generative AI are ceaselessly used and abused for a wide variety of purposes. There are at least two reasons for this massive uptake: ease of use and free availability on the one hand, and breadth of applicability on the other.

The core of generative AI is the capacity to produce new verbal or visual products of increasingly high quality based on patterns discovered in massive amounts of data sources, most of them scraped from the web. As such, these tools use probability calculations to generate plausible texts, pictures, video, or audio files which may sound elegant or look realistic, but don’t have any grounding in truth. Indeed, given the underlying probabilistic generation, generative AI “getting it right” (i.e., providing a correct response to a query) must be considered an instance of epistemic luck.

Ethical Issues of Generative AI

Philosophical and ethical analyses of AI in general and generative AI have exploded in the last years—sometimes expanding, sometimes merely replicating much older debates in the philosophy of AI and computer ethics (fields considered rather fringe within academic philosophy until recently). Such ethical issues range from privacy, data protection and surveillance, to questions on bias, discrimination and fairness; from demands for transparency, explainability, accountability, and responsibility to a critical view on the lacking sustainability and poor working conditions in AI development. Other foundational issues concern questions of agency, but also the increasing reliance on statistical reasoning in many societal domains and its potentially transformative effect on autonomy and freedom, solidarity, and justice. All these concerns are pressing and, albeit to varying degrees, characteristic of most if not all data-based AI systems.

There are, however, additional concerns which are unique to generative AI, or are at least drastically aggravated through it. These are related to unique interactional features of generative AI enabling different forms of deception. Deception can cause ethical and epistemological harm by compromising our ability to place trust well, as it impairs our ability to assess trustworthiness correctly. Apart from few exceptions, it is normatively appropriate to trust someone or something if and only if they are trustworthy. In other words: we are right in trusting those who are trustworthy and distrusting those who aren’t. Trusting those who aren’t trustworthy, or distrusting those who would have been trustworthy, on the other hand, can lead to epistemic and ethical harm.

In the following, I want to propose the notion of quadruple deception to capture a set of related, yet distinctive forms of deceptions caused by generative AI and assess its effects on trust: (1) deception regarding the ontological status of one’s interactional counterpart, (2) deception regarding the capacities of AI, (3) deception through content created with generative AI, as well as (4) deception resulting from integration of Generative AI into other software.

In contrast to philosophical accounts of deception which require intention, I adopt the broader understanding of deception, which covers both intended and unintended deception. There are at least three reasons for doing so: first, while intent is relevant for the ethical assessment of an action (e.g., its blameworthiness), it is not necessary for detrimental ethical, epistemic, or political consequences of deception to occur. Second, whether or not deception was intended is often not discernible. Third, dropping the requirement for intent lowers the barrier to attribute responsibility to AI developers for the potential harm they may cause. I will defend this choice in the conclusions.

Quadruple Deception

The first form of deception regarding generative AI concerns the danger that users may be misled into believing that they interact with a human being while indeed interacting with a chatbot. Deception 1 thus refers to the misconception regarding the ontological status of one’s counterpart. An example would be a user who assumes they are talking to a customer agent while actually being confronted with a chatbot, or more worryingly, clients assuming that they interact with a psychotherapist, while actually interacting with software. Historically, it was this type of deception which constituted the benchmark for the realization of artificial intelligence according to Alan Turing’s famous test. This type of deception is therefore at the core of the development and by no means a novel worry. However, given the pervasiveness and increased performance of generative AI, it has become much more pressing.

The example of the chatbot-psychotherapist leads us to the second type of deception and once more back into the history of AI: Deception 2 refers to deception about the capacities of AI. Since the launch of ChatGPT and other forms of generative AI, some have claimed that such tools are more than probabilistic content creators, but instead supposedly express intelligence, understanding, or even consciousness. The tendency of humans to anthropomorphize technologies (i.e., to attribute human characteristics to them) accompanies the development of AI from its onset as Joseph Weizenbaum’s ELIZA illustrated already in the 1960s. Experiencing users attributing intelligence and empathy to ELIZA—even if they knew it was just a simple software program—turned its creator Joseph Weizenbaum into one of the earliest and most fervent critics of AI technologies. Nowadays, this type of deception is on the rise, and it is at times hard (if not impossible) to find out whether those claiming that current AI systems are capable of empathy or even conscious truly believe their claims or are intentionally misleading their audience. Irrespective of this, two issues appear obvious, however. First, current AI systems do not possess consciousness or empathy and merely simulate those. Second, the true existence of such capacities is not necessary for humans to infer such capacities based upon the behavior of such systems, thereby being deceived in their understanding of AI systems. And this mere perception of empathy and intelligence of AI by the users has already led to atrocious consequences, with some users having committed suicide as a consequence of believing such systems to be friends of therapists.

The third type of deception related to generative AI, Deception 3, concerns the deception caused by misleading content produced with generative AI. Examples of this type of mis- or disinformation are fake scientific publications with fictitious references, but also deepfakes in the form pictures, videos, or audio files. The potential impact of such content ranges from amusing cases in entertainment to severely damaging examples aiming to deceive and manipulate public opinion. Clearly, manipulation and propaganda have a long history and the role of technologies in aggravating these dangers has been extensively addressed and assessed in the public, but also in philosophical analyses. However, the combination of generative AI and social media has massively increased the threat of this type of deception due to the ease and speed by which misleading content of sufficiently high quality can be produced and disseminated. Indeed, the Global Risk Report 2024 of the World Economic Forum lists AI-generated misinformation and disinformation as the most severe anticipated global risk for the next two years.

The fourth type of deception, Deception 4, concerns deception regarding the function of Generative AI. From its launch, ChatGPT and its competitors were heralded as the future of online search. However, such a comparison is deeply misleading. User may indeed use ChatGPT to search for information online and in many cases the results may look similar to information found online, e.g. on Wikipedia. However, the functioning of a search engine differs from that of a LLM in epistemologically highly significant ways: most importantly, LLMs are not retrieving existing texts, but generating new texts. While this novel stochastically produced text is based upon the materials found online (because the text patterns are extracted from the training material found online), the difference between information retrieval and probabilistic pattern production matters epistemically, and it adds an extra layer of potential misinformation and disinformation regarding the epistemic status of the material.

Quadruple Deception & Trust

So how do these four types of deception affect trust? The first type of deception, Deception 1, already affects trust in various ways. First, falsely believing to interact with a human while indeed dealing with a chatbot can lead to unjustified trust in the capacities of this counterpart, such as its supposed understanding or empathy, thereby relating Deception 1 to Deception 2, as well as into the information provided by the chatbot, thereby relating it to Deception 3.

At the intersection of Deception 1 and Deception 3, epistemic trust in particular may be challenged if one happens to believe false information provided by manipulative fake profiles on social media sites. This would be a case of unjustified epistemic trust in an untrustworthy source, leading to epistemic and possibly also practical or ethical harm depending on the kind of deception occurring. Yet not only the deception itself, but also the revelation of the deception regarding the ontological status of one’s counterpart affects trust in different ways. Under ideal circumstances, a reduction of trust may lead to a more appropriate level of trust into the chatbot where the expectations of the human user match the real capacities of the chatbot. It may, however, also cause a more widespread and detrimental loss of trust into the institution employing the chatbot—or even into societal communication at large—if one feels that reliably distinguishing between humans and machines becomes difficult or impossible.

Deception 2 refers to the various ways in which AI is being characterized or perceived to have capacities, such as empathy or even consciousness. The degree to which people truly believe claims about AI being sentient or are merely stating them for strategic purposes is not always clear. However, a lesson learned from Searle’s Chinese Room argument is that simulating understanding is not the same as understanding—and neither is simulating empathy through verbal expression the same as being empathetic. It is this discrepancy between how things are and how they appear to us, that opens the possibility for deception.

What would be the implications of Deception 2 for trust? First, there is a danger of misplaced trust in anthropomorphic systems. If one assumes that a chatbot truly understands the meaning of one’s own communicative acts instead of merely processing the words and producing a likely response, this elicits certain expectations—not only regarding the communicative behavior of such systems, but also regarding its intentional stance towards the user. Take the example of chatbots used in therapeutic contexts. When interacting with a human psychotherapist, the client may not only have certain contextual expectations regarding the conversational behavior of the psychotherapist, but also regarding her stance towards him and the relationship that binds them together. And while the communicative behavior of a psychotherapist may be simulated to a reasonable degree by a chatbot, it is also the intentional stance of the psychotherapist and the relation between client and psychotherapist—which is crucially relevant for most therapeutic approaches. By stating that this relational aspect may be relevant for most, but not all psychotherapeutic approaches, I grant that defenders of strictly behaviorist forms of psychotherapy may plausibly deny the necessity of any relationality between client and psychotherapist, and thus would also accept a chatbot psychotherapist as a full substitute of a human psychotherapist.

Deception 3 concerns deceptive pictures, videos, or text produced with generative AI. While ChatGPT and other LLMs can be used to create deceptive texts, using generative AI for creating deceptive visual and audio content may cause even more severe societal problems resulting from misplaced trust. Deceptive content can first and foremost lead to unjustified trust in this content and those providing it. Thus, from an epistemological perspective, deepfakes may lead to false beliefs, which may have severe societal consequences. And indeed, this worry does not appear farfetched, since various incidents of manipulative use of deepfakes have recently been reported worldwide. Moreover, the difficulty to distinguish between true and false content can lead to an overall reduction of trust within societies. If fake content is presented as scientific evidence, if faked statements by politicians are circulated, overall loss of trust in politics, the media or science can be the result irrespective of their actual trustworthiness. Accordingly, AI-generated fake content poses significant societal challenges by soliciting trust in untrustworthy content and sources and by making the distinction between false and true information more difficult to impossible. As a consequence, trust overall gets eroded, including trust in trustworthy sources. As such, the negative implications of deep fakes through generative AI are a major threat for contemporary democracies by severely and negatively affecting epistemic, interpersonal, and societal trust.

Finally, both the presentation of tools such as ChatGPT as the future of search as well as the integration of generative AI into various tools of information retrieval also poses significant challenges for epistemic and ethical trust. Information retrieval (e.g., in the form of online search, but also search within one’s email program) works under the premise that existing content is being searched for. Generative AI, however, as the term “generative” indicates, is not about finding, but creating new content based upon patterns in the data upon which it was trained. As such, the integration of generative AI into search and other services has significant consequences especially for epistemic trust. After all, how can I rely upon and trust these tools—and my memory—if I cannot be certain that the email I am “finding” existed before and was hopefully also written by a human being—and not created as a consequence of my search? The consequences this integration of generative AI and the resulting confusion of functionalities may have on epistemic and cognitive processes are far from sufficiently addressed or even assessed.

Conclusions

I have argued that generative AI deceptive capacities exploit human cognitive and emotional vulnerabilities and can lead to various forms of ethical, epistemic, and societal harm. But are all four types of deception really deception? Above I have provided two arguments for dropping the requirement for intent for deception. First, proving intent is often difficult to impossible. Second, deception can have the same ethical, epistemic, or societal consequences irrespective of any intention to deceive. Still, one may argue that this nonetheless muddies conceptual waters between being deceived and making a mistake. While I acknowledge this, I argue that there are strong ethical reasons for employing a wider notion of deception which does not require intent.

To illustrate take the case of discrimination as an analogy. The philosophical and legal discourse on discrimination centers around the question whether discrimination fundamentally depends upon the intention to discriminate (disparate treatment) or whether differential impact of a given practice on different groups of individuals is sufficient to classify a practice as discriminatory (disparate impact). While the implications of a given practice for individuals or groups facing discrimination are the same under both doctrines, the hurdles for indicting someone of discrimination are substantially higher under the doctrine of disparate treatment, as it is notoriously difficult to assess intention to discriminate let alone to prove it in court.

Now compare this with our four cases of deception: shall we conceive the developers of AI technologies morally responsible and thus blameworthy for the harm caused if and only if they intended to deceive or can they be blamed for the harm caused, even without necessarily having intended it? To my mind, adopting a stance on deception which drops intent and uses the analogy of disparate impact has the advantage of being more sensitive to harm experienced by those who are exposed to deception. It lowers the barrier to attributing responsibility to AI developers for the potential harm they may cause just as much as it lowers the barrier for those facing discrimination in the work environment to sue their employers. While this is a conceptual choice, it is by no means unfounded; instead, it is based upon the ethical premise that power demands responsibility, and that those who have more power accordingly have a higher responsibility for the consequences of their actions and non-actions (i.e., their negligence). It is high time we hold tech companies to account for the epistemic, ethical and societal harm they are causing, irrespective of whether it results from ill intent or negligence.

Acknowledgements

This blog post is based upon the paper “Generative AI and the Danger of Quadruple Deception,” which includes more details and also references to literature. The paper has been published in the journal Social Epistemology as part of a forthcoming Special Issue The Mind-Technology Problem in the Time of Generative AI (Clowes, Gärtner, & Theiner, eds.).