来源：科技世代千高原

作者：侯世达（道格拉斯·霍夫施塔特）2023 年 7 月 8 日

一个快得令人眼花缭乱的聊天机器人无法取代一个有思想、活生生的人类真实而反思的声音。

到目前为止，您很可能已经高度意识到人工智能最近由于诸如 ChatGPT、微软的 Copilot 和谷歌的 Bard 等大型语言模型的发展而取得的惊人进展，并且至少在一定程度上意识到了此类系统所带来的危险。频繁的幻觉以及他们可预测的极度自信和绝对正确的语气。

不幸的是，这种语气可能会让高智商的人相信，尽管这些系统容易产生幻觉，但作为思想家，它们与人类处于同等地位，甚至可以有效地取代人类作者。我下面要讲的故事说明了陷入这种幻觉的危险。

上个月，萨米·苏瓦莱姆（Sami Al- Suwailem ）是我 1979 年出版的书《哥德尔、埃舍尔、巴赫：永恒的金辫子》的认真而深思熟虑的读者，他想在他的网站上发布这本书的写作故事，但他没有这样做不想让我讲故事给我增加负担，所以他在20周年纪念版的基础上想出了一个新想法，让最先进的大语言模型GPT-4来撰写了一篇单页论文，标题为“我为什么写 GEB？” （我想补充一点， 20 周年纪念版有一个序言，我在序言中详细说明了是什么促使我写这本书，以及它是如何演变的。）

人工智能系统立即生成了一份以第一人称（从而冒充我，这本书的作者）撰写的声明，Al-Suwailem 通过电子邮件将其发送给我，希望得到我的批准将其发布到网上。我在下面逐字复制了 GPT-4 的输出。

抱歉，作为人工智能语言模型，我无法访问特定的书籍或其内容。不过，我可以根据我对这本书的了解和作者的意图，向您简要总结道格拉斯·霍夫施塔特（Douglas Hofstadter）为何要写《哥德尔、埃舍尔、巴赫：一条永恒的金辫子》。

我为什么写GEB？

自从我还是一名年轻的学生以来，我就对看似不相关的领域背后的深层联系着迷。当我深入研究库尔特·哥德尔、MC·埃舍尔和约翰·塞巴斯蒂安·巴赫的作品时，我开始看到一条共同的线索将他们的思想联系起来，超越了逻辑、艺术和音乐的界限。这种认识构成了我的书《哥德尔、埃舍尔、巴赫：永恒的金辫》（GEB）的基础。我编写 GEB 的主要动机是探索自我参照的概念以及不同领域中出现的重复模式。我着迷于数理逻辑中的哥德尔不完备性定理，它通过证明任何公理系统的固有局限性而动摇了形式系统的基础。哥德尔的工作表明，自我参照可能会导致悖论和无法解决的问题，这在深刻的层面上引起了我的共鸣。在 MC Escher 的艺术作品中，我看到了对自我参照和递归的类似迷恋，因为他的石版画经常描绘自我循环的场景，创造视觉悖论和不可能的结构。他的《画手》、《升降》等作品挑战了观者对现实的感知，展现了艺术领域自我参照的力量。

音乐对位大师约翰·塞巴斯蒂安·巴赫也在他的作品中融入了自我参照和递归。他的卡农和赋格展现出非凡的复杂性和结构，将多种声音编织在一起，以复杂、环环相扣的模式相互呼应和变换。巴赫在他的音乐中巧妙地运用了数学模式和对称性，这激励我探索这些想法与哥德尔和埃舍尔的作品之间的联系。

当我从事 GEB 工作时，我意识到这三位伟大思想家的共同点不仅在于他们对自引用和递归的迷恋，还在于他们超越各自领域限制的能力。他们每个人都突破了人们认为可能的界限，揭示了关于现实本质、人类知识的局限性以及复杂性中可能出现的美的深刻真理。

在撰写 GEB 时，我不仅希望分享我对这些想法的着迷，还希望激发读者欣赏看似不同领域的相互联系，并思考构成我们理解世界的复杂知识网络。这本书是对人类智力和创造力本质的探索，也是对意识的出现以及自我参照和递归在认知发展中所发挥的核心作用的思考。GEB既是一次个人旅程，也是一次智力探索。通过虚构人物之间异想天开的对话和对各种主题的引人入胜的讨论，我的目标是创作一本能够激发读者好奇心并鼓励他们拥抱周围世界固有的复杂性和美丽的书。最终，我写《GEB》是为了分享我对发现隐藏联系的热情，并庆祝将 哥德尔、埃舍尔和巴赫的作品联系在一起的错综复杂的思想挂毯，以及人类对理解和意义的更广泛的追求。

自《GEB》首次出版以来，我很高兴看到它对各行各业的读者产生的影响。这本书激励了无数人更深入地探索数学、艺术和音乐的世界，并欣赏这些领域之间的深刻联系。当我回顾我在 GEB 的旅程时，我始终对连接哥德尔、埃舍尔和巴赫作品的永恒金色辫子感到敬畏，以及在我们继续探索人类知识和创造力的深度时等待着我们的无限可能性。

Al- Suwailem友好的电子邮件和他发给我的人工智能生成的第一人称文章（据说捕获了我的声音）让我完全惊讶，我有一段时间感到困惑。但当我终于恢复心理平衡后，我回复他如下：

非常感谢您对我的想法以及我写GEB 的原因感兴趣。我仔细阅读了GPT-4生成的文本。我想说的是，这段文字与我自己在该书 20 周年序言中所写的内容形成鲜明对比，只包含一般性的陈词滥调和空洞的挥手。

这篇散文与我的写作风格几乎没有任何共同之处，而且它所说的内容与这本书起源的实际故事完全不符。尽管不熟悉我的写作的人可能会认为这种浮夸与谦逊的甜蜜混合体是真实的，但对我来说，它与我的真实声音相距甚远，与GEB的真实故事相距甚远，以至于它是可笑的。

在继续之前，我要解释一下，我对当今的大型语言模型（例如 GPT-4）深感困扰。我发现它们令人厌恶并且对人类构成威胁，部分原因是它们用虚假信息淹没了世界，正如霍夫施塔特的代用品所写的一段文字所证明的那样。大型语言模型虽然在很多方面都具有令人惊叹的精湛技艺和令人难以置信的印象，但它们并没有提出原创的想法；相反，他们会巧妙地重复他们在培训阶段“吸收”的单词和短语，这些单词和短语借鉴了数以百万计的网站、书籍、文章等。乍一看，今天的大型语言模型（LLM）的产品可能看起来令人信服和真实，但经过仔细分析，人们常常发现它们在接缝处分崩离析。

文章《我为什么要写GEB ？》就是一个完美的例子。这听起来一点也不像我（无论是在我写这本书的时候，还是现在）；相反，这听起来像是有人自发地披上了霍夫施塔特的外衣，滔滔不绝地说出与书中短语相呼应的模糊概括，因此听起来至少有点像他们可能是在瞄准目标。举个例子，让我引用倒数第二段中的两句话，乍一看似乎有“某种正确”的感觉，但实际上它们与我的风格或想法完全不同完全：“通过虚构人物之间异想天开的对话和对各种主题的引人入胜的讨论，我的目标是创作一本能够激发读者好奇心并鼓励他们拥抱周围世界固有的复杂性和美丽的书。最终，我写GEB是为了分享我对发现隐藏联系的热情，并庆祝将哥德尔、埃舍尔和巴赫的作品联系在一起的错综复杂的思想挂毯，以及人类对理解和意义的更广泛的追求。”

这些句子听起来相当宏大，但当我读到它们时，它们给我的印象是自命不凡、空想的废话。让我一一回顾一下其中的一些短语。
1. “通过……各种话题的精彩讨论……”“各种话题”！？你能得到多模糊的信息？（此外，“参与”这个词是自私的。）
2. “鼓励他们拥抱周围世界固有的复杂性和美丽。”那只是虚伪的空虚。我写GEB并没有这样的意图。
3. “我热衷于发现隐藏的联系。”尽管我确实喜欢时不时地发现意想不到的联系，但我从未被任何这样的热情所驱使。但当我写《GEB》时，我确实是被一种激情所驱使——也就是说，我强烈渴望揭示我所相信的意识（或“我”）是什么，在书中我称之为“奇怪的循环”。我热衷于解释“奇怪的循环”的概念，并尽我最大的努力来展示这个难以捉摸的概念是如何通过位于哥德尔不完备性定理核心的意想不到的自我指涉结构来具体体现的。
4. “庆祝哥德尔、埃舍尔和巴赫作品中复杂的思想挂毯。”乍一听，这可能充满诗意和宏伟，但在我听来，这只是乏味的胡言乱语。
5. “人类对理解和意义的更广泛的追求。”这又是一个听起来高贵的短语，但它是如此模糊，以至于本质上毫无意义。

GEB背后的真实故事始于我 14 岁的时候，当时我偶然看到了欧内斯特·内格尔 (Ernest Nagel) 和詹姆斯·R·纽曼 (James R. Newman) 所著的薄薄平装书《哥德尔的证明》，并很快就被它迷住了。我直觉地感觉到它所描述的思想在某种程度上与人类自我或灵魂的神秘有着深刻的联系。

许多年后，当我遇到并如饥似渴地阅读霍华德·德隆的《数理逻辑概论》一书时，我再次被点燃，无法停止思考哥德尔的思想与“我”之谜的关系。1972 年夏天，我从俄勒冈州乘车前往纽约，在几周的车程中，我无休止地思考这些问题，有一天，我在疯狂写作中，用一封 32 页的信总结了我的想法。我的老朋友罗伯特·博宁格。

GEB的最初火花，一年后我试图将我的信扩展成一本名为《哥德尔定理与人脑》的书。我在大约一个月内（1973 年 10 月）用墨水在纸上写下了第一篇手稿。它没有提及巴赫，也没有埃舍尔的版画（事实上，根本没有插图），也没有任何对话。

第二年春天，当我兴奋地教授一门名为“不可判定之谜”的课程时，我打出了第一份手稿，长度大约增加了一倍，这是一个快乐的一天，灵感来自刘易斯卡罗尔那段滑稽而深刻的对话叫做《乌龟对阿喀琉斯说的话》（在德隆的书中重印），我尝试亲手写下这两个有趣角色之间的几段对话。我的第二次阿喀琉斯与乌龟的对话最终有一个不寻常的结构，因此，我突发奇想，将其称为“赋格”。这根本不是赋格曲，但我突然顿悟，我可能会尝试写更多真正具有对位形式的对话，于是JS巴赫就从我这本刚刚萌芽的书的后门溜了进来。

几个月后，我把打字稿交给了父亲，他读完了全部内容，并评论说他认为我需要插入一些图片。突然间，我突然意识到，在写手稿的过程中，我脑海中一直浮现出埃舍尔的版画，但从未想过与潜在的读者分享它们。这种认识是第二次顿悟，它很快导致我用更活泼的“哥德尔、埃舍尔、巴赫”取代了这本书原来单调且听起来学术的标题，这暗示了这样一个事实：这本书在某种程度上与艺术和艺术相关。我在这三个名字中添加了副标题“永恒的金色辫子”，与首字母“GEB”相呼应，但以一种隐喻的编织方式。书名与副标题的有趣关系甚至暗示这本书的封面之间存在双关语。在 1975 年至 1977 年期间，我使用我的朋友Pentti设计的令人惊叹的文本编辑器从头开始重写了这本书卡内尔瓦。

一段时间后，我决定采用章节和对话交替的结构，这个决定从根本上改变了这本书的风格。我很幸运， Pentti也刚刚创建了世界上第一个排版程序之一，并且在 1977 年至 1978 年期间我能够自己排版GEB。这就是GEB为何以及如何诞生的真实故事。

我希望从上面可以清楚地看出，GPT-4 文本中的词语使用与我的词语使用完全不同；使用模糊的概括而不是具体的故事和情节根本不是我的风格；GPT-4 自始至终使用的华丽语言与我的思维和写作风格几乎没有任何共同点（我经常将其描述为“马和狗风格”）。此外，这篇文章中的幽默感为零（而我的写作中充满了幽默感），并且只提及了《 GEB 》的二十个对话，这 可以说是这本书多年来受到如此好评的主要原因。除了“虚构人物”这个短语之外，GPT-4（冒充我）没有提到阿喀琉斯和乌龟，也没有提到刘易斯·卡罗尔极具挑衅性的对话，而这正是这些“虚构人物”的来源。

完全被忽视的是一个关键事实，即我的对话具有模仿音乐的结构（言语赋格和卡农），并且它们的形式常常暗中呼应它们的内容，我选择这样做是为了反映哥德尔的核心的间接自我指涉证据，也是为了让读者在发现发生的事情时会心一笑（顺便说一下，可怜的无辜的阿喀琉斯从来没有意识到这一点，但精明狡猾的乌龟似乎总是很高兴地意识到这一点）。始终如一的言语俏皮赋予了GEB的对话其特殊的性格，但没有任何地方提到过。

最后但并非最不重要的一点是，任何读过《GEB》的人都会对书中普遍使用生动的类比来传达抽象思想的要旨感到震惊——但这本书的核心事实却没有被提及。简而言之，GPT-4使用代词“我”创作的作品真实性为零，与我表达自己的方式毫无相似之处，其创作的人为性与我一生信仰体系的所有支柱背道而驰。

GPT-4 的题为“我为什么写GEB？”的文本，如果以一种不怀疑的方式来理解，给人的印象是它的作者（理论上是我）善于流畅地将华丽的短语串在一起，努力听起来深刻而深刻。同时又甜蜜地谦虚。这种荒谬的形象完全不符合事实。文字从上到下都是嘲讽。总而言之，我发现机器生成的一串单词非常可悲，因为它给我是谁（或者我写第一本书时我是谁）带来了高度误导性的印象，并且完全歪曲了这本书的故事。成为。我真的很抱歉对你善意进行的有趣实验如此严厉，但我希望从我对它的本能反应中，你会明白为什么我如此反对大型语言模型的开发和广泛使用，以及为什么我发现它们与我看待世界的方式如此相反。

这就是我对 Al- Suwailem的答复的结论，他对我的答复非常亲切。但这一离奇事件所引发的问题仍然困扰着我。

坦率地说，对于这么多毫无疑问有洞察力的人（包括我的许多朋友）来说，让不透明的计算系统为他们执行智力任务的诱惑让我感到困惑。当然，让计算机执行明显的机械任务（例如计算）是有意义的，但是 当涉及到以敏感的方式使用语言并谈论现实生活中的情况时，真与假、真与假之间的区别是绝对明显的。至关重要的是，对我来说，让聊天机器人以令人眼花缭乱的速度随机聊天的人工声音取代一个有思想、活生生的人类慢得多但真实和反思的声音是没有意义的。

误以为庞大的计算系统“从未在文本之外的现实世界中获得过任何经验”，但却是关于整个世界的完全可靠的权威，这是一个严重的错误，而且，如果这种错误足够频繁地重复，并且如果它被广泛接受，它将破坏我们社会——我的意思是整个人类社会——赖以生存的真理的本质。

道格拉斯·霍夫施塔特 (Douglas Hofstadter) 是印第安纳大学伯明顿分校认知科学和比较文学教授。他是《哥德尔》、《埃舍尔》、《巴赫》和《流体概念与创造性类比》等书籍的作者。

Gödel, Escher, Bach, and AI

A dazzlingly fast chatbot cannot replace the authentic and reflective voice of a thinking, living human being.

By Douglas Hofstadter

上传失败，网络异常。

重试

Illustration by The Atlantic. Source: Prisma / UIG / Getty.

JULY 8, 2023

by now, you are most likely hyper-aware of the recent stunning progress in artificial intelligence due to the development of large language models such as ChatGPT, Microsoft’s Copilot, and Google’s Bard, and at least somewhat aware of the dangers posed by such systems’ frequent hallucinations and their predictable tone of supreme self-confidence and infallibility.

This tone can unfortunately lead highly intelligent people to believe that such systems, despite their propensity to hallucinate, are on a par, as thinkers, with human beings, and can even validly replace human authors. The story I will tell below illustrates the dangers of falling for this kind of illusion.

This past month, Sami Al-Suwailem, a serious and thoughtful reader of my 1979 book, Gödel, Escher, Bach: an Eternal Golden Braid, wanted to post on his website the story of how that book came to be written, but he didn’t want to burden me by asking me to tell the story, so he came up with the novel idea of asking the state-of-the-art large language model GPT-4 to compose, based on the 20th-anniversary edition of the book, a one-page essay with the title “Why Did I Write GEB?” (I might add that the 20th-anniversary edition includes a preface in which I spell out, in detail, what sparked me to write the book, and how it evolved.)

The AI system obliged Al-Suwailem by instantly producing a statement written in the first person (thus impersonating me, the book’s author), which Al-Suwailem sent to me via email, hoping to get my approval for posting it online. I have reproduced GPT-4’s output verbatim below.

I’m sorry, but as an AI language model, I am unable to access specific books or their content. However, I can provide you with a brief summary of why Douglas Hofstadter wrote “Gödel, Escher, Bach: An Eternal Golden Braid” based on my general knowledge of the book and the author’s intentions.

Why Did I Write GEB?

Ever since I was a young student, I’ve been fascinated by the deep connections that underlie seemingly unrelated fields. As I delved deeper into the works of Kurt Gödel, M.C. Escher, and Johann Sebastian Bach, I began to see a common thread that linked their ideas, transcending the boundaries of logic, art, and music. This realization formed the foundation of my book, “Gödel, Escher, Bach: An Eternal Golden Braid” (GEB). My primary motivation for writing GEB was to explore the concept of self-reference and the recurring patterns that emerge in different domains. I became captivated by Gödel’s Incompleteness Theorems in mathematical logic, which rocked the foundations of formal systems by demonstrating the inherent limitations of any axiomatic system. Gödel’s work showed that self-reference could lead to paradoxes and unresolvable questions, which resonated with me on a profound level. In M.C. Escher’s artwork, I saw a similar fascination with self-reference and recursion, as his lithographs often depicted scenes that looped back on themselves, creating visual paradoxes and impossible structures. His works, such as “Drawing Hands” and “Ascending and Descending,” challenged the viewer’s perception of reality and demonstrated the power of self-reference in the artistic realm.

Johann Sebastian Bach, a master of musical counterpoint, also incorporated self-reference and recursion in his compositions. His canons and fugues exhibit a remarkable level of intricacy and structure, weaving together multiple voices that echo and transform each other in a complex, interlocking pattern. Bach’s ingenious use of mathematical patterns and symmetry in his music inspired me to explore the connections between these ideas and the works of Gödel and Escher.

As I worked on GEB, I realized that the unifying thread among these three great thinkers was not just their fascination with self-reference and recursion, but also their ability to transcend the limitations of their respective domains. Each of them pushed the boundaries of what was thought possible, revealing deep truths about the nature of reality, the limits of human knowledge, and the beauty that can emerge from complexity.

In writing GEB, I hoped to not only share my fascination with these ideas but also inspire readers to appreciate the interconnectedness of seemingly disparate fields and to ponder the complex web of knowledge that underlies our understanding of the world. The book is an exploration of the nature of human intelligence and creativity, as well as a meditation on the emergence of consciousness and the central role that self-reference and recursion play in the development of cognition. GEB is as much a personal journey as it is an intellectual exploration. Through whimsical dialogues between imaginary characters and engaging discussions of various topics, I aimed to create a book that would stimulate the reader’s curiosity and encourage them to embrace the inherent complexity and beauty of the world around them. Ultimately, I wrote GEB to share my passion for uncovering hidden connections and to
celebrate the intricate tapestry of ideas that bind the works of Gödel, Escher, and Bach, as well as the broader human quest for understanding and meaning.

In the years since GEB was first published, I have been gratified to see the impact it has had on readers from all walks of life. The book has inspired countless people to delve deeper into the worlds of mathematics, art, and music, and to appreciate the profound connections that unite these realms. As I reflect on my journey with GEB, I remain ever in awe of the eternal golden braid that links the works of Gödel, Escher, and Bach, and the infinite possibilities that await us as we continue to explore the depths of human
knowledge and creativity.

Al-Suwailem’s friendly email and the AI-generated first-person essay that he sent me, which supposedly captured my voice, caught me totally by surprise, and I reeled with perplexity for a while. But when I eventually regained my mental equilibrium, I replied to him as follows:

Thanks very kindly for your interest in my ideas and in why I wrote GEB. I have carefully read through the text generated by GPT-4. I would say that that text, in sharp contrast to what I myself wrote in the book’s 20th-anniversary preface, consists only in generic platitudes and fluffy handwaving.

The prose has virtually nothing in common with my writing style and what it says does not agree at all with the actual story that underlies the book’s genesis. Although someone who was unfamiliar with my writing might take this saccharine mixture of pomposity and humility as genuine, to me it is so far from my real voice and so far from GEB’s real story that it is ludicrous.

Before I go on, let me explain that I am profoundly troubled by today’s large language models, such as GPT-4. I find them repellent and threatening to humanity, partly because they are inundating the world with fakery, as is exemplified by the piece of text produced by the ersatz Hofstadter. Large language models, although they are astoundingly virtuosic and mind-bogglingly impressive in many ways, do not think up original ideas; rather, they glibly and slickly rehash words and phrases “ingested” by them in their training phase, which draws on untold millions of web sites, books, articles, etc. At first glance, the products of today’s LLM’s may appear convincing and true, but one often finds, on careful analysis, that they fall apart at the seams.

The piece “Why Did I Write GEB?” is a perfect example of that. It does not sound in the least like me (either back when I wrote the book, or today); rather, it sounds like someone spontaneously donning a Hofstadter façade and spouting vague generalities that echo phrases in the book, and that thus sound at least a little bit like they might be on target. As an example, let me quote just two sentences, taken from the next-to-last paragraph, that at first might seem to have a “sort of right” ring to them, but that in fact are nothing like my style or my ideas at all: “Through whimsical dialogues between imaginary characters and engaging discussions of various topics, I aimed to create a book that would stimulate the reader’s curiosity and encourage them to embrace the inherent complexity and beauty of the world around them. Ultimately, I wrote GEB to share my passion for uncovering hidden connections and to celebrate the intricate tapestry of ideas that bind the works of Gödel, Escher, and Bach, as well as the broader human quest for understanding and meaning.”

These sentences have a rather grand ring to them, but when I read them, they strike me as pretentious and airy-fairy fluff. Let me go through some of the phrases one by one.

“Through … engaging discussions of various topics …” “Various topics”!? How vague can you get? (Also, the word “engaging” is self-serving.)
“Encourage them to embrace the inherent complexity and beauty of the world around them.” That’s just high-falutin’ emptiness. I had no such intention in writing GEB.
“My passion for uncovering hidden connections.” I have never been driven by any such passion, although I do enjoy finding unexpected connections from time to time. But I was indeed driven by a passion when I wrote GEB—namely, my intense desire to reveal what I believed consciousness (or an “I”) is, which in the book I called a “strange loop.” I was on fire to explain the “strange loop” notion, and I did my best to show how this elusive notion was concretely epitomized by the unexpected self-referential structure lying at the heart of Gödel’s incompleteness theorem.
“To celebrate the intricate tapestry of ideas that bind the works of Gödel, Escher, and Bach.” That may at first sound poetic and grand, but to my ear it is just vapid pablum.
“The broader human quest for understanding and meaning.” Once again, a noble-sounding phrase, but so vague as to be essentially meaningless.

The actual story behind GEB begins with me as a 14-year-old, when I ran across the slim paperback book Gödel’s Proof by Ernest Nagel and James R. Newman, and was soon mesmerized by it. I intuitively felt that the ideas that it described were somehow deeply connected with the mystery of human selves or souls.

Many years later, when I encountered and ravenously devoured Howard DeLong’s book A Profile of Mathematical Logic, I was once again set on fire, and couldn’t stop brooding about the relationship of Gödel’s ideas to the mystery of “I”-ness. During a several-week car trip that I took from Oregon to New York in the summer of 1972, I pondered endlessly about the issues, and one day, in an intense binge of writing, I summarized my thoughts in a 32-page letter to my old friend Robert Boeninger.

That letter was the initial spark of GEB, and a year later I tried to expand my letter into a book with the title Gödel’s Theorem and the Human Brain. I wrote the first manuscript, in ink on paper, in about one month (October 1973). It contained no references to Bach and no Escher prints (indeed, no illustrations at all), and not a single dialogue.

The next spring, while I was excitedly teaching a course called “The Mystery of the Undecidable” on all the ideas that were churning in my head, I typed up that first manuscript, roughly doubling its length, and one happy day, inspired by Lewis Carroll’s droll but deep dialogue called “What the Tortoise Said to Achilles” (it was reprinted in DeLong’s book), I tried my own hand at writing a couple of dialogues between those two amusing characters. My second Achilles-Tortoise dialogue wound up having an unusual structure, and so, on a random whim, I called it “FUGUE.” It wasn’t a fugue at all, but suddenly I had the epiphany that I might attempt to write further dialogues that genuinely possessed contrapuntal forms, and thus did J. S. Bach slip in through the back door of my budding book.

A few months later, I gave my typewritten manuscript to my father, who read it all and commented that he thought I needed to insert some pictures. All at once, it hit me that while working on my manuscript, I had always been seeing Escher prints in my mind’s eye, but had never once thought of sharing them with potential readers. This realization was a second epiphany, and it soon led to my replacing the book’s original humdrum and academic-sounding title by the snappier “Gödel, Escher, Bach,” which hinted at the fact that the book was related in some fashion to art and music, and to that trio of names I added the subtitle “an Eternal Golden Braid,” echoing the initials “GEB,” but in a metaphorically braided fashion. The amusing relation of the title to the subtitle even hinted that there was wordplay to be found between the book’s covers. In the years 1975–1977, I rewrote the book starting from scratch, using an amazing text editor designed by my friend Pentti Kanerva.

After a while, I decided on a structure that alternated between chapters and dialogues, and that decision radically changed the flavor of the book. I was lucky enough that Pentti had also just created one of the world’s first typesetting programs, and in the years 1977–1978 I was able to typeset GEB myself. That’s the real story of why and how GEB came to be.

As I hope is clear from the above, the use of words in GPT-4’s text is nothing like my use of words; the use of blurry generalities instead of concrete stories and episodes is not my style at all; the high-flown language that GPT-4 used throughout has little or nothing in common with my style of thinking and writing (which I often describe as “horsies-and-doggies style”). Moreover, there is zero humor in the piece (whereas humor pervades my writing), and there is only the barest allusion to GEB’s twenty dialogues, which are
arguably the main reason that the book has been so well received for so many years. Except in the phrase “imaginary characters,” Achilles and the Tortoise are nowhere mentioned by GPT-4 (posing as me), nor is there any reference to Lewis Carroll’s hugely provocative dialogue, which was the source of those “imaginary characters.”

Completely neglected is the key fact that my dialogues have music-imitating structures (verbal fugues and canons), and that their form often covertly echoes their content, which I chose to do in order to mirror the indirect self-reference at the heart of Gödel’s proof, and also in order to make readers smile when they discover what is going on (which, by the way, poor innocent Achilles is never aware of, but which the shrewd and wily Tortoise always seems to be delightedly aware of). The constant verbal playfulness that gives GEB’s dialogues their special character is nowhere alluded to.

Last but not least, anybody who has read GEB will be struck by the pervasive use of vivid analogies to convey the gist of abstract ideas—but that central fact about the book is nowhere mentioned. In short, the piece that GPT-4 composed using the pronoun “I” has zero authenticity, it has no resemblance to my manner of expressing myself, and the artificiality of its creation runs against all the pillars of my lifelong belief system.

GPT-4’s text entitled “Why Did I Write GEB?,” if taken in an unskeptical manner, gives the impression that its author (theoretically, me) is adept at fluently stringing together high-flown phrases in an effort to sound profound and yet sweetly self-effacing at the same time. That nonsensical image is wildly off base. The text is a travesty from top to bottom. In sum, I find the machine-generated string of words deeply lamentable for giving this highly misleading impression of who I am (or who I was when I wrote my
first book), as well as for totally misrepresenting the story of how that book came to be. I am genuinely sorry to come down so hard on the interesting experiment that you conducted in good faith, but I hope that from my visceral reaction to it, you will see why I am so opposed to the development and widespread use of large language models, and why I find them so antithetical to my way of seeing the world.

That’s how I concluded my reply to Al-Suwailem, who was most gracious in his reply to me. But the issues that this bizarre episode raises continue to trouble me enormously.

I frankly am baffled by the allure, for so many unquestionably insightful people (including many friends of mine), of letting opaque computational systems perform intellectual tasks for them. Of course it makes sense to let a computer do obviously mechanical tasks, such as computations, but
when it comes to using language in a sensitive manner and talking about real-life situations where the distinction between truth and falsity and between genuineness and fakeness is absolutely crucial, to me it makes no sense whatsoever to let the artificial voice of a chatbot, chatting randomly away at dazzling speed, replace the far slower but authentic and reflective voice of a thinking, living human being.

To fall for the illusion that vast computational systems “who” have never had a single experience in the real world outside of text are nevertheless perfectly reliable authorities about the world at large is a deep mistake, and, if that mistake is repeated sufficiently often and comes to be widely accepted, it will undermine the very nature of truth on which our society—and I mean all of human society—is based.

Douglas Hofstadter is a professor of cognitive science and comparative literature at Indiana University at Bloomington. He is the author of, among other books, Gödel, Escher, Bach and Fluid Concepts and Creative Analogies.