The Training of ChatGPT 

ChatGPT 的训练

OK, so we’ve now given an outline of how ChatGPT works once it’s set up. But how did it get set up? How were all those 175 billion weights in its neural net determined? Basically they’re the result of very large-scale training, based on a huge corpus of text—on the web, in books, etc.—written by humans. As we’ve said, even given all that training data, it’s certainly not obvious that a neural net would be able to successfully produce “human-like” text. And, once again, there seem to be detailed pieces of engineering needed to make that happen. But the big surprise—and discovery—of ChatGPT is that it’s possible at all. And that—in effect—a neural net with “just” 175 billion weights can make a “reasonable model” of text humans write.

好了，我们现在已经描述了一旦 ChatGPT 建立起来，它是如何工作的。但是它是如何建立起来的呢？神经网络中的那 1750 亿个权重是如何确定的？基本上，它们是基于大量的文本（来自网络、书籍等）所进行的大规模训练的结果，这些文本都是人类编写的。正如我们所说，即使有了所有的训练数据，神经网络能够成功地生成“类人”的文本也绝非显而易见。而且，再次强调，似乎需要详细的工程部分来实现这一目标。但 ChatGPT 的一个巨大的惊喜和发现是，这样的事情竟然是可能的。实际上，一个拥有“仅仅” 1750 亿权重的神经网络，就可以成为一个对人类书写文本的“合理模型”。

In modern times, there’s lots of text written by humans that’s out there in digital form. The public web has at least several billion human-written pages, with altogether perhaps a trillion words of text. And if one includes non-public webpages, the numbers might be at least 100 times larger. So far, more than 5 million digitized books have been made available (out of 100 million or so that have ever been published), giving another 100 billion or so words of text. And that’s not even mentioning text derived from speech in videos, etc. (As a personal comparison, my total lifetime output of published material has been a bit under 3 million words, and over the past 30 years I’ve written about 15 million words of email, and altogether typed perhaps 50 million words—and in just the past couple of years I’ve spoken more than 10 million words on livestreams. And, yes, I’ll train a bot from all of that.)

在现代社会，人类编写的大量文本以数字形式存在。公共网络上至少有数十亿个由人类编写的页面，总共可能有一万亿字左右的文本。如果算上非公共网页，这个数字可能至少增加 100 倍。到目前为止，已经有 500 多万本数字化图书可供使用（从历史上看，共出版了约 1 亿本图书），另有大约 1000 亿字的文本。这还不包括来自视频中的语音转化成的文本等。（作为个人对比，我一生中出版的材料总共不到 300 万字，过去 30 年里我写了大约 1500 万字的电子邮件，总共可能敲了 5000 万字。仅在过去的几年里，我在直播中就说了超过 1000 万字。而且，是的，我将从所有这些内容中训练一个机器人。）

But, OK, given all this data, how does one train a neural net from it? The basic process is very much as we discussed it in the simple examples above. You present a batch of examples, and then you adjust the weights in the network to minimize the error (“loss”) that the network makes on those examples. The main thing that’s expensive about “back propagating” from the error is that each time you do this, every weight in the network will typically change at least a tiny bit, and there are just a lot of weights to deal with. (The actual “back computation” is typically only a small constant factor harder than the forward one.)

那么，既然有了这些数据，如何从中训练神经网络呢？基本过程与我们在上面简单示例中讨论的非常相似。你呈现一批示例，然后调整网络中的权重以使网络在这些示例上产生的误差（“损失”）最小化。关于“反向传播”错误的主要昂贵之处在于，每次执行此操作时，网络中的每个权重通常都会发生至少微小的变化，而且还有许多权重需要处理。（实际的“反向计算”通常只比正向计算稍难一些。）

With modern GPU hardware, it’s straightforward to compute the results from batches of thousands of examples in parallel. But when it comes to actually updating the weights in the neural net, current methods require one to do this basically batch by batch. (And, yes, this is probably where actual brains—with their combined computation and memory elements—have, for now, at least an architectural advantage.)

借助现代 GPU 硬件，可以轻松地并行计算数千个示例的结果。但是在实际更新神经网络中的权重时，当前的方法需要逐批次进行。（是的，这可能是实际大脑（具有结合计算和存储单元）至少在架构上具有优势的地方。）

Even in the seemingly simple cases of learning numerical functions that we discussed earlier, we found we often had to use millions of examples to successfully train a network, at least from scratch. So how many examples does this mean we’ll need in order to train a “human-like language” model? There doesn’t seem to be any fundamental “theoretical” way to know. But in practice ChatGPT was successfully trained on a few hundred billion words of text.

即使在我们之前讨论过的学习数值函数的看似简单的情况下，我们也发现通常需要使用数百万个示例来成功训练网络，至少是从头开始的。那么，这意味着我们需要多少个示例来训练一个“类人语言”的模型呢？似乎没有任何基本的“理论”方法来知道。但实际上，ChatGPT 已经成功地通过几千亿字的文本进行了训练。

Some of the text it was fed several times, some of it only once. But somehow it “got what it needed” from the text it saw. But given this volume of text to learn from, how large a network should it require to “learn it well”? Again, we don’t yet have a fundamental theoretical way to say. Ultimately—as we’ll discuss further below—there’s presumably a certain “total algorithmic content” to human language and what humans typically say with it. But the next question is how efficient a neural net will be at implementing a model based on that algorithmic content. And again we don’t know—although the success of ChatGPT suggests it’s reasonably efficient.

有些文本被反复输入了多次，有些只输入了一次。但不知何故，它从所看到的文本中“获得了所需”。考虑到这么多的文本供其学习，需要多大的网络才能“学得好”呢？同样，我们尚没有一个基本的理论方法来回答这个问题。归根结底，正如我们将在下面进一步讨论的那样，人类语言及其典型表达应该具有一定的“总体算法内容”。但下一个问题是神经网络在实现基于该算法内容的模型时的效率如何。我们同样不知道，尽管 ChatGPT 的成功表明其效率相当不错。

And in the end we can just note that ChatGPT does what it does using a couple hundred billion weights—comparable in number to the total number of words (or tokens) of training data it’s been given. In some ways it’s perhaps surprising (though empirically observed also in smaller analogs of ChatGPT) that the “size of the network” that seems to work well is so comparable to the “size of the training data”. After all, it’s certainly not that somehow “inside ChatGPT” all that text from the web and books and so on is “directly stored”. Because what’s actually inside ChatGPT are a bunch of numbers—with a bit less than 10 digits of precision—that are some kind of distributed encoding of the aggregate structure of all that text.

最后我们可以注意到，ChatGPT 在处理时使用了大约几百亿个权重——与其接受的训练数据的总单词数（或标记数）相当。从某种程度上说，可能令人惊讶的是（尽管在 ChatGPT 的较小类似物中也经验观察到），似乎能够良好运作的“网络规模”与“训练数据规模”如此相近。毕竟，肯定不是因为“inside ChatGPT”，网络和书籍等所有文本都是“直接存储”的。因为实际上，ChatGPT 内部是一堆数字——精度略低于 10 位数——这些数字以某种分布式编码方式表示了所有文本的总体结构。

Put another way, we might ask what the “effective information content” is of human language and what’s typically said with it. There’s the raw corpus of examples of language. And then there’s the representation in the neural net of ChatGPT. That representation is very likely far from the “algorithmically minimal” representation (as we’ll discuss below). But it’s a representation that’s readily usable by the neural net. And in this representation it seems there’s in the end rather little “compression” of the training data; it seems on average to basically take only a bit less than one neural net weight to carry the “information content” of a word of training data.

换句话说，我们可能会问，人类语言及其典型表达的“有效信息内容”是什么。有原始的语言示例语料库，然后还有 ChatGPT 神经网络中的表示。这种表示很可能远非“算法最小”的表示（我们将在下文讨论）。但这是神经网络容易使用的表示。在这种表示中，最终对训练数据的“压缩”似乎相当有限；平均而言，大约每个神经网络权重只携带了训练数据中一个词的“信息内容”。

When we run ChatGPT to generate text, we’re basically having to use each weight once. So if there are n weights, we’ve got of order n computational steps to do—though in practice many of them can typically be done in parallel in GPUs. But if we need about n words of training data to set up those weights, then from what we’ve said above we can conclude that we’ll need about n2 computational steps to do the training of the network—which is why, with current methods, one ends up needing to talk about billion-dollar training efforts.

当我们运行 ChatGPT 生成文本时，我们基本上需要一次性使用每个权重。因此，如果有 n 个权重，我们需要执行 n 个计算步骤，尽管实际上许多计算步骤通常可以在 GPU 中并行执行。但是，如果我们需要大约 n 个训练数据字来设置这些权重，那么根据我们上面的讨论，我们可以得出我们将需要大约 n² 个计算步骤来完成网络的训练——这就是为什么，使用当前方法，我们最终需要谈论十亿美元的训练投入。

“点赞有美意，赞赏是鼓励”