來源：https://e2eml.school/transformers.html#softmax
中英雙語版，由各類翻譯程序和少量自己理解的意思做中文注釋

相關文章匯總在文集：Transformers from Scratch（中文注釋）

--------------------------------------------------------------------------------------------------------------------

On more careful consideration, this is unsatisfying.?

仔細想想，這是令人不滿意的。

The difference between a vote total of 4 and 5 is relatively small.?

投票總數(shù)為4票和5票之間的差距相對較小。

It suggests that the model isn't as confident as it could be.?

這表明該模型并沒有想象中的那么自信。

And in a larger, more organic language model it's easy to imagine that such a slight difference could be lost in the statistical noise.

在一個更大的更有機的語言模型中，很容易想象這種微小的差異可能會在統(tǒng)計噪聲中丟失。

We can sharpen the prediction by weeding out all the uninformative feature votes.

我們可以通過剔除所有無信息的特征投票來增強預測。

With the exception of?battery, ran?and?program, ran.?It's helpful to remember at this point that we pull the relevant rows out of the transition matrix by multiplying it with a vector showing which features are currently active.

除“battery“,?”ran“?and ”program, ran“. 外，在這一點上，我們需要記住的是，我們將轉(zhuǎn)換矩陣中的相關行與顯示當前活動的特征的向量相乘。

For this example so far, we've been using the implied feature vector shown here.

到目前為止，對于此示例，我們一直在使用此處顯示的隱含特征向量。

It includes a one for each feature that is a combination of ran with each of the words that come before it.

它包括一個1對每個特征，它是ran與前面的每個單詞的組合。

Any words that come after it don't get included in the feature set. (In the next word prediction problem these haven't been seen yet, and so it's not fair to use them predict what comes next.)

之后出現(xiàn)的任何單詞都不會包含在特征集中。（在下一個單詞預測問題中，這些還沒有被發(fā)現(xiàn)，因此使用它們來預測下一個詞是不公平的。）

And this doesn't include all the other possible word combinations. We can safely ignore these for this example because they will all be zero.

這不包括所有其他可能的單詞組合。在這個例子中，我們可以安全地忽略這些，因為它們都是零。

To improve our results, we can additionally force the unhelpful features to zero by creating a mask.

為了改善我們的結果，我們還可以通過創(chuàng)建一個掩碼，將無用的特性強制為零。

It's a vector full of ones except for the positions you'd like to hide or mask, and those are set to zero.
這是一個都是1的向量，你想要隱藏或屏蔽的位置都設置為0。

In our case we'd like to mask everything except for battery, ran and program, ran, the only two features that have been of any help.

在我們的案例中，我們希望屏蔽除battery, ran 和 program, ran之外的所有特征，這是唯一有幫助的兩個特征。

To apply the mask, we multiply the two vectors element by element.

要應用這個掩碼，我們將兩個向量逐個元素相乘。

Any feature activity value in an unmasked position will be multiplied by one and left unchanged.

未掩碼位置中的任何特征活動值都將乘以1并保持不變。

Any feature activity value in a masked position will be multiplied by zero, and thus forced to zero.

掩碼位置中的任何特征活動值都將乘以0，從而強制為0。

The mask has the effect of hiding a lot of the transition matrix.

掩碼具有隱藏大量轉(zhuǎn)換矩陣的效果。

It hides the combination of ran with everything except battery and program, leaving just the features that matter.

它隱藏了ran與除了battery和program外的所有特征的組合，只留下重要的功能。

After masking the unhelpful features, the next word predictions become much stronger.

在掩蓋了這些無用的特征之后，下一個單詞的預測會變得更加強烈。

When the word?battery?occurs earlier in the sentence, the word after?ran?is predicted to be?down?with a weight of 1 and?please?with a weight of 0.

當單詞battery出現(xiàn)在句子的前面時，單詞ran之后的權重為1，please的權重為0。

What was a weight difference of 25 percent has become a difference of infinity percent.

由原來25%的權重差異變成了無窮大的差異。

There is no doubt what word comes next.

毫無疑問，下一個詞是什么。

The same strong prediction occurs for?please?when?program?occurs early on.

同樣強烈的預測發(fā)生在“please”，當“program”早期出現(xiàn)時。

This process of selective masking is the?attention?called out in the title of the original?paper?on transformers.

這種選擇性掩蔽的過程是transformers原始論文標題中所提到的注意力。

So far, what we've descibed is a just an approximation of how attention is implemented in the paper.

到目前為止，我們所描述的只是本文中注意力如何實現(xiàn)的一個近似。

It captures the important concepts, but the details are different. We'll close that gap later.

它抓住了重要的概念，但細節(jié)不同。我們稍后會縮小差距。