來源：https://e2eml.school/transformers.html#softmax

中英雙語版，由微軟翻譯和少量自己理解的意思做中文注釋

We can set aside matrices for a minute and get back to what? we really care about,sequences of words.

我們可以先把矩陣放在一邊，回到我們真正關心的問題上來，詞的序列。

Imagine that as we start to develop our natural language computer interface we want to handle just three possible commands:

想象一下，當我們開始開發(fā)我們的自然語言計算機界面時，我們只想處理三種可能的命令。

• Show me my directories please.（請給我看看我的目錄。）

• Show me my files please.（請給我看我的文件。）

• Show me my photos please.（請給我看看我的照片。）

Our vocabulary size is now seven: {directories, files, me, my, photos, please, show}.

我們的詞匯量現(xiàn)在是7個: {directories, files, me, my, photos, please, show}.

One useful way to represent sequences is with a transition model.

表示序列的一個有用的方法是用一個過渡模型。

For every word in the vocabulary, it shows what the next word is likely to be.

對于詞匯中的每一個單詞，它顯示下一個詞可能是什么。

If users ask about photos half? the time, files 30% of the time, and directories the rest of the time,

如果用戶一半時間詢問照片，30% 時間詢問文件，其余時間詢問目錄，

the transition model will look like this. The sum of the transitions away from any word will always add up to one.

過渡模型將如下所示。任何單詞的過渡之和將始終加起來為 1。

? ? ? ?

This particular transition model is called a Markov chain,

這種特殊的過渡模型稱為馬爾可夫鏈，

because it satisfies the Markov property that the probabilities for the next word depend only on recent words.

因為它滿足馬爾可夫性質，即下一個單詞的概率僅取決于最近的單詞。

More specifically, it is a first order Markov model because it only looks at the single most recent word.

更具體地說，它是一個一階馬爾可夫模型，因為它只查看一個最近的單詞。

If it considered the two most recent words it would be a second order Markov model.

如果它考慮最近的兩個詞，它將是一個二階馬爾可夫模型。

Our break from matrices is over.

我們回到矩陣。

It turns out that Markov chains can be expressed conveniently in matrix form.?

事實證明，馬爾可夫鏈可以方便地以矩陣形式表示。

Using the same indexing scheme that we used when creating one-hot vectors,

使用我們在創(chuàng)建獨熱向量時使用的相同索引方案，

each row represents one of the words in our vocabulary.

每一行代表我們詞匯表中的一個單詞。

So does each column. The matrix transition model treats a matrix as a lookup table.

每列也是如此。矩陣轉換模型將矩陣視為查找表。

Find the row that corresponds to the word you’re interested in.

找到與您感興趣的字詞對應的行。

The value in each column shows the probability of that word coming next.

每列中的值顯示該單詞接下來出現(xiàn)的概率。

Because the value of each element in the matrix represents a probability, they will all fall between zero and one. ? ? ? ??
由于矩陣中每個元素的值表示一個概率，因此它們都將介于 0 和 1 之間。???

Because probabilities always sum to one, the values in each row will always add up to one.

?由于概率總和始終為 1，因此每行中的值總和始終為 1。

In the transition matrix here we can see the structure of our three sentences clearly.

在過渡矩陣中，我們可以清楚地看到三個句子的結構。

Almost all of the transition probabilities are zero or one.

幾乎所有的轉移概率都是零或一。

There is only one place in the Markov chain where branching happens.

馬爾可夫鏈中只有一個地方發(fā)生分支。

After my, the words directories, files, or photos might appear, each with a different probability.

在 my 之后，可能會出現(xiàn)目錄、文件或照片等詞，每個詞都有不同的概率。

Other than that,? there’s no uncertainty about which word will come next.

除此之外，沒有不確定性下一個詞。

That certainty is reflected by having mostly ones and zeros in the transition matrix.

這種確定性反映在過渡矩陣中主要有 1 和 0。

We can revisit our trick of using matrix multiplication with a one-hot vector to pull out the transition probabilities associated with any given word.

我們可以重新審視我們的技巧，即使用矩陣乘法和 one-hot 向量來提取與任何給定單詞相關的轉移概率。

For instance, if we just wanted to isolate the probabilities of which word comes after my,

例如，如果我們只是想隔離哪個詞在我之后的概率，

we can create a one-hot vector representing the word my? and multiply it by our transition matrix.

我們可以創(chuàng)建一個表示單詞 My 的 one-hot 向量，并將其乘以我們的轉移矩陣。

This pulls out the row? the relevant row and shows us the probability distribution of what the next word will be.

這將拉出相關行的行，并向我們顯示下一個單詞的概率分布。