機(jī)器學(xué)習(xí)--高數(shù)概率論第一章

2023-03-19 12:20 作者:圣母和正負(fù)喜歡沒辦法 0人讀過 | 我要投稿

一、微積分

1 夾逼定理

$x%5Cin%20U(x_%7B0%7D%20)%E6%97%B6%EF%BC%8Cg(x)%5Cleq%20f(x)%5Cleq%20h(x)%E6%88%90%E7%AB%8B%EF%BC%8C%E4%B8%94%5Clim_%7Bx%5Cto0%7D%20g(x)%3DA%2C%5Clim_%7Bx%5Cto0%7D%20h(x)%3DA%EF%BC%8C%E5%88%99%5Clim_%7Bx%5Cto0%7D%20f(x)%3DA$

$%5Csin%20x%20%3C%20x%20%3C%20%5Ctan%20x%20%20$ ? ?? $x%20%5Cin%20U(0%2C%5Cvarepsilon%20)$

$%5Clim_%7Bx%5Cto0%7D%20%5Cfrac%7Bsinx%7D%7Bx%7D%20%3D%201$

2 導(dǎo)數(shù)

即是曲線的斜率，是曲線變化的快慢。

路程是原始函數(shù)

一階導(dǎo)數(shù)------速度

二階導(dǎo)數(shù)------加速度

$(a%5Ex)'%3Da%5Ex%5Cln%20a$

題目:

1? $f(x)%3Dx%5Ex$ 的最小值

? 兩邊同時(shí)取對(duì)數(shù)然后求導(dǎo)，

? 令t'=0帶入計(jì)算求出t。

?之后做最小優(yōu)化時(shí)要用，基本也是求最小值

2 泰勒公式

$f(x)%3Df(x_%7B0%7D%20)%2Bf'(x_%7B0%7D%20)(x-x_%7B0%7D%20)%2B%5Cfrac%7Bf'(x_%7B0%7D%20)%7D%7B2!%7D(x-x_%7B0%7D%20)%5E2%2B...%2B%5Cfrac%7Bf'(x_%7B0%7D%20)%7D%7Bn!%7D(x-x_%7B0%7D)%5En%2BR_%7B0%7D(x)%20$

?其實(shí)多項(xiàng)式求解要容易一些。

$%20e%5Ex%20%20%20%20%20$ ? ? $sinx%0A$ ? 都是一樣計(jì)算，可以理解成無限逼近。

3 基尼指數(shù)

$f(x)%3D-lnx%E5%9C%A8x%3D1%E5%A4%84%E7%9A%84%E4%B8%80%E9%98%B6%E5%B1%95%E5%BC%80$

$H(x)%3D-%5Csum_%7Bk%3D1%7D%5EKp_%7Bk%7Dln%5Cln%20p_%7Bk%7D%3D%20%5Csum_%7Bk%3D1%7D%5EKp_%7Bk%7D(1-%20p_%7Bk%7D)$

4 方向?qū)?shù)

$z%3Df(x%2Cy)%E5%9C%A8%E7%82%B9P(x%2Cy)%E5%8F%AF%E5%BE%AE%EF%BC%8C%E5%88%99%E5%87%BD%E6%95%B0%E5%9C%A8%E8%AF%A5%E7%82%B9%E6%B2%BF%E4%BB%BB%E6%84%8F%E6%96%B9%E5%90%91%E5%AF%BC%E6%95%B0%E9%83%BD%E5%AD%98%E5%9C%A8%0A%5Cfrac%7B%5Cvartheta%20f%7D%7B%5Cvartheta%20l%7D%20%3D%20%20%5Cfrac%7B%5Cvartheta%20f%7D%7B%5Cvartheta%20x%7Dcos%5Cvarphi%20%2B%20%5Cfrac%7B%5Cvartheta%20f%7D%7B%5Cvartheta%20y%7Dsin%5Cvarphi%20$

$%5Cpsi%20$ 是x軸到L的轉(zhuǎn)角

5 梯度

? 梯度的方向是函數(shù)在該方向變化最快的方向

即：解析式z=H(x,y)的山，（ $x_%7B0%7D%EF%BC%8Cy_%7B0%7D%20$ ）的梯度變化最快

梯度下降法：

考慮自己下山方向和梯度呈 $%5Ctheta%20$ 夾角，下降速度是多少？

?6?凹函數(shù)(二階導(dǎo)數(shù)大于0)

（碗狀函數(shù)）

$f(%5Ctheta%20x%2B(1-%5Ctheta)y%20)%5Cleq%20%5Ctheta%20f(x)%2B(1-%5Ctheta)f(y)%20%20%E5%AF%B9%E4%BA%8E%5Cforall%20x%2Cy%5Cin%20dom%20f%20%2C%20%200%20%5Cleq%20%5Ctheta%5Cleq1$

也有： $f(%5Ctheta_%7B1%7D%20x_%7B1%7D%20%2B...%2B%5Ctheta_%7Bn%7D%20x_%7Bn%7D%20%20)%5Cleq%20%5Ctheta_%7B1%7Df(%5Ctheta_%7B1%7D)%2B...%2B%5Ctheta_%7Bn%7Df(%5Ctheta_%7Bn%7D)%2C%E5%85%B6%E4%B8%AD0%5Cleq%5Ctheta_%7Bi%7D%5Cleq1%2C%5Ctheta_%7B1%7D%2B...%2B%5Ctheta_%7Bn%7D%3D1$

有最小值，便于優(yōu)化。

應(yīng)用：最大熵模型---互相損失

$D(p%7C%7Cq)%3D%5Csum_%7Bx%7Dp(x)%5Clog%20%5Cfrac%7Bp(x)%7D%7Bq(x)%7D%3DE_%7Bp(x)%7D%5Clog%5Cfrac%7Bp(x)%7D%7Bq(x)%7D%20%0A$

證明D(p||q)?

$%E5%B0%B1%E6%B1%82-%5Clog%5Csum_%7Bx%7D(p(x)%5Cfrac%7Bq(x)%7D%7Bp(x)%7D%20)%3D-%5Clog%5Csum_%7Bx%7Dq(x)%3D0$

7概率論

事件和概率沒有必然關(guān)系

概率為0不代表事件就不發(fā)生

1>累計(jì)分布：

$%5Cphi%20(x)%E5%8D%95%E5%A2%9E%EF%BC%8Cmin(%5Cphi(x))%3D0%2Cmax(%5Cphi(x))%3D1$

$%E5%80%BC%E5%9F%9F%5Cin%20%5B0%2C1%5D%E4%B8%8Ay%3Df(x)%E7%9C%8B%E6%88%90y%E7%9A%84%E4%BA%8B%E4%BB%B6%E7%B4%AF%E8%AE%A1%E6%A6%82%E7%8E%87$

$y%E5%A6%82%E6%9E%9C%E5%8F%AF%E5%AF%BC%EF%BC%8Cp%3Df'(x)%E7%9C%8B%E6%88%90%E5%85%B6%E6%A6%82%E7%8E%87%E5%AF%86%E5%BA%A6%E5%87%BD%E6%95%B0%E3%80%82$

2>古典概率

n個(gè)不同球放入N（N>n）個(gè)盒子，盒子不限，求事件A={每個(gè)盒子最多有1個(gè)球}

$P(A)%3D%5Cfrac%7BC_%7BN%7D%5En%20%7D%7BN%5En%20%7D%20%EF%BC%8C%E5%9F%BA%E6%9C%AC%E4%BA%8B%E4%BB%B6N%5En%E4%B8%AA%EF%BC%8C%E5%85%B1N(N-1)...(N-N%2B1)%E7%A7%8D$

3>生日悖論

套用上訴公式

會(huì)發(fā)現(xiàn)人數(shù)越多，概率越大

4>古典概率

$%E9%BA%BB%E5%B0%86136%E5%BC%A0%EF%BC%8C%E9%9A%8F%E6%9C%BA%E9%80%894%E5%BC%A0%EF%BC%8C%E5%8F%96%E6%B3%95C_%7B136%7D%5E%7B14%7D$

$%E5%BA%84%E5%AE%B614%E5%BC%A0%EF%BC%8C%E5%85%B6%E4%BB%9613%E5%BC%A0$

$p%3D%5Cfrac%7BC_%7B34%7D%5E%7B14%7D%204%5E%7B14%7D%20%7D%7BC_%7B136%7D%5E%7B14%7D%20%7D%20%3D%200.0879$

5>裝箱問題

$12%E4%BB%B6%E6%AD%A3%E5%93%81%E3%80%813%E4%BB%B6%E6%AC%A1%E5%93%81%EF%BC%8C%E9%9A%8F%E6%9C%BA%E8%A3%85%E5%85%A53%E4%B8%AA%E7%AE%B1%E5%AD%90%E3%80%81%E6%AF%8F%E7%AE%B1%E8%A3%855%E4%BB%B6%EF%BC%8C%E6%AF%8F%E7%AE%B1%E6%81%B0%E8%A3%851%E4%BB%B6%E6%AC%A1%E5%93%81%E6%A6%82%E7%8E%87$

$%E5%85%B1%E6%9C%89%EF%BC%9A15!%2F(5!5!5!)%E8%A3%85%E6%B3%95$

$%E6%AC%A1%E5%93%81%E8%A3%85%E6%B3%95%3A%203%EF%BC%81$

$%E6%AD%A3%E5%93%81%E8%A3%85%E6%B3%95%EF%BC%9A12!%2F(4!4!4!)$

$P(A)%3D(3!*12!(4!4!4!))%2F(15!%2F(5!5!5!))%3D25%2F91$

6>和組合數(shù)關(guān)系

n個(gè)物品分成k組，每組物品個(gè)數(shù)n1,n2,n3,n4...nk,(n1+...+nk=n), $%E5%88%86%E7%BB%84%E6%96%B9%E6%B3%95%5Cfrac%7Bn!%7D%7Bn_%7B1%7D%20n_%7B2%7D%20...n_%7Bk%7D%20%7D%20$

優(yōu)化：物品分組：第一組m個(gè)，第二組n-m個(gè)， $%5Cfrac%7Bn!%7D%7Bm!(n-m)!%7D%20%3D%20C_%7Bn%7D%5Em%20$

7>推薦系統(tǒng)

驚喜度、喜愛度

A和B兩個(gè)商品和用戶匹配度為0.8和0.2，系統(tǒng)將隨機(jī)為A生成一個(gè)均勻分布在0-0.8之間。B在0-0.2之間，計(jì)算B最終分?jǐn)?shù)大于A的概率。

$A%3DB%E7%9A%84%E7%9B%B4%E7%BA%BF%E4%B8%8A%E6%96%B9%E5%8C%BA%E5%9F%9F%EF%BC%8CB%3EA%E7%9A%84%E6%83%85%E5%86%B5$

$S_%7BA%7D%3D0.02%20%2CS_%7BB%7D%3D0.16%20$

$p%3D0.02%2F0.16%3D0.125$

8> 概率公式

1 條件概率

在B發(fā)生條件下A發(fā)生的概率

$P(A%7CB)%3D%5Cfrac%7BP(AB)%7D%7BP(B)%7D%20$

2 全概率公式

$P(A)%3D%5Csum_%7Bi%7DP(A%7CB_%7Bi%7D%20)P(B_%7Bi%7D)$

3 貝葉斯公式

$P(B_%7Bi%7D%7CA)%3D%20%5Cfrac%7BP(A%7CB_%7Bi%7D)P(B_%7Bi%7D)%7D%20%7B%5Csum_%7Bj%7DP(A%7CB_%7Bj%7D%20)P(B_%7Bj%7D)%7D$

用于反推

例程：

8支槍，5支校準(zhǔn)，3支沒校準(zhǔn)，校準(zhǔn)射中靶概率0.8，沒校準(zhǔn)的0.3，從8支任取一把射擊中靶，這把是校準(zhǔn)的概率。

典型反推用貝葉斯，已知結(jié)果求概率

$P(G%3D1)%3D%5Cfrac%7B5%7D%7B8%7D%20%20%2C%20P(G%3D0)%3D%5Cfrac%7B3%7D%7B8%7D%20$

$P(A%3D1%7CG%3D1)%3D0.8%20%2C%20P(A%3D0%7CG%3D1)%3D0.2%20$

$P(A%3D1%7CG%3D0)%3D0.3%20%2C%20P(A%3D0%7CG%3D0)%3D0.7%20$

$P(G%3D1%7CA%3D1)%3D%3F$

$P(G%3D1%7CA%3D1)%3D%5Cfrac%7BP(A%3D1%7CG%3D1)P(G%3D1)%7D%20%7B%5Csum_%7Bi%7DP(A%3D1%7CG%3Di%20)P(G%3Di)%7D%3D0.8163$

兩大學(xué)派：

頻率學(xué)派：假定參數(shù)是某個(gè)未知定值，求這些參數(shù)如何取值，能達(dá)到目標(biāo)函數(shù)極大、極小取值

貝葉斯派：假定參數(shù)可變，服從某個(gè)分布，求這些分布下某個(gè)目標(biāo)函數(shù)極大、極小

大數(shù)據(jù)：屬于頻率學(xué)派

9 常見分布

1 0-1分布

$E(X)%3D1*p%2B0*q%3Dp$

$D(X)%3DE(x%5E2%20)-%5BE(X)%5D%5E2%3D1*p%2B0*(1-p)-p%5E2$

2 二項(xiàng)分布（伯努利分布）

服從參數(shù)為n，概率為p的分布

比如拋硬幣

$X%3D%5Csum_%7Bi%3D1%7D%5EnX_%7Bi%7D%20$

$E(X)%3D%5Csum_%7Bi%3D1%7D%5EnE(X_%7Bi%7D)%3Dnp$

$D(X)%3D%5Csum_%7Bi%3D1%7D%5EnD(X_%7Bi%7D)%3Dnp(1-p)$

分布律

$P(X%3Dk)%3DC_%7Bn%7D%5Ek%20p%5Ek(1-p)%5E%7Bn-k%7D$

$%E5%88%99E(X)%3D%5Csum_%7Bk%3D0%7D%5EnkP(X%3Dk)%3D%5Csum_%7Bk%3D0%7D%5EnkC_%7Bn%7D%5E%7Bk%7Dp%5E%7Bk%7D(1-p)%5E%7Bn-k%7D$

$%20%3D%5Csum_%7Bk%3D0%7D%5En%20%5Cfrac%7Bkn!%7D%7Bk!(n-k)!%7Dp%5Ek(1-p)%5E%7Bn-k%7D$

$%3D%5Csum_%7Bk%3D1%7D%5E%7Bn%7D%5Cfrac%7Bnp(n-1)!%7D%7B(k-1)!%5B(n-1)-(k-1)%5D!%7Dp%5E%7Bk-1%7D(1-p)%5E%7B(n-1)-(k-1)%7D%20$

$%3Dnp%5Bp%2B(1-p)%5D%5E%7Bn-1%7D%3Dnp$

3 泊松分布

$e%5Ex%3D1%2Bx%2B%5Cfrac%7Bx%5E2%7D%7B2!%7D%2B...%2B%5Cfrac%7Bx%5Ek%7D%7Bk!%7D%2BR_%7Bk%7D$

$1%3D1*e%5E%7B-x%7D%2Bx%2B%5Cfrac%7Bx%5E2%7D%7B2!%7De%5E%7B-x%7D%2B...%2B%5Cfrac%7Bx%5Ek%7D%7Bk!%7De%5E%7B-x%7D%2BR_%7Bk%7De%5E%7B-x%7D$

$%E5%90%83%E5%B1%8E%EF%BC%9A%E5%B0%86x%E7%9C%8B%E6%88%90%5Clambda%20$

$%5Cfrac%7Bx%5Ek%7D%7Bk!%7D*e%5E%7B-x%7D----%3E%20%5Cfrac%7B%5Clambda%5Ek%7D%7Bk!%7D*e%5E%7B-%5Clambda%20%7D$

$%E5%88%86%E5%B8%83%E5%BE%8B%EF%BC%9AP%7B(X%3Dk)%7D%3D%5Cfrac%7B%5Clambda%5E%7Bk%7D%7D%7Bk!%7De%5E%7B-%5Clambda%7D$

$E(X)%3D%5Csum_%7Bk%3D0%7D%5Enk%5Cfrac%7B%5Clambda%5Ek%7D%7Bk!%7D*e%5E%7B-%5Clambda%7D%3De%5E%7B-%5Clambda%7D%5Csum_%7Bk%3D1%7D%5En%5Cfrac%7B%5Clambda%5E%7Bk-1%7D%7D%7B(k-1)!%7D*%5Clambda%20%3D%5Clambda*e%5E%7B-%5Clambda%7D*e%5E%7B%5Clambda%7D%3D%5Clambda$

應(yīng)用：機(jī)器故障、產(chǎn)品缺陷、細(xì)菌分布、放射性物質(zhì)單位時(shí)間發(fā)射粒子數(shù)、火車客戶、次數(shù)。

4 均勻分布

$E(X)%3D%5Cint_%7B%5Cvarpi%7D%5E%7B%5Cvarpi%20%7D%20xf(x)dx%3D%5Cint_%7Ba%7D%5E%7Bb%7D%20%5Cfrac%7B1%7D%7Bb-a%7D%20xdx%3D%5Cfrac%7B1%7D%7B2%7D(a%2Bb)%20$

$D(x)%3DE(X%5E2)-%5BE(X)%5D%5E2%3D%5Cint_%7Ba%7D%5E%7Bb%7D%20x%5E2%5Cfrac%7B1%7D%7Bb-a%7Ddx-(%5Cfrac%7Ba%2Bb%7D%7B2%7D)%5E2%3D%5Cfrac%7B(b-a)%5E2%7D%7B12%7D$

5 指數(shù)分布

$E(X)%3D%5Cint_%7B%5Cvarpi%7D%5E%7B%5Cvarpi%7D%20xf(x)dx%3D%5Cint_%7B0%7D%5E%7B%2B%5Cvarpi%7Dx%5Cfrac%7B1%7D%7B%5Ctheta%7De%5E%7B-%5Cfrac%7Bx%7D%7B%5Ctheta%7D%7Ddx%3D-xe%5E%7B-%5Cfrac%7Bx%7D%7B%5Ctheta%7D%7D%5Cvert%5E%7B%2B%7B%5Cvarpi%7D%20%7D_%7B0%7D%2B%5Cint_%7B0%7D%5E%7B%2B%5Cvarpi%7De%5E%7B-%7B%5Cfrac%7Bx%7D%7B%5Ctheta%7D%7D%7Ddx%3D%5Ctheta%20$

D() $D(X)%3DE(X%5E2)-%5BE(X)%5D%5E2%3D%5Cint_%7B0%7D%5E%7B%2B%5Cvarpi%7D%20x%5E2*%5Cfrac%7B1%7D%7B%5Ctheta%7De%5E%7B-%5Cfrac%7Bx%7D%7B%5Ctheta%7D%7Ddx-%5Ctheta%5E2%3D2%5Ctheta%5E2-%5Ctheta%5E2%3D%5Ctheta%5E2$

無記憶性

$P(x%3Es%2Bt%7Cx%3Es)%3DP(x%3Et)$

6 正態(tài)分布

$f(x)%3D%5Cfrac%7B1%7D%7B%5Csqrt%7B2%5Cpi%5Csigma%20%7D%20%7De%5E%7B-%5Cfrac%7B(x-%5Cmu)%5E2%7D%7B2%5Csigma%5E2%20%7D%7D%20%20%2C%20%5Csigma%3E0%2C%20-%E2%88%9E%3Cx%3C%2B%E2%88%9E$

$E(x)%3D%5Cint_%7B-%E2%88%9E%7D%5E%7B%2B%E2%88%9E%7D%20xf(x)dx%0A%3D%5Cint_%7B-%E2%88%9E%7D%5E%7B%2B%E2%88%9E%7Dx%5Cfrac%7B1%7D%7B%5Csqrt%7B2%5Cpi%5Csigma%7D%7De%5E%7B%5Cfrac%7B(x-%5Cmu)%5E2%7D%7B2%5Csigma%5E2%7D%7D%20dx$

$%5Cfrac%7Bx-%5Cmu%7D%7B%5Csigma%7D%3Dt---%3Ex%3D%5Cmu%2B%5Csigma*t$

$E(X)%3D%5Cfrac%7B1%7D%7B%5Csqrt%7B2%5Cpi%7D%7D%5Cint%5E%7B%2B%E2%88%9E%7D_%7B-%E2%88%9E%7D(%5Cmu%2B%5Csigma*t)e%5E%7B-%5Cfrac%7Bt%5E2%7D%7B2%7D%7Ddt%3D%5Cmu$

$D(X)%3D%5Cint%5E%7B%2B%E2%88%9E%7D_%7B-%E2%88%9E%7D(x-%5Cmu)%5E2f(x)dx%3D%5Cint%5E%7B%2B%E2%88%9E%7D_%7B-%E2%88%9E%7D(x-%5Cmu)%5E2*%5Cfrac%7B1%7D%7B%5Csqrt%7B2%5Cpi%5Csigma%7D%7De%5E-%5Cfrac%7B(x-%5Cmu)%5E2%7D%7B2%5Csigma%5E2%7Ddx$

$%5Cfrac%7Bx-%5Cmu%7D%7B%5Csigma%7D%3Dt---%3Ex%3D%5Cmu%2B%5Csigma*t%0A$

$D(X)%3D%5Cfrac%7B%5Csigma%5E2%7D%7B%5Csqrt%7B2%5Cpi%7D%7D%5Cint%5E%7B%2B%E2%88%9E%7D_%7B-%E2%88%9E%7Dt%5E2e%5E%7B%5Cfrac%7Bt%5E2%7D%7B2%7D%7Ddt%3D0%2B%5Cfrac%7B%5Csigma%5E2%7D%7B%5Csqrt%7B2%5Cpi%7D%7D*%5Csqrt%7B2%5Cpi%7D%3D%5Csigma%5E2$