{"ops":[{"insert":"了解 Whistle 的工作原理以及如何實(shí)現(xiàn)一個(gè)簡(jiǎn)單的數(shù)據(jù)包捕獲調(diào)試工具。\n\nGithub地址：https://github.com/avwo/whistle\n\nWhistle 是一個(gè)基於 Node.js 的跨平臺(tái) Web 數(shù)據(jù)包捕獲和調(diào)試 (HTTP) 代理。\n\n實(shí)時(shí)抓包：支持HTTP、HTTPS、HTTP2、WebSocket、TCP等常見(jiàn)Web請(qǐng)求的抓包。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"修改請(qǐng)求響應(yīng)：與常見(jiàn)的抓包調(diào)試工具使用的斷點(diǎn)不同，Whistle 使用類(lèi)似於系統(tǒng)主機(jī)的配置規(guī)則方法。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"擴(kuò)展：支持通過(guò)Node編寫(xiě)插件，或者作為獨(dú)立的npm包引入項(xiàng)目?jī)煞N擴(kuò)展方式。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/2a86d79e266160ac36149b9d5d3c5cfcde827245.jpg","width":1080,"height":785,"size":288362,"status":"loaded"}}},{"insert":"\n本文從基礎(chǔ)開(kāi)始逐步引導(dǎo)您完成 Whistle 功能：\n\n什麼是 HTTP 代理"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"實(shí)現(xiàn)一個(gè)簡(jiǎn)單的 HTTP 代理"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"完整的 HTTP 代理架構(gòu)（Whistle）"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"具體實(shí)現(xiàn)原理"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"資源"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n1.什麼是HTTP代理"},{"attributes":{"header":1},"insert":"\n"},{"insert":"\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/ac15448058ace757f5714cc54d546821b09c1dae.jpg","width":999,"height":460,"size":71683,"status":"loaded"}}},{"insert":"\n代理是客戶(hù)端到服務(wù)器的傳輸服務(wù)，其中：\n\n免代理：客戶(hù)端與服務(wù)器建立直連後，即可進(jìn)行數(shù)據(jù)交換。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"代理請(qǐng)求：客戶(hù)端不直接與服務(wù)器建立連接。相反，客戶(hù)端與代理建立連接後，將目標(biāo)服務(wù)器的地址發(fā)送給代理，然後通過(guò)代理與服務(wù)器建立連接。如果代理服務(wù)是 HTTP Server，則稱(chēng)為 HTTP 代理。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n讓我們看看客戶(hù)端如何將目標(biāo)服務(wù)器地址傳遞給 HTTP 代理，以及 HTTP 代理如何建立與目標(biāo)服務(wù)器的連接。\n\n2.實(shí)現(xiàn)簡(jiǎn)單的HTTP代理"},{"attributes":{"header":1},"insert":"\n"},{"insert":"讓我們從 Node.js 中實(shí)現(xiàn)的最簡(jiǎn)單的 HTTP 代理開(kāi)始：\n\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/b56aa85d4fbf7b0a6d259f658d16f1474ed78e52.jpg","width":1820,"height":1192,"size":673936,"status":"loaded"}}},{"insert":"\n上述代碼實(shí)現(xiàn)了一個(gè)具有轉(zhuǎn)發(fā)請(qǐng)求功能的HTTP代理。從代碼中可以看出，HTTP代理就是一個(gè)普通的HTTP Server，監(jiān)聽(tīng)request和connect兩個(gè)事件。客戶(hù)端會(huì)通過(guò)這兩個(gè)事件傳遞目標(biāo)服務(wù)器的地址。\n\n"},{"attributes":{"color":"#ff6699"},"insert":"request"},{"insert":"：普通HTTP通過(guò)這個(gè)事件傳遞目標(biāo)服務(wù)器的地址。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"attributes":{"color":"#ff6699"},"insert":"connect"},{"insert":"：非HTTP請(qǐng)求，如HTTPS、HTTP/2、WebSocket、TCP等，使用該事件發(fā)送目標(biāo)服務(wù)器的地址。觸發(fā)此事件的代理請(qǐng)求也稱(chēng)為隧道代理。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n請(qǐng)求。url 或 req.headers。主機(jī)可以在事件中獲取目標(biāo)服務(wù)器的地址（host:port），與服務(wù)器的地址建立連接，並將結(jié)果以HTTP響應(yīng)的方式回傳給客戶(hù)端。除了請(qǐng)求轉(zhuǎn)發(fā)，完整的 HTTP 至少應(yīng)該有：\n\n查看實(shí)時(shí)抓包；"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"解析 HTTPS 請(qǐng)求；"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"修改請(qǐng)求響應(yīng)內(nèi)容。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"擴(kuò)展功能。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n以 Whistle 為例，我們將了解如何使用 Node.js 實(shí)現(xiàn)一個(gè)完整的 HTTP 代理。\n\n3.完整的HTTP代理架構(gòu)（Whistle）"},{"attributes":{"header":1},"insert":"\n"},{"insert":"\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/60320844464b57e00b394ddc44cc5eaee26cdfa7.jpg","width":1080,"height":543,"size":203256,"status":"loaded"}}},{"insert":"\n主要分為五個(gè)模塊：\n\n請(qǐng)求訪問(wèn)模塊"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"隧道代理模塊"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"處理HTTP請(qǐng)求模塊"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"規(guī)則管理模塊"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"插件管理模塊"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n4.具體實(shí)施原則"},{"attributes":{"header":1},"insert":"\n"},{"insert":"下面我們來(lái)看看這五個(gè)模塊是如何實(shí)現(xiàn)的。\n\n4.1 請(qǐng)求訪問(wèn)模塊"},{"attributes":{"header":2},"insert":"\n"},{"insert":"\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/bb4630d53ad3ac90700c3e75b71baa6cb592a86a.jpg","width":1080,"height":496,"size":70978,"status":"loaded"}}},{"insert":"\n所有請(qǐng)求首先通過(guò)請(qǐng)求訪問(wèn)模塊。Whistle 支持四種請(qǐng)求訪問(wèn)模式：\n\nHTTP HTTPS 直接請(qǐng)求：通過(guò)配置主機(jī)或 DNS 將請(qǐng)求轉(zhuǎn)發(fā)給 Whistle。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"HTTP 代理：默認(rèn) Whistle 訪問(wèn)方式通過(guò)瀏覽器插件配置系統(tǒng)代理或 HTTP 代理。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"HTTPS 代理：對(duì) HTTP 代理（即 HTTPS Server）上的代理請(qǐng)求進(jìn)行加密。您可以使用指定的證書(shū)將請(qǐng)求傳輸?shù)?HTTP 代理。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"Socks5 代理：使用 NPM 包"},{"attributes":{"color":"#ff6699"},"insert":"socksv5"},{"insert":"將 TCP 請(qǐng)求轉(zhuǎn)換為普通的 TCP 請(qǐng)求，將 TCP 請(qǐng)求轉(zhuǎn)換為隧道代理請(qǐng)求。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n基本實(shí)現(xiàn)原理是：將所有請(qǐng)求轉(zhuǎn)換為T(mén)UNNEL代理請(qǐng)求或HTTP代理的HTTP請(qǐng)求，再將隧道代理請(qǐng)求解析為HTTP請(qǐng)求。\n\n如何將普通的TCP請(qǐng)求轉(zhuǎn)化為隧道代理請(qǐng)求？有關(guān)詳細(xì)信息，請(qǐng)參閱黑色代理\n\n讓我們看看如何從隧道代理請(qǐng)求中解析 HTTP 請(qǐng)求。\n\n4.2 隧道代理模塊"},{"attributes":{"header":2},"insert":"\n"},{"insert":"\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/e03667f297439e613edb6d1585cd329f05eb50cc.jpg","width":1080,"height":375,"size":72962,"status":"loaded"}}},{"insert":"\n關(guān)鍵點(diǎn)（HTTP請(qǐng)求也可以通過(guò)隧道代理）：\n\n1. 匹配的全局規(guī)則決定是否解析隧道代理請(qǐng)求。如果否，則隧道代理請(qǐng)求被視為普通 TCP 請(qǐng)求。\n\n2. Socket.once('data', handler) 必要時(shí)讀取請(qǐng)求點(diǎn)的第一幀；\n\n3. 將第一幀數(shù)據(jù)轉(zhuǎn)換成字符串，通過(guò)正則/^(\\w+)\\s+(\\s+)HTTP\\\\s+/1\\d$/mi是HTTP請(qǐng)求嗎？如果是HTTP請(qǐng)求，檢查是否是CONNECT請(qǐng)求，即TUNNEL代理請(qǐng)求（隧道代理請(qǐng)求也可以是隧道代理請(qǐng)求）。如果是，則將請(qǐng)求轉(zhuǎn)發(fā)給隧道代理方法進(jìn)行處理。如果否，則將請(qǐng)求轉(zhuǎn)發(fā)給HTTP請(qǐng)求模塊進(jìn)行處理。\n\n4. 如果不是 HTTP 請(qǐng)求，則視為 HTTPS 請(qǐng)求。在這種情況下，使用中間人將 HTTPS 請(qǐng)求轉(zhuǎn)換為 HTTP 請(qǐng)求。\n\n5. Whistle 首先按以下順序獲取請(qǐng)求的證書(shū)：\n\n通過(guò)匹配插件（可以通過(guò)規(guī)則"},{"attributes":{"color":"#ff6699"},"insert":"sniCallback://plugin"},{"insert":"指定插件加載證書(shū)）；"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"通過(guò)引導(dǎo)參數(shù)"},{"attributes":{"color":"#ff6699"},"insert":"-z certDir"},{"insert":"指定目錄或"},{"attributes":{"color":"#ff6699"},"insert":"~/.WhistleAppData/custom_certs"},{"insert":"加載自定義證書(shū)；"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"如果您沒(méi)有這些自動(dòng)證書(shū)，Whistle 會(huì)自動(dòng)生成一個(gè)默認(rèn)證書(shū)。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n6. 獲取證書(shū)後，使用該證書(shū)啟動(dòng)HTTPS Server，將HTTPS請(qǐng)求轉(zhuǎn)換為HTTP請(qǐng)求，提交給HTTP請(qǐng)求模塊處理。\n\n4.3 HTTP請(qǐng)求處理模塊"},{"attributes":{"header":2},"insert":"\n"},{"insert":"\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/02cfb1bfb696e704a3cba1037c571b576aebc0b4.jpg","width":1002,"height":310,"size":75317,"status":"loaded"}}},{"insert":"\nHTTP請(qǐng)求處理可以分為兩個(gè)階段：\n\n1. 請(qǐng)求階段：\n\n匹配全局規(guī)則；"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"如果有類(lèi)似口哨的規(guī)則。http://whistle.xxx，執(zhí)行對(duì)應(yīng)的插件鉤子，檢索插件規(guī)則，並與匹配的全局規(guī)則合併。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"執(zhí)行規(guī)則，記錄狀態(tài)，並向指定服務(wù)發(fā)出請(qǐng)求。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n2. 反應(yīng)階段：\n\n執(zhí)行匹配插件鉤子獲取插件規(guī)則並將其與匹配的全局規(guī)則合併。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"執(zhí)行規(guī)則，記錄狀態(tài)，並請(qǐng)求返回給客戶(hù)端。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n4.4 規(guī)則管理"},{"attributes":{"header":2},"insert":"\n"},{"insert":"與使用斷點(diǎn)修改請(qǐng)求響應(yīng)數(shù)據(jù)的傳統(tǒng)數(shù)據(jù)包捕獲調(diào)試代理不同，Whistle 使用配置規(guī)則修改請(qǐng)求響應(yīng)數(shù)據(jù)。這種方法的優(yōu)點(diǎn)是操作簡(jiǎn)單、持久存儲(chǔ)和操作共享。讓我們看一些例子：\n\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/3f911b0eaebbabbdc14eac008c28ddeb30fc9931.jpg","width":1080,"height":278,"size":136381,"status":"loaded"}}},{"insert":"\n"},{"attributes":{"bold":true},"insert":"Whistle 的規(guī)則管理有兩個(gè)主要功能："},{"insert":"\n\n解析規(guī)則"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"匹配規(guī)則"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n解析規(guī)則"},{"attributes":{"header":1},"insert":"\n"},{"attributes":{"bold":true},"insert":"Whistle 有兩種規(guī)則："},{"insert":"\n\n1. 全局規(guī)則（通用規(guī)則），所有請(qǐng)求都會(huì)嘗試匹配的規(guī)則，由以下規(guī)則組成：\n\n規(guī)則 表示配置的規(guī)則。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n插件根目錄rules.txt配置文件；"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n 文件：https://github.com/whistle-plugins/whistle.autosave/blob/master/rules.txt\n\n接口或插件rules.txt通過(guò)@URL引入的遠(yuǎn)程規(guī)則（單行，Whistle定期更新遠(yuǎn)程規(guī)則）。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/d46e9a1be0c1d85642d660455a87e259b2eacd6a.jpg","width":708,"height":496,"size":125190,"status":"loaded"}}},{"insert":"\n2. 插件規(guī)則（私有規(guī)則）是只有在全局規(guī)則中匹配 whistle.xxx 協(xié)議的插件請(qǐng)求才會(huì)匹配的規(guī)則。它們由以下規(guī)則組成：\n\nreqRulesServer 等插件動(dòng)態(tài)返回鉤子；"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n插件根目錄_rules.txt中配置的靜態(tài)規(guī)則。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":" 文件：https://wproxy.org/whistle/plugins.html\n\n匹配規(guī)則"},{"attributes":{"header":1},"insert":"\n"},{"insert":"Whistle 規(guī)則的完整結(jié)構(gòu)是：\n\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/f8d4db5451b3993765d04b4e89376f6a5948d021.jpg","width":998,"height":402,"size":106309,"status":"loaded"}}},{"insert":"\n文件：https://wproxy.org/whistle/mode.html\n"},{"attributes":{"header":2},"insert":"\n"},{"insert":"4.5 插件管理"},{"attributes":{"header":2},"insert":"\n"},{"insert":"\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/bcb6c5ffd9e5ecb1d0984da15e02fae19b9b29e5.jpg","width":1080,"height":194,"size":87264,"status":"loaded"}}},{"insert":"\nWhistle 插件有很多功能。它不僅擁有 Node 的所有功能，還可以運(yùn)行 Whistle 的所有規(guī)則。它用於執(zhí)行以下操作：\n\n認(rèn)證功能"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"提供UI界面"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"充當(dāng)請(qǐng)求服務(wù)器（直接響應(yīng)或轉(zhuǎn)發(fā)和修改請(qǐng)求響應(yīng)）"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"統(tǒng)計(jì)請(qǐng)求信息（查看報(bào)告/日誌數(shù)據(jù)等）"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"設(shè)置規(guī)則（動(dòng)態(tài)、靜態(tài)、全局和私有規(guī)則）"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"獲取抓包數(shù)據(jù)"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"編解碼器請(qǐng)求響應(yīng)數(shù)據(jù)流（管道流功能）"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"擴(kuò)展界面右鍵菜單（如分享抓包數(shù)據(jù)）"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"保存和同步規(guī)則值數(shù)據(jù)"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"自定義 HTTPS 請(qǐng)求證書(shū)"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n如：\n\nWhistle. script：通過(guò)自定義腳本實(shí)現(xiàn)規(guī)則的動(dòng)態(tài)設(shè)置"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"Whistle. vase：提供靈活強(qiáng)大的模擬能力"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"Whistle. inspect：輕鬆快速注入vConsole、Eruda等頁(yè)面調(diào)試工具"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"Whistle. sni-callback：自定義證書(shū)插件"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n其他插件示例見(jiàn)：https://github.com/whistle-plugins\n\nWhistle 如何作為插件發(fā)揮作用？遵循以下三個(gè)設(shè)計(jì)原則：\n\n1. "},{"attributes":{"bold":true},"insert":"完整性"},{"insert":"：\n\n確保所有功能點(diǎn)都是可擴(kuò)展的，例如請(qǐng)求認(rèn)證、證書(shū)生成、抓包、規(guī)則設(shè)置和請(qǐng)求處理。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n2. "},{"attributes":{"bold":true},"insert":"穩(wěn)定"},{"insert":"：\n\n內(nèi)部異常不會(huì)影響其他功能。每個(gè) Whistle 插件都是一個(gè)獨(dú)立的進(jìn)程，插件通過(guò) HTTP 與 Whistle 交互。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\nWhistle 使用 NPM 包 "},{"attributes":{"color":"#ff6699"},"insert":"pfork"},{"insert":" 來(lái)啟動(dòng)插件進(jìn)程。進(jìn)程間的切換直接通過(guò)Node的HTTP模塊實(shí)現(xiàn)。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n3. "},{"attributes":{"bold":true},"insert":"使用方便"},{"insert":"：\n\n方便用戶(hù)開(kāi)發(fā)和使用。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n4. "},{"attributes":{"bold":true},"insert":"開(kāi)發(fā)"},{"insert":"：結(jié)構(gòu)簡(jiǎn)單（NPM包）+腳手架"},{"attributes":{"color":"#ff6699"},"insert":"lack"},{"insert":"\n\n使用： 安裝npm包，使用同內(nèi)置協(xié)議，內(nèi)置交互界面。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n有關(guān)插件的更多詳細(xì)信息，請(qǐng)參閱：https://wproxy.org/whistle/plugins.html\n\n事實(shí)上，Whistle 支持插件擴(kuò)展，也可以在項(xiàng)目中作為獨(dú)立模塊使用；除了本地開(kāi)發(fā)外，Whistle 還可用於開(kāi)發(fā)多人開(kāi)發(fā)和協(xié)調(diào)工具。比如Whistle的實(shí)現(xiàn)原理，後面會(huì)介紹。\n\n1. Whistle基於多人多環(huán)境遠(yuǎn)程抓包調(diào)試工具的實(shí)現(xiàn)\n\nNohost：https://github.com/Tencent/nohost"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n2. TDE是一款基於Whistle和Nohost的分佈式遠(yuǎn)程抓包調(diào)試工具\(yùn)n\nTDE目前僅在騰訊使用，將逐步開(kāi)源。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n5. 參考資料"},{"attributes":{"header":1},"insert":"\n"},{"insert":"Github 存儲(chǔ)庫(kù)：https://github.com/avwo/whistle\n官方插件庫(kù)：https://github.com/whistle-plugins\n詳細(xì)文檔：https://wproxy.org/whistle/\n\n網(wǎng)站地址：https://www.mo4tech.com/implementation-principle-of-whistle.html\n翻譯：阿曜ちゃん\n\n最後，謝謝大家的瀏覽，十分感謝大家的支持。\n"}]}