原文標題：Nvidia H100: Are 550,000 GPUs Enough for This Year?作者：Doug Eadline

August 17, 2023

The GPU Squeeze continues to place a premium on Nvidia H100 GPUs. In a recent?Financial Times?article, Nvidia reports that it expects to ship 550,000 of its latest H100 GPUs worldwide in 2023. The appetite for GPUs is obviously coming from the generative AI boom, but the HPC market is also competing for these accelerators. It is not clear if this number includes the throttled China-specific A800 and H800 models.

在《金融時報》最近的一篇文章中，Nvidia 報告稱，預計 2023 年將在全球范圍內(nèi)出貨 550,000 個最新的 H100 GPU。對 GPU 的需求顯然來自生成式 AI 熱潮，但 HPC 市場也在爭奪這些加速器。 目前尚不清楚這個數(shù)字是否包括中國專用的 A800 和 H800 。

The bulk of the GPUs will be going to US technology companies, but the Financial Times notes that Saudi Arabia has purchased at least 3,000 Nvidia H100 GPUs and the UAE has also purchased thousands of Nvidia chips. UAE has already developed its own open-source large language model using 384 A100 GPUs, called Falcon, at the state-owned Technology Innovation Institute in Masdar City, Abu Dhabi.

大部分 GPU 將流向美國科技公司，但英國《金融時報》指出，沙特阿拉伯已經(jīng)購買了至少 3,000 個 Nvidia H100 GPU，阿聯(lián)酋也購買了數(shù)千個 Nvidia 芯片。 阿聯(lián)酋已經(jīng)在阿布扎比馬斯達爾城的國有技術(shù)創(chuàng)新研究所使用 384 個 A100 GPU 開發(fā)了自己的開源大型語言模型，稱為 Falcon。

The flagship H100 GPU (14,592 CUDA cores, 80GB of HBM3 capacity, 5,120-bit memory bus) is priced at a massive $30,000 (average), which Nvidia CEO Jensen Huang calls the first chip designed for generative AI. The Saudi university is building its own GPU-based supercomputer called Shaheen III. It employs 700 Grace Hopper chips that combine a Grace CPU and an H100 Tensor Core GPU. Interestingly, the GPUs are being used to create an LLM developed by Chinese researchers who can’t study or work in the US.

旗艦級 H100 GPU（14,592 個 CUDA 核心、80GB HBM3 容量、5,120 位內(nèi)存總線）售價高達 30,000 美元（平均），Nvidia 首席執(zhí)行官黃仁勛 (Jensen Huang) 稱其為首款為生成式 AI 設計的芯片。 沙特大學正在構(gòu)建自己的基于 GPU 的超級計算機，名為 Shaheen III。 它采用 700 個 Grace Hopper 芯片，結(jié)合了 Grace CPU 和 H100 Tensor Core GPU。 有趣的是，GPU被用來創(chuàng)建LLM，該LLM由不能在美國學習或工作的中國研究人員開發(fā)。

Meanwhile, generative AI? (GAI) investments continue to fund?GPU infrastructure purchases. As reported, in the first 6 months of 2023, funding to GAI start-ups is up more than 5x compared to full-year 2022 and the generative AI infrastructure category has seen over 70% of the funding since Q3’22.

與此同時，生成式人工智能 (GAI) 投資繼續(xù)為 GPU 基礎(chǔ)設施采購提供資金。 據(jù)報道，2023 年前 6 個月，GAI 初創(chuàng)企業(yè)獲得的資金比 2022 年全年增長了 5 倍以上，自 2022 年第三季度以來，生成式 AI 基礎(chǔ)設施類別已占資金的 70% 以上。

Worth the Wait

The cost of a H100 varies depending on how it is packaged and presumably how many you are able to purchase. The current (Aug-2023) retail price for an H100 PCIe card is around $30,000 (lead times can vary as well.) A back-of-the-envelope estimate gives a market spending of $16.5 billion for 2023 — a big chunk of which will be going to Nvidia. According to estimates made by Barron’s senior writer?Tae Kim?in a?recent social media post?estimates it costs Nvidia? $3,320? to make a H100.? That is a 1000% percent profit based on the retail cost of an Nvidia H100 card.

H100 的成本因包裝方式以及您能夠購買的數(shù)量而異。 目前（2023 年 8 月）H100 PCIe 卡的零售價約為 30,000 美元（交貨時間也可能有所不同。）粗略估計，2023 年的市場支出為 165 億美元——其中很大一部分 將去Nvidia。 根據(jù)《巴倫周刊》資深撰稿人 Tae Kim 最近在社交媒體上發(fā)布的估計，Nvidia 制造 H100 的成本為 3,320 美元，1000% 利潤。

As often reported, Nvidia’s partner TSMC can barely meet the demand for GPUs. The GPUs require a more complex CoWoS manufacturing process (Chip on Wafer on Substrate — a “2.5D” packaging technology from TSMC where multiple active silicon dies, usually GPUs and HBM stacks, are integrated on a passive silicon interposer.) Using CoWoS adds a complex multi-step, high-precision engineering process that slows down the rate of GPU production.?

正如經(jīng)常報道的那樣，Nvidia 的合作伙伴臺積電幾乎無法滿足 GPU 的需求。 GPU 需要更復雜的 CoWoS 制造工藝（基板上晶圓芯片 — 臺積電的“2.5D”封裝技術(shù)，其中多個有源硅芯片（通常是 GPU 和 HBM 堆棧）集成在無源硅中介層上。）使用 CoWoS 會增加復雜的多步驟、高精度工程流程會降低 GPU 的生產(chǎn)速度。

This situation was confirmed by Charlie Boyle, VP and GM of Nvidia’s DGX systems.?Boyle states?that delays are not from miscalculating demand or wafer yield issues from TSMC, but instead from the chip packaging CoWoS technology.??

英偉達DGX系統(tǒng)副總裁兼總經(jīng)理Charlie Boyle證實了這一情況。博伊爾表示，延遲不是因為臺積電的需求計算錯誤或晶圓產(chǎn)量問題，而是因為芯片封裝CoWoS技術(shù)。

原文鏈接：https://www.hpcwire.com/2023/08/17/nvidia-h100-are-550000-gpus-enough-for-this-year/

//你都看到這里了，不如我們嘮叨幾句吧！

1. 有人問“大模型掙錢了嗎？”，我不知道怎么回答，但，nvidia已經(jīng)在摘取低垂的果實了。它的先發(fā)優(yōu)勢來自于十數(shù)年前CUDA軟件棧的布局，和n多年在GPU架構(gòu)方向的積累。

2. 國內(nèi)三十多家加速卡公司，2024年都將卷入高峰時刻，做幾個預測：

• 爭上市的，爭推大模型專用卡的

• 小公司或走得慢的公司明年會很危險，并購不失為退路。

• 算力中心/信創(chuàng)市場/城市布局，份額之爭。

• 2024年，將是算力基礎(chǔ)軟件公司爆發(fā)的高光時刻！

• 美國佬將精確選擇目標，精準選擇打擊時間。好事者/間諜無處不在！