AI Cluster Cost Breakdown: CapEx (2026)
Raising debt for a GPU cluster up to $200M? Get GPU financing
An AI cluster's CapEx (capital expenditure) is defined by its Bill of Materials (BOM), the complete list of hardware needed to build it. GPU servers account for about three quarters of total cost. Networking runs 10-25%. Storage, power distribution, cabling, and management infrastructure fill the rest. Based on current Blackwell-generation BOMs we've reviewed in 2025-2026, a 16-GPU cluster costs roughly $1.3M, a 576-GPU B300 deployment runs about $48M, and a 24,576-GPU hyperscale cluster would cost roughly $1.5B.
What is a BOM
A BOM (rhymes with "Tom") is the full parts list for a system: every component, its quantity, and its unit price. It might be 10 line items or 200. Some integrators bundle everything into one number: "$48M for a 576-GPU cluster, delivered and racked." Others break it down to individual cable lengths and optical modules.
If you can't read the BOM, you can't tell whether the storage is oversized for your workload, whether the network design matches what you need, or whether per-unit pricing is competitive. A detailed BOM also lets you compare quotes from competing vendors. Our two-page guide to reading a cluster BOM covers the quote formats and the lines that hide equipment value.
What's in an AI cluster's BOM
An AI cluster's BOM breaks down into seven categories. Proportions shift with scale, GPU generation, and workload, but the categories stay consistent. The pricing and examples below are based on B200 BOMs unless noted otherwise. We also reference Meta's prior-generation H100 cluster as a hyperscale case study.
| Category | Typical % of CapEx | What it covers |
|---|---|---|
| GPU servers | 70-80% | GPU boards, CPUs, RAM, fast storage drives, power supplies, chassis, cooling |
| GPU networking | 10-25% | InfiniBand or Ethernet switches, network adapters, optical transceivers, fiber cabling |
| CPU/management nodes | 2-5% | Servers that run the scheduler, monitoring, and network management |
| Storage servers | 0-15% | Dedicated storage nodes for datasets and checkpoints (workload-dependent) |
| Infrastructure | 2-5% | Racks, power distribution units (PDUs), firewalls, structured cabling |
| Cabling and optics | 3-8% | Fiber runs, copper cables, optical modules that connect cables to switches |
| Software and services | 1-3% | Network management licenses, integration, warranty |
| Scale | GPUs | GPU servers | Networking | Storage | Everything else | Total | Per GPU |
|---|---|---|---|---|---|---|---|
| Single server | 8 | $466K | — | — | — | $466K | $58K |
| Small cluster | 16 | $932K | $195K | $75K | $55K | $1.3M | $79K |
| Mid-size cluster | 576 | $33.6M | $6.0M | $1.8M | $2.9M | $44.3M | $77K |
| Hyperscale (est.) | 24,576 | $1.1B | $221.2M | — | $98.3M | $1.5B | $60K |
In Blackwell deployments we've reviewed, GPU servers take roughly three quarters of total spend. [2]Based on our review of a 72-node (576-GPU) B300 cluster BOM (October 2026) Meta's prior-generation H100 cluster, built on cheaper GPU boards, came in at 65.8%. [1]Pytorch to Atoms, "Meta's 24k H100 Cluster Capex/TCO and BoM Analysis" (May 2024)https://pytorchtoatoms.substack.com/p/metas-24k-h100-cluster-capextco-and
Networking ranks as the second biggest cost, and its share of total CapEx grows with scale. High-speed interconnects like InfiniBand and RoCE (RDMA over Converged Ethernet) connect GPUs across servers into a shared fabric so they can work together on the same training job. A 16-GPU cluster needs a few switches. NDR (400G) InfiniBand switches like the QM9700 run ~$30-35K each as of early 2026. [3]Based on our review of real BOMs and industry pricing (2025-2026) A 576-GPU cluster needs a dozen or more. A 24,576-GPU cluster needs 1,920. Each switch also needs optical transceivers, the small modules that plug into switches and convert signals for fiber cables, at ~$1-1.5K each on both ends. At Meta's scale, transceivers alone cost $88M. [1]Pytorch to Atoms, "Meta's 24k H100 Cluster Capex/TCO and BoM Analysis" (May 2024)https://pytorchtoatoms.substack.com/p/metas-24k-h100-cluster-capextco-and
Networking earns its spend because it's the cheapest way to make GPUs faster. A 5% training speed-up from better networking gear costs far less than buying enough additional GPUs to get the same improvement, and those extra GPUs also need rack space, cooling, and power. Underinvesting in networking means your GPUs spend more time waiting for data than computing.
Storage varies the most of any category. A training cluster with large datasets might dedicate 15% of CapEx to dedicated storage nodes. Pricing varies widely based on capacity, CPU configuration, and NVMe drive count, but a typical all-NVMe storage server (50-200 TB raw) currently runs in the ~$35-40K range. [3]Based on our review of real BOMs and industry pricing (2025-2026) An inference cluster might use zero beyond the NVMe drives, a type of fast solid-state storage, already built into each GPU server.
Inside a GPU server
Each GPU server contains 8 GPUs on an HGX baseboard, NVIDIA's standard board that packages 8 GPUs with high-speed links between them. [5]NVIDIA, "HGX Platform" (accessed March 2026)https://www.nvidia.com/en-us/data-center/hgx/ It also has CPUs, system RAM, fast storage, network adapters, power supplies, and cooling. The GPUs do the training math. Everything else exists to keep data flowing into them fast enough that they're not sitting idle.
Inside a typical 8-GPU server at $460-575K (B200/B300 generation, in 2026 orders): [3]Based on our review of real BOMs and industry pricing (2025-2026)
| Component | What it does | Typical cost |
|---|---|---|
| HGX GPU board (8 GPUs) | The compute engine. 8 GPUs with high-speed direct links between them so they can share data without going through the network. | $200K-$300K+ |
| CPUs (2x) | Manage data flow between storage and GPUs, run the operating system. | $3K-$15K each |
| RAM (1-3 TB DDR5) | System memory that holds data in queue before it reaches the GPUs. | $5K-$10K |
| NVMe drives (2-10 TB) | Fast solid-state drives for the operating system, training checkpoints, and working datasets. | $3K-$15K |
| Network adapters (8x) | One network adapter per GPU, connecting the server to the rest of the cluster at 400Gbps (NDR) to 800Gbps (XDR) depending on generation. B300 systems ship with XDR (800G). | $1K-$1.5K each |
| BlueField-3 DPUs (1-2x) | Data processing units that offload networking, storage, and security tasks from the CPUs. DGX systems include 2; OEM HGX builds vary. | ~$2-6K each |
| Power supplies, chassis, cooling, rails | Redundant power supplies, server enclosure, fans or liquid cooling, rack-mount hardware. | $3K-$8K combined |
| Category | Typical % of CapEx |
|---|---|
| GPU servers | 76% |
| HGX GPU board | 69% |
| OEM integration | 18% |
| CPUs (2x) | 4% |
| Network adapters (8x) | 3% |
| NVMe storage | 3% |
| RAM | 2% |
| Power & chassis | 2% |
| DPU | 1% |
| Networking | 14% |
| InfiniBand switches | 38% |
| Optical transceivers | 28% |
| Fiber & cabling | 15% |
| Ethernet switches | 10% |
| Network management | 5% |
| Network adapters | 4% |
| Everything else | 6% |
| CPU/mgmt nodes | 30% |
| Racks & enclosures | 20% |
| PDUs | 15% |
| Firewall | 15% |
| Cabling | 10% |
| Software & licenses | 10% |
| Storage | 4% |
| NVMe drives | 65% |
| Server hardware | 30% |
| Software | 5% |
The rest of the BOM
Beyond the GPU servers, every cluster needs management nodes, networking equipment, and physical infrastructure. None of these categories are individually dominant, but they add up.
Management nodes (~$15K each) run the job scheduler that assigns work to GPUs, manage the InfiniBand network, and handle monitoring and logging. [3]Based on our review of real BOMs and industry pricing (2025-2026)
InfiniBand is the high-speed network that connects GPU servers into a cluster. Without it, each server works alone. The cost comes from switches (NDR-gen QM9700s run ~$30-35K each as of early 2026), a network adapter in each server for every GPU, optical transceivers on both ends, and the fiber connecting them. Some clusters use RDMA over Converged Ethernet (RoCE) instead of InfiniBand, trading some performance for lower cost and more familiar networking hardware. [3]Based on our review of real BOMs and industry pricing (2025-2026)
Infrastructure covers racks (~$2-3K each), PDUs, the power distribution units that feed electricity to each rack (~$1K each, 2-3 per rack for redundancy), firewalls (~$20-30K), and structured cabling. A 576-GPU cluster uses roughly 40 racks and around 80 PDUs. [2]Based on our review of a 72-node (576-GPU) B300 cluster BOM (October 2026) None of these items are individually expensive, but they have long lead times and are easy to forget during budgeting.
Real BOMs at three scales
A single 8-GPU B200 server is a $460-495K purchase. Plug it into facility power and Ethernet, and it works, with no network management layer and no InfiniBand.
Adding a second server doubles GPU count but introduces networking. Two servers need switches, transceivers, and fiber: $35-50K in hardware that wasn't part of the single-server BOM. The bigger the cluster, the more layers of switches and cabling you need to connect everything, so networking takes a larger share of CapEx at scale.
| Cluster | GPU servers | Networking | Storage | Everything else | Total |
|---|---|---|---|---|---|
| 16 GPUs(B200) | $932K (74%) | $195K (16%) | $75K (6%) | $55K (4%) | $1.3M |
| 576 GPUs(B200) | $33.6M (76%) | $6.0M (14%) | $1.8M (4%) | $2.9M (6%) | $44.3M |
| 24,576 GPUs(B200 (est.)) | $1.1B (78%) | $221.0M (15%) | — | $98.0M (7%) | $1.5B |
| Cluster size | GPUs | Approximate CapEx | CapEx per GPU | Networking % |
|---|---|---|---|---|
| Single server (B200) | 8 | $460-495K | ~$57-62K | ~0% |
| Small cluster (B200) | 16 | ~$1.3M | ~$79K | ~16% |
| Mid-size cluster (B200) | 576 | ~$44M | ~$77K | ~14% |
| Hyperscale (B200 est.) | 24,576 | ~$1.5B | ~$60K | ~15% |
Per-GPU costs drop at hyperscale. At 24,576 GPUs, buyers can negotiate directly with component manufacturers and use an ODM (original design manufacturer) like Quanta instead of an OEM like Dell or SuperMicro, cutting per-server costs by roughly 15-20%. [1]Pytorch to Atoms, "Meta's 24k H100 Cluster Capex/TCO and BoM Analysis" (May 2024)https://pytorchtoatoms.substack.com/p/metas-24k-h100-cluster-capextco-and All rows above use B200 pricing. For reference, Meta's actual 24,576-GPU H100 cluster cost $910M ($37K per GPU), but that used the prior-generation H100 with a cheaper GPU baseboard ($195K vs. ~$280-300K for B200 at OEM pricing). The detailed H100 breakdown appears below. [1]Pytorch to Atoms, "Meta's 24k H100 Cluster Capex/TCO and BoM Analysis" (May 2024)https://pytorchtoatoms.substack.com/p/metas-24k-h100-cluster-capextco-and
16-GPU B200 cluster (~$1.3M)
A typical small cluster, based on our review of real BOMs: 2 GPU servers (16 B200 GPUs total), a few CPU management nodes, a couple of storage nodes, InfiniBand switches, and a firewall. [3]Based on our review of real BOMs and industry pricing (2025-2026)
| Category | Items | Cost | % |
|---|---|---|---|
| GPU servers | 2x 8-GPU servers | ~$930K | ~74% |
| CPU/management | Management and scheduling nodes | ~$55K | ~4% |
| Storage | Dedicated storage nodes | ~$75K | ~6% |
| InfiniBand | Switches + network management licenses | ~$110K | ~9% |
| Ethernet, firewall, cabling | Switches, firewall, transceivers, fiber | ~$85K | ~7% |
| Total | ~$1.25M |
Storage is the most discretionary category here. Some clusters need dedicated storage nodes for large datasets; others rely entirely on the NVMe drives already inside each GPU server.
576-GPU B300 cluster (~$48M)
This section details one B300 deployment. B300 servers ran $490-575K in 2026 orders against $460-495K for B200, roughly $60-80K more per server at the median, due to the higher-end GPU board. The proportional BOM breakdown is similar. For a single server bought, hosted and run for you, see the B300 server page.
A mid-scale deployment from a BOM we reviewed in October 2026. [2]Based on our review of a 72-node (576-GPU) B300 cluster BOM (October 2026) 72 GPU servers (576 GPUs total, 8 per server) with XDR InfiniBand and Ethernet networking, management servers, and a storage cluster.
- 72 GPU servers with InfiniBand networking per server
- ~12 XDR InfiniBand switches (Q-3400, 144 ports each), ~18 Ethernet switches
- ~1,500 fiber runs and about 2,100 transceivers
- Racks, PDUs, integration, and deployment priced separately
The servers come to $40.9M (about $568K each), the network to $6.2M, and storage to $0.6M: about $47.7M, or $83,000 per GPU, before racks, power distribution, and deployment.
Deployment adds more cash up front. Installing, racking, cabling and burning in runs about $6K a server. Colocation setup is about $3,500 a server, plus a deposit of two months of colocation. Closing and legal for a loan run about $75K. For these 72 servers that is about $1.4M more, before sales tax, which runs 0-8.6% of the hardware depending on the state.
Meta's 24,576-GPU H100 cluster ($910M)
The hyperscale extreme, estimated by Pytorch to Atoms (May 2024). [1]Pytorch to Atoms, "Meta's 24k H100 Cluster Capex/TCO and BoM Analysis" (May 2024)https://pytorchtoatoms.substack.com/p/metas-24k-h100-cluster-capextco-and Meta partnered directly with Quanta to design custom H100 server hardware, bypassing OEM markups entirely.
| Component | Qty | Unit price | Total | % |
|---|---|---|---|---|
| GPU boards (8x H100 each) | 3,072 | $195,000 | $599,040,000 | 65.8% |
| InfiniBand switches (QM9700) | 1,920 | $35,000 | $67,200,000 | 7.4% |
| Optical transceivers | 73,728 | $1,000-$1,300 | $88,474,000 | 9.7% |
| InfiniBand network adapters | 24,576 | $1,200 | $29,491,000 | 3.2% |
| DDR5 RAM | 3,072 | $7,860 | $24,146,000 | 2.7% |
| Intel Xeon CPUs | 6,144 | $2,600 | $15,974,000 | 1.8% |
| Everything else | $86,041,000 | 9.4% | ||
| Total | $910,366,000 |
DPUs, storage drives, Ethernet, fiber, chassis, cooling, power supplies, racks, power distribution, and contract manufacturer markup grouped as "Everything else." Source: Pytorch to Atoms estimates (May 2024).
NVIDIA components account for most of this BOM. GPU boards, InfiniBand switches, and network adapters alone total $696M (76.4% of CapEx). The $88M in optical transceivers flows to a mix of NVIDIA and third-party vendors (InnoLight, Coherent, and others). NVIDIA dominates cost across both compute and networking.
References
- Pytorch to Atoms, "Meta's 24k H100 Cluster Capex/TCO and BoM Analysis" (May 2024)
- Based on our review of a 72-node (576-GPU) B300 cluster BOM (October 2026)
- Based on our review of real BOMs and industry pricing (2025-2026)
- NVIDIA, "QM9700 InfiniBand Switch — Specifications" (accessed March 2026)
- NVIDIA, "HGX Platform" (accessed March 2026)
Get your project seen by our 50+ partner lenders. Clusters up to $200M, global.
Get GPU financingLarger, or outside the U.S.? Get in touch
Frequently Asked Questions
What is a BOM for AI cluster CapEx?
An AI cluster's CapEx (capital expenditure) is defined by its Bill of Materials (BOM), the complete list of hardware needed to build it. A BOM is the full parts list for a system: every component, its quantity, and its unit price.
How much of AI cluster CapEx is GPUs versus networking?
GPU servers account for about three quarters of total cost, and networking runs 10-25%. Meta's prior-generation H100 cluster, built on cheaper GPU boards, came in at 65.8%.
How much does a 16-GPU, 576-GPU, or 24,576-GPU cluster cost?
Based on Blackwell-generation BOMs reviewed in 2025-2026, a 16-GPU cluster costs roughly $1.3M, a 576-GPU B300 deployment runs about $48M before racks and deployment, and a 24,576-GPU hyperscale cluster costs roughly $1.5B. For reference, Meta's actual 24,576-GPU H100 cluster cost $910M ($37K per GPU).
Why invest in networking for GPU training clusters?
Networking is worth the spend because it's the cheapest way to make GPUs faster. A 5% training speed-up from better networking gear costs far less than buying enough additional GPUs to get the same improvement. Underinvesting in networking means your GPUs spend more time waiting for data than computing.
Get the GPU Modeling + Loan vs Lease
An editable spreadsheet comparing a GPU loan and a full-payout lease, each with and without residual value insurance: cash at closing, payments, total cost to own and coverage. XLSX, free by email.