Choosing GPUs for LLM inference vs training
Inference is a memory problem, training is a bandwidth-and-interconnect problem. How to size VRAM, read precision figures and decide between PCIe cards and NVLink systems.
Insights
Buying guidesand field notes.
Comparisons and infrastructure notes from the engineers who spec, source and commission AI hardware. No press releases.
H200 NVL vs RTX PRO 6000 vs L40S: where each wins
Three PCIe cards, three price bands. A side-by-side of memory, bandwidth, power and cooling, and the workloads where each is the right answer rather than the expensive one.
Memory sizing for AI servers: DDR5 RDIMM rules of thumb
How many channels to populate, how much system RAM per GB of VRAM, and which speed bins and ranks to specify so the memory keeps up with the GPUs.
Networking for GPU clusters: InfiniBand vs Ethernet
NDR InfiniBand or 400/800G RoCE Ethernet, rail-optimized fabrics, and the optics and cables that connect them. When Ethernet is enough, and when it is not.
Building a hosting-grade AI data platform
Power per rack, cooling choices, server specifications, storage and the project stages from capacity plan to commissioning. What hosting providers should decide before the first purchase order.
DGX Spark and workstation-class AI: what fits on a desk
DGX Spark, RTX PRO Blackwell workstation cards and GeForce RTX 5090/5080 for development and local inference. What each runs comfortably, and where the desk stops being enough.
How to read a GPU spec sheet
Memory, bandwidth, TFLOPS by precision, TDP, form factor, PCIe generation, MIG, NVLink, licences and warranty. A field guide to the numbers on a data center GPU datasheet and the ones that are missing.
