How to read a GPU spec sheet
Memory, bandwidth, TFLOPS by precision, TDP, form factor, PCIe generation, MIG, NVLink, licences and warranty. A field guide to the numbers on a data center GPU datasheet and the ones that are missing.
A GPU datasheet is a page of numbers designed to look comparable across products that are not. This guide goes through each line in the order it should matter for an AI purchase, explains the footnotes that change the meaning, and lists what to ask for that the datasheet does not say.
1. Memory: type, size, bandwidth
Three lines, read together. Type tells you the bandwidth class: HBM3/HBM3e (3.3-8 TB/s) sits on the package; GDDR7 (1-1.8 TB/s) and GDDR6/6X (0.6-1 TB/s) sit on the board. Size sets the largest model that fits. Bandwidth sets decode speed and, for training, how fast activations move. A card with more memory but less bandwidth is not "better"; it is different.
- Check whether ECC is on by default and whether enabling it reduces usable capacity (it does not on HBM or GDDR7 with inline ECC; it did on some GDDR6 cards).
- Usable memory is a little less than nominal: expect ~1-2 GB reserved by driver and firmware.
- "Up to" bandwidth figures assume the top memory clock; server OEMs sometimes ship lower-clocked SKUs.
2. Compute: TFLOPS by precision
Datasheets list peak throughput for FP64, FP32, TF32, BF16/FP16, FP8, INT8 and (on Blackwell) FP4. Two rules: compare at the precision you will actually run, and check the footnote for sparsity. Most headline numbers assume 2:4 structured sparsity, which doubles the figure; dense throughput is half. Real LLM workloads do not use structured sparsity today, so divide the marketing number by two before comparing.
| Line on the sheet | What it is used for | Watch out for |
|---|---|---|
| FP64 / FP64 Tensor | HPC, scientific simulation | Near zero on RTX PRO / GeForce; only matters for HPC |
| TF32 Tensor | Default training precision in PyTorch for FP32 code | Almost always quoted with sparsity |
| BF16 / FP16 Tensor | Mixed-precision training, high-quality inference | Sparse vs dense |
| FP8 Tensor | Inference on Hopper/Ada/Blackwell, selective training matmuls | Requires framework support |
| FP4 / NVFP4 | Blackwell inference | Absent on Hopper/Ada; quality depends on quantization recipe |
| INT8 / INT4 | Older quantized inference, vision models | Being replaced by FP8/FP4 |
| RT cores / rasterization | Rendering, simulation | Irrelevant to LLM work; present on RTX PRO, absent on H-series |
3. Power: TDP and configurable limits
TDP is the sustained board power the cooling system must remove. Many data center cards are "configurable": H200 NVL and RTX PRO 6000 Server Edition ship with a default and a range (for example 400-600 W), and the OEM sets the cap in firmware. Confirm the value on the SKU you are buying, because it changes both the performance and which servers the card is qualified in. Transient spikes can exceed TDP by 20-30% for milliseconds; PSU sizing uses the sustained figure plus headroom.
4. Form factor and cooling
- FHFL (full-height, full-length): 312 mm, dual-slot is the standard for server cards. Check slot count carefully: some workstation cards are triple-slot.
- Passive: no fan, relies on chassis airflow, mandatory in servers. Active: has its own blower or axial fans, for workstations only.
- SXM / OAM: mezzanine modules, not cards; only sold inside HGX or OEM baseboards. Not user-installable.
- Power connector: 12V-2x6 (CEM5) on current cards vs. EPS or 8-pin on older ones; the server must have the right cable.
5. Host interface: PCIe generation and lanes
PCIe 5.0 x16 delivers ~64 GB/s each way; 4.0 x16 about half. For single-card inference it barely matters. For multi-card tensor parallelism without NVLink, and for feeding data during training, it does. Confirm that the server delivers x16 to every slot at the chosen population; many 8-GPU boards route two cards through a switch and share bandwidth.
6. MIG and NVLink
MIG (Multi-Instance GPU) partitions one card into isolated instances with dedicated memory and compute: up to 7 on H100/H200, 4 on RTX PRO 6000 Blackwell, none on L40S or GeForce. Essential for multi-tenant hosting with hard isolation. NVLink lists whether GPU-to-GPU bridges are supported and at what bandwidth; on PCIe cards this is a physical bridge sold separately, on SXM it is built into the baseboard.
7. Drivers, licences and software
Data center and RTX PRO cards use the production driver branch with long support windows; GeForce uses the consumer branch and its licence prohibits data center use. vGPU (virtual GPU for VDI or GPU sharing inside VMs) requires a per-GPU or per-user licence from NVIDIA on supported cards; MIG on bare metal does not. NVIDIA AI Enterprise is bundled with some SKUs (for example H200 NVL and RTX PRO 6000 Server Edition include a multi-year subscription) and sold separately for others, which changes the total cost of the comparison.
8. What the datasheet does not say
- Part number and revision
- The same marketing name can cover several PNs with different TDP, firmware or OEM branding. Quote by PN.
- Warranty
- 3 years is standard for NVIDIA data center GPUs; who provides RMA (manufacturer, OEM, distributor) matters more than the length.
- Serial numbers
- Ask for serials on the invoice and confirm they are new (not refurbished or pulled) and not region-locked for support.
- Country of origin and export classification
- Needed for customs and, for some destinations, for export licences.
- Qualified servers
- The OEM compatibility list for the exact card; a card that fits physically may not be supported thermally.
- Availability and lead time
- In stock, in transit or on allocation. This is often the deciding factor between two comparable cards.
Nodeforge quotes every GPU by part number with the configured TDP, warranty terms and serials on delivery. If a spec sheet leaves you unsure, send it to us and we will annotate it.
Next step
Need help choosing the hardware?
Send the model and the workload. An engineer replies with a BOM, availability in Dubai and Hong Kong, and lead times.
Related
Keep reading
Choosing GPUs for LLM inference vs training
Inference is a memory problem, training is a bandwidth-and-interconnect problem. How to size VRAM, read precision figures and decide between PCIe cards and NVLink systems.
DGX Spark and workstation-class AI: what fits on a desk
DGX Spark, RTX PRO Blackwell workstation cards and GeForce RTX 5090/5080 for development and local inference. What each runs comfortably, and where the desk stops being enough.
H200 NVL vs RTX PRO 6000 vs L40S: where each wins
Three PCIe cards, three price bands. A side-by-side of memory, bandwidth, power and cooling, and the workloads where each is the right answer rather than the expensive one.
