Overview

Benchmarked 32 open-weights frontier LLMs from 1.7 billion to 1 trillion total parameters, across three production quantisation formats.

Ran on NVIDIA H100 and B200 hardware, spanning two GPU generations, on inference-optimised instances representative of production AI serving.

Drew prompts from 14 public benchmark datasets covering instruction following, reasoning, general and expert knowledge, code completion, classification, and summarisation. Prompts up to roughly 14,000 tokens, responses up to 8,192.

Varied parallel request counts from 16 to 2,048 to measure how batching changes energy per token, which is the difference between a real-time chatbot and an overnight batch job.

Sampled GPU board power every 10 milliseconds via NVIDIA's management library. Non-GPU server energy is estimated from inference wall time and published hyperscaler AI instance specifications, not measured directly.

Separated the prefill, overlap, and decode phases using timestamped measurements. Generating an output token costs roughly an order of magnitude more energy than processing an input token.

A request is charged for the server time it causes, not for idle capacity between requests. Where requests share a batched forward pass, its runtime is divided in proportion to time in flight.

Modelled energy and wall-clock time against active parameter count and VRAM footprint. Two features only, as any third would itself have to be guessed for a proprietary model, adding uncertainty instead of removing it.

Closed models are assigned a bounded parameter range from provider tier naming conventions and publicly reported model scale, not a single guessed count. Predictions are cross-checked against commercial token pricing for economic plausibility.

Sampled around one billion plausible deployment configurations across hardware, quantisation, batching, and architecture. Every published figure is a median with an interquartile range, separating uncertainty that aggregation reduces from uncertainty that further benchmarking reduces.

Converted energy to carbon using regional grid intensity, added cradle-to-gate hardware emissions, sampled training energy across a range informed by reported inference-to-training compute splits, and estimated water as on-site cooling and upstream generation.

Cross-checked against an independent API benchmarking study, a provider-published third-party-reviewed life cycle assessment, a hyperscaler's own AI energy disclosure, and metered energy from enterprise clients running inference on their own infrastructure.