What is quantisation?
Quantisation is a compression technique that stores an AI model's numbers at lower precision, so it uses less GPU memory and can run faster at lower cost.
Why it matters
Model weights are often stored as 32-bit or 16-bit floating point numbers. Quantisation lowers the precision of a model's parameters and activations, for example to 8-bit or 4-bit formats, while trying to keep as much accuracy as possible. The weights then take up less GPU memory, and the memory a model needs sets how many GPUs it takes to serve it.
Precision is one of the factors that changes the energy used per token. Formats such as FP8, INT8 and NVFP4 reflect how hyperscalers serve models efficiently. Choosing a smaller model is a separate lever, because larger models need more electricity for every token they process.
An example
The same open-weights model can be served at FP8, INT8 or NVFP4. Greenpixie benchmarked models across these three formats on two GPU generations, so the effect of precision on energy per token could be measured. A customer calling a model through a cloud API sees neither the format nor the GPU on the bill.
How Greenpixie measures it
Greenpixie benchmarked 32 open-weights LLMs at FP8, INT8 and NVFP4 on NVIDIA H100 and B200 GPUs. These formats reflect real-world, energy-efficient hyperscaler deployments.
Providers do not disclose the precision behind a billing line. The methodology treats quantisation level as a range, samples across it along with the other unknown settings, and reports each model's energy as a median with an interquartile range. Where a customer can confirm how a model is deployed, the range narrows.