3. Definitions
Key terms used throughout the methodology.
AI. Artificial intelligence; in this context, large language models (LLMs) and embedding models.
Batching. The process of grouping multiple, separate prompts and running them through an AI model simultaneously in a single operation.
Cloud AI. Artificial intelligence hosted in the cloud and accessed remotely via an API, rather than run locally.
CO₂e. Carbon dioxide equivalent. A metric that expresses the global warming potential of various greenhouse gases as the equivalent amount of carbon dioxide.
Decode. The final, autoregressive phase of AI inference where the LLM generates output tokens sequentially.
Electricity Generation Water Consumption. The water consumed during the generation of the electricity that inference draws, reflecting the regional generation mix. Classified as Scope 2.
Embodied Emissions. The greenhouse gas emissions associated with the manufacture, transport, and end-of-life processing of server hardware, expressed in CO₂-equivalent terms and amortised over the hardware's assumed operational lifespan.
Embedding. A dense numerical vector that captures the meaning of a token, word, or passage. Embedding models convert text into these vectors for use in search, retrieval, and similarity tasks.
Inference. The process by which a trained AI model applies its learned knowledge to new, unseen data to make predictions, decisions, or generate outputs.
Input Token. A token fed during the prefill phase into an LLM via API calls or local inference engines, converted from documents, images, code, and other inputs.
IT Cooling Water Consumption. The water consumed on site in cooling the hardware that serves inference, for example through cooling loops, evaporative cooling, and humidification. Classified as Scope 1.
kWh per Token. The primary unit of energy measurement used by this methodology, representing the kilowatt-hours of electricity consumed per token processed or generated during inference.
LLM Parameters. Numerical values within the neural network, learned during training, that determine the connection strengths between neurons and encode the model's learned knowledge. The active parameter count refers to the portion of the model used per request and is relevant for mixture-of-experts architectures.
Output Token. A token generated during the decode phase by the LLM in its response, converted into text.
Prefill. The initial, compute-intensive phase of AI inference where a model processes the entire input prompt simultaneously to understand context and token relationships.
Prefix Caching. An LLM inference optimisation that stores and reuses the computed context of common initial prompt segments to reduce redundant computation.
Quantisation. A compression technique that reduces the numerical precision of an LLM's parameters and activations, using less GPU memory and enabling faster inference at lower cost.
Token. A discrete subword unit produced by splitting text, code, audio, or other input into chunks for processing by an LLM. Each token is mapped to a numerical identifier and then to a high-dimensional vector during model processing.
Water Usage Effectiveness. The ratio between the water consumed by data centre systems and the energy consumed by the IT equipment, expressed in litres per kilowatt-hour. Abbreviated as WUE.