Carbon per Token

The "Carbon per Token" metric quantifies the CO2 emissions generated by a single token in a Large Language Model (LLM). This emission measurement is of critical importance as it directly impacts green inference decisions. Influencing factors such as model size (e.g., the number of parameters of GPT-4), the geographical region of the data center, the specific cloud provider (e.g., AWS, Azure, Google Cloud Compute), and the underlying hardware infrastructure (e.g., GPU types like NVIDIA A100) determine the exact value. The calculation is based on comprehensive energy consumption models of the hardware, multiplied by the specific emission factor of the regional power grid.

CMOs with sustainability objectives are increasingly required to disclose AI emissions in their reports, especially given the substantial energy consumption associated with training large language models (LLMs) such as GPT-4. Incorporating the Carbon per Token metric facilitates precise quantification, aiding in the selection of energy-efficient inference strategies and operational planning.

An AI team analyzes the Carbon per Token values of their language models and determines that hosting in Azure's "West Europe" region with the same LLM emits 2.5 grams of CO2 per 1,000 tokens. By switching to a lower-carbon region like "North Europe," emissions are reduced to 1.4 grams per 1,000 tokens. This results in an average reduction of 44% in CO2 output for all text generation batch jobs.

The concept of Carbon per Token gained traction from 2024 with the increasing proliferation of large language models (LLMs). Given the rising computational power and associated energy consumption, industry leaders and regulators demanded greater transparency. By 2026, metrics like Carbon per Token became standard indicators for the ecological sustainability of AI applications, significantly influenced by initiatives from cloud providers such as Google Cloud and Microsoft Azure.