This exhibit gives participants the chance to explore the costs and the impacts of energy consumption on model performance as analyzed in Dr. Luccioni’s and Dr. Hernandez-Garcia’s research
AI/ML requires using energy to carry out computations during the model training process. The generation of this energy comes with an environmental cost in terms of greenhouse gas emissions, depending on quantity used and the energy source.
What are the main sources of energy used to train the top 95 AI/ML models?
Estimating the carbon footprint of training AI/ML models requires examination of the main sources of energy used for training, the order of magnitude of CO2 emissions produced, and the evolution of these emissions over time.
Few models are trained using clean energy sources. 40% Coal 12.6% Oil 24.2% Natural Gas 20% Hydroelectricity 3.1% Nuclear
How much equivalent CO2 is produced
per kilowatt hour (gCO2eq/kWh)
The energy source that powers the hardware to train AI/ML models can result in differences of up to 60 times more CO2eq in terms of total emissions.
Coal produces 5 times more carbon than hydroelectricity, yet most models are trained in places where clean energy is not offered. Removing comments from image: Could reuse somewhere What might happen if we used cleaner energy to train our toughest modeling challenges? Moving modeling tasks to countries with an abundance of clean energy resources could reduce CO2 emissions by upto 80% (Coal to Hydroelectric).
Which types of model cost the most to train?
As production uses of AI/ML demand higher quality predictions, training to higher-degrees of precision, energy costs grow exponentially.
We need to take action sooner than later.
As models perform more complex tasks and we train larger models, we are also increasing emissions faster than we are making advancements in energy conservation.
The current trend is toward creating bigger and more closed and opaque models. But there’s still time to push back, demand transparency, and get a better understanding of the costs and impacts of LLMs while limiting how they are deployed in society at large.
CO2 emitted (in kg) by the all models included in the data set, on a logarithmic scale. Each small marker corresponds to a model and the large markers indicate the 99 % trimmed mean within each task and year(s) of publication. The error lines cover the bootstrapped 99 % confidence intervals. The gray line corresponds to the average over all tasks.
Dr. Sasha Luccioni
Sasha is a researcher in ethical and sustainable artificial intelligence at HuggingFace, as well as a Founding Member of Climate Change AI and a Board Member of Women in Machine Learning. Her work focuses on having a better understanding of the societal and environmental impacts of AI models, datasets and systems.
