The Impact of Artificial Intelligence on Carbon Footprint

The impact of human activities on the environment is discussed more and more. Many of you have definitely heard about the impact of cattle breeding, transport and the fashion industry on the environment. But we would like to introduce you to how, many times unexpectedly, artificial intelligence, related research and re-training of large machine learning models can affect the environment.

Artificial intelligence models, especially those based on so-called deep neural networks, are continuously getting bigger, with more and more data coming in. Currently, AI development does not pay much attention to the optimization of models. Simply a larger model is created by adding more layers or increasing the number of parameters of the neural network. This increases the computing power needed for each task. It also increases the energy consumption of the model as well as the price for training the model.

The environmental impact of models using deep neural networks is getting bigger. But the problem is not unsolvable. At KInIT, we care about the environmental impact of our work and are working on measures to reduce these impacts. In this article, we have summarized a few approaches on how to mitigate the impact of AI development. We hope it will serve as an inspiration and that you too will decide to apply some of these principles in your own work. 

In the field of artificial intelligence research, we can come across the terms Red AI and Green AI. Red AI refers to research that does not take into account how much money and resources have been put into research. It is only focused on achieving better results.

On the other side, there is the Green AI approach. Green AI is a way of research in which researchers are motivated to look not only at the results but also at the resources that need to be used to achieve those results.

On the other side, there is the Green AI approach. Green AI is a way of doing research in which researchers are encouraged to look not only at the results, but also at the resources needed to achieve those results. 

We’ve summarized some principles that can help AI researchers reduce the carbon footprint of their work. What can we do to reduce our carbon footprint in artificial intelligence research? 

In this context, computation (inference) is understood as the use of the artificial intelligence model on real data in use (production). 

One approach is to prune the neural network. This involves optimizing the model by removing parts that contribute minimally to the result. Connections that do not contribute to improving the results are simply removed. 

After trimming the unnecessary parts of the neural networks (connections, but sometimes also the neurons themselves), fewer connections enter the computation (inference). This reduces the energy consumption, as no electricity is used to run them. 

Frankl a Carvin looked at whether there was a subnet that could work as efficiently as the original network from initialization in their work The Lottery Ticket Hypothesis. In the case of training and pruning the net, we first need to train the net at full capacity. They dealt with the idea of whether we could determine a subnet at the beginning of training, which would be able to achieve the same results with the same number of data and the same number of training epochs.

After removing unnecessary connections, they reset the network to its original state and then tried to train the entire network. It turned out that such a network is part of every network. They were able to train the network even with better accuracy, being only 20% of its original size.

This would be useless if we did not have hardware support. In the case of training and pruning the neural network a mask is created to determine which neurons will be further trained and which will not.

Another very important factor that can help us to carefully choose the location and providers of infrastructure services, cloud solutions and data centers. 

Different companies and different countries use different types of energy. With regard to the use of sustainable energy sources, Google (uses 100% of renewable energy), Amazon (17%) and Microsoft (32%) are currently leading the way. 

Each of the larger companies also has an energy recovery plan. Microsoft is committed to being carbon neutral by 2030, which means that they will contribute to carbon reduction to the extent of their current production. Amazon became the largest buyer of renewable energy in December, pledging to become carbon neutral by 2025.

Google was carbon neutral in 2007 already and now they focus more on the differences in the usage of renewable and carbon resources. They have committed themselves to using only renewable energy sources by 2030.

There are various ways to use renewable energy sources. Microsoft has placed a server room on the sea bottom to use the natural underwater environment for cooling. It turned out to work relatively well and some components worked even more efficiently. Google is building data centers close to rivers so that they can use energy directly from local hydropower plants.

Let’s focus on how to report the amount of energy consumed and the time it took to use it during the research process. In the Carbon Impact Statement, Peter Henderson and his team created an example of how publications could indicate how much emissions and energy were used for a given research. They also explained why these results need to be published. They have created both rankings and a framework that they say is easy to implement in different solutions.

Every researcher needs to compare their work with other solutions to see if their method is more effective or not. Let’s share trained models and results to make research easier for others, and so that the same work doesn’t require two or even more times as much resources due to re-training. This is one of the simplest mechanisms we can use to help reduce our carbon footprint. In addition to environmental responsibility, we will contribute to faster progress and support the principle of ‘open research’.

There are a number of companies that provide the option to purchase offsets. This means that a company can choose which projects to support financially and to what extent. In this way, it is possible to compensate for the generated carbon footprint.

However, when choosing carbon offsets, it is important that they do not replace emission reduction measures. It should be the last measure to be resorted to when all other options have been exhausted.

There are many benefits to artificial intelligence research, but there are also some downsides that must not be forgotten. It is one of our duties as researchers to reduce the carbon footprint of our work. 

At KInIT, we are committed to making AI research and development not only beneficial, but also environmentally responsible. We are continuously fine tuning our internal guidelines for our researchers to minimize the impact of AI on carbon footprint. We have contributed to the publication of the largest Slovak neural language model to date, SlovakBERT. We have also published other trained models so that other researchers can use them without repeatedly spending resources to train their own. 

We care about the environment. As an institute that actively works with artificial intelligence methods, we will always do our very best to protect the environment. We base our project proposals and grant applications on reusable solutions. We are part of international projects that focus on the reusability of outputs. We want our work to have a positive impact, not to contribute to the environmental burden on the world we live in.