Recently, Sam Altman, CEO of OpenAI, stated that saying “Please” and “Thank you” to ChatGPT costs tens of millions of dollars in energy. This shocked the world, and the electricity consumption of Generative AI models (GenAI) became a hot topic on social media.
GenAI, a branch of artificial intelligence focused on creating text, videos, images and more, has exploded in popularity, driven by breakthroughs in Large Language Models (LLMs). These models, such as OpenAI’s ChatGPT, Mistral’s Le Chat and Google’s Gemini, are capable of producing human-like language thanks to their huge number of parameters and extensive training on massive datasets.
However, this technological leap comes at a cost. As the use of GenAI systems expanded, so did their demand for energy. For a long time, the excitement surrounding their capabilities overshadowed growing concerns about their environmental footprint. Now, as these systems become deeply embedded in everyday life, it’s time to ask: how sustainable is our AI future?
In response to these growing concerns, the concept of frugal AI has emerged. Frugal AI methods are focused on building more efficient, sustainable AI systems without compromising performance.
As we continue to integrate AI into various aspects of our lives, it’s crucial to consider not only its capabilities and positive aspects but also its environmental and negative impacts. By adopting frugal AI principles, we can strive for a balance between technological advancement and sustainability, ensuring a future where innovation is welcomed and is environmentally friendly.
What started as fiction soon transitioned into reality as our interaction with technology changed. From simple machine learning algorithms to LLMs, the complexity of the models increases, and so do their consequences. Today, it is difficult to ignore the environmental and ethical costs behind AI.
The negative impact of AI
AI refers to machines and algorithms designed specifically to mimic human intelligence. They learn from data, discover patterns and make decisions, and then learn from their mistakes. We can’t deny the tremendous power of AI, whether in the healthcare industry, the financial sector or other fields. However, we should not neglect the negative impacts that come with it. Some of the most talked about negative impacts remain related to ethical issues such as scraping data without the consent of its owner, and economic issues, such as layoffs related to jobs being replaced by AIs. Yet, one major impact usually overlooked is the impact of AI models, and even more LLM models, on the environment.
The environmental impact of AI models is not just limited to the models themselves, as we have to consider their training and inference, but also the data centers they are hosted on, the hardware used, and their replacement frequency.
The cost of training and inference
Training large models like GPT requires a lot of computing power and contributes to high carbon emissions. A common mistake is to think that training is the only source of carbon emissions, as depending on the number of inferences made by a model, the latter can emit as much carbon, if not more, than during the training phase. For instance, training OpenAI’s GPT-3 model with 175 billion parameters consumed approximately 1,287 megawatt-hours (MWh) of electricity and emitted around 502 metric tons of CO₂. This is equivalent to the annual emissions of 112 gasoline-powered cars, whereas querying ChatGPT emits 4.32g of CO2 per query. (Source)
Google estimates that in fact 60% of the energy consumed by AI is actually during inference and not training.
Furthermore, data centers need to have cooling systems to avoid overheating and damage. Usually, this uses water. In areas where drought is already near, this can put tension on a resource that is already low and in demand. As for the hardware replacements, always acquiring more powerful systems to build even bigger models increases electronic waste and mining for cobalt and rare minerals needed to build these pieces, leads to environmental degradation, deforestation, and pollution.
Nonetheless, AI giants continue to encourage the use of their models through advertising campaigns, leading to an overconsumption of LLMs and increasing the population’s carbon footprint. Today, there is an urgent need to stop investing in powerful AI and shift to sustainable AI. Luckily, awareness campaigns calling for reducing AI usage and promoting frugality have also increased in popularity.
Understanding Frugal AI
When it comes to AI, frugality refers to the conception, development, and deployment of AI solutions that are resource efficient, meaning they use minimal computational power, data, energy, and infrastructure while maintaining high performance.
A frugal solution will require a small dataset for training purposes, be energy efficient, with a low carbon footprint compared to bigger models, and be algorithmically efficient, as it will be built with the smallest number of parameters possible without compromising its performance.
A company making the shift to frugality can also benefit from many advantages: cost reduction, improved accessibility, enhanced sustainability, and faster deployment, which can contribute to a better image in society.
As a result, instead of rushing to build the most powerful AI models, we should stop for a minute and ask ourselves how we can build a more sustainable, eco-friendly AI.
Transitioning to frugal AI is not only the responsibility of AI engineers, data scientists, and developers. It is a combined effort, and each one of us has a role to play. Whatever our job is, there is always a way to contribute to the shift to frugality.
Empowering Frugal AI: It Starts with You
As an AI user, contributing to frugality involves a shift in mindset and making conscious choices in how we interact with AI tools and technology in general. A conscious AI user will start by questioning the necessity of using an AI-powered tool. Sometimes there is a simpler, more efficient alternative to achieve our goal. In case using an AI solution is inevitable, we can choose an AI tool that prioritises energy efficiency.
For instance, an AI that generates an image at a lower resolution. And of course, we should always optimise prompts when interacting with LLMs, making them concise and as concise as possible. In addition to the actions we can take as users, we can also be advocates for frugal AI, constantly educating ourselves on the environmental challenges facing AI and raising awareness around us. By choosing responsible providers and participating in green initiatives, we can each contribute to the shift to Frugal AI on our own level.
Building Smarter, Greener Models: The Developer’s Mandate
Certain jobs require people to use AI services daily. AI engineers and Data scientists, for example, have to constantly work with AI tools. When developing AI models, a data scientist or AI engineer should prioritise energy-efficient models. Using Small Language Models known as SLMs, requires fewer parameters than a large LLM and should be used when possible in order to reduce energy consumption. In addition, to improve the performance of the models and make them learn faster and thus reduce the training time, there are techniques such as knowledge distillation, pruning and quantisation that can be used.
There are also other things to apply as a developer in the field, such as trying to use the least amount of data by prioritising quality over quantity, using cloud infrastructures that are green themselves and optimising the hyperparameters of models by using RAG methods, for example, to cite a few.
Beyond the Code: Frugal AI Across the Organisation
Contrary to common belief, Data Scientist and AI engineers are not the only ones who can help in shifting to frugality. Shifting to frugality starts with the executive leadership of a company by integrating frugal AI into the company’s overall strategy and sustainable goals. Establishing clear policies and guidelines and adhering to frameworks to like AFNOR Spec 2314 for frugal AI, is also important to ensure every department is complying with frugality. Product managers can integrate frugality into their designs, whereas HR and Marketing teams can promote green practices and deliver sustainable awareness campaigns for all employees. As for the procurement and purchasing departments, they can instil green policies and thus only authorise the use of greener genAI tools.
Frugal AI: Facing the hurdles, Charting the Future
As we established, frugal AI is essential. We need a more sustainable, accessible, and efficient AI. Yet, as with every new movement, the path to change isn’t all smooth and easy. Doing more with less comes with significant challenges.
Balancing Performance and Efficiency
Finding the tradeoff between model size and accuracy as for a long time, as the best models were those with the most parameters. Using techniques presented earlier (ex: pruning) increases the risk of a dip in the performance. The trick is to find the smaller model we can use without altering the AI’s capabilities to do its job. And it’s not always that easy!
The Need for Specialised Expertise and Tools
Transitioning into frugal AI is not as simple as flipping a switch. There is a need for new specialists that not only understands how AI models work but also how to compress them. Unlike large-scale AI solutions, tools and framework for frugal solutions are still evolving and are not always readily available. Lacking a specialist in the team makes it difficult to take the step and go frugal.
Ensuring Robustness in Constrained Environments
Guaranteeing robustness is not always possible. Frugal AI means deploying a small AI model into environments with constraints. Ensuring that the small model is reliable and can handle real-world data is a task on its own. Making a model smaller can accidentally make it more fragile and less robust.
Addressing Bias with Data Parsimony
As one aspect of frugality is using less data, ensuring fairness and avoidness bias is another challenge to overcome. Small datasets present both advantages and disadvantages. Indeed, it is more frugal but if not well balanced can be a source of biasness, unfairness and discrimination. Adding bias check steps and ethical data collection is more crucial than ever.
Despite these challenges, the momentum behind Frugal AI is undeniable. From the global push to sustainability to technological breakthroughs, the future looks promising for frugal AI.
Smarter Architectures and Greener Training
Frugal AI is no longer only limited to only compressing models. There is real work done in order to build AI architectures directly. From knowledge distillation to federated learning and self-supervised learning, the training of models is becoming greener.
Expanding to Everyday Applications through Edge Computing
With the deployment of edge computing hardware allowing smart devices to function without the need of full connections to central networks, frugal AI is slowly yet surely going to expand to many everyday applications reducing the energy footprint of always on cloud connections.
AI as a Tool for Environmental Sustainability
Frugal AI is not only limited to frugal AI architectures but can also include using AI as a tool to help the environment. We can use AI to optimise renewable energy grids, such as managing smart cities to lower energy consumption. Transitioning to green data centers and infrastructures will naturally make all AI more sustainable.
Ethical Deployment in Resource-Limited Contexts
Frugal AI is also a way to enable wider deployment, especially when resources are limited. However, as with every new technology, questions about its ethical considerations are to be studied. How can we ensure fair, transparent and unbiased models without using a massive amount of data to validate the models? “Trustworthy AI” comes hand in hand with frugal AI as frugal models should be efficient but also equitable and accountable ,especially if deployed in sensitive applications.
Frugal AI is a testament to the idea that technological advancement doesn’t have to come at the expense of our planet. By confronting these challenges head-on and embracing innovation, we’re building an AI future that’s not just powerful, but also mindful, efficient, and truly sustainable.
Conclusion: A Responsible Path Forward for AI
We have explored the world of frugal AI, delving into both its promise and its practical challenges. One thing is sure, artificial intelligence centers no longer around power and complexity. The idea of doing more with less and to build responsible AI is at the core of the field and a key aspect for the future.
Lower energy consumption and carbon footprint of AI systems are key benefits of frugal AI. They make technology part of the climate solution rather than just a contributor to the problem.
Ultimately, Frugal AI isn’t just about technical optimisation; it’s about building a more sustainable, accessible, and equitable future for artificial intelligence. It challenges us to rethink how we design, develop, and deploy AI, prioritising efficiency, ethical data use, and broad societal benefit. It pushes us towards an AI landscape where innovation and responsibility are inextricably linked, where the pursuit of intelligence goes hand-in-hand with environmental stewardship and social inclusion.
So, as you navigate the ever-expanding digital landscape, I encourage you to consider the principles of Frugal AI. Whether you’re a developer, a business leader, or simply an engaged user, ask yourself:
- Can we do this more efficiently?
- Is this truly necessary?
- How can my choices contribute to a leaner, greener, and fairer AI future?
The collective answers to these questions will define the next era of artificial intelligence. Let’s make it a responsible one.
Here at Devoteam Research, we have already taken a step forward in this direction. Through our Efficient, Smart, and Sustainable Solutions research axis, we aim to find solutions that enable each employee to achieve frugality.