How AI eats away at your sustainability goals

AI isn’t green.

AI is an energy hog with a user-friendly interface.

On the surface, it seems almost weightless. You type a prompt, get an answer, create an image, summarize a document, or have a model write code. Clean. Quiet. Fast. As if a smart assistant were waiting for you somewhere in the cloud.

But AI doesn’t run in the cloud.

AI runs in data centers that guzzle power, consume water, wear out hardware, and put pressure on entire power grids. Behind every “smart” application lies a physical machine room full of GPUs, cooling systems, backup power, transformers, cables, and concrete.

And that machine room is growing at an absurd rate.

Data centers on the scale of Colossus show where this is headed. Not toward a few extra server racks. Toward industrial infrastructure that demands so much energy that the existing power grid cannot supply it. Then suddenly, dozens of gas turbines appear next to the data center. Not because that’s sustainable. But because the demand is greater than the grid can handle.

Therein lies the uncomfortable truth.

AI is marketed as progress. As efficiency. As the technology that helps us work smarter, make decisions faster, and perhaps even become more sustainable. But as long as the underlying infrastructure demands ever-increasing amounts of power, cooling, hardware, and fossil fuel backup, we’re mainly just kicking the can down the road.

Then AI won’t be a solution for sustainability.

Then AI will be the reason sustainability goals slip out of sight.

In this blog, we’ll look at the causes of that energy hunger. And at what actually needs to happen: more efficient models, more discerning choices, better measurement, smarter infrastructure, and above all, less thoughtless use of AI where it adds little value.

Because AI can be smarter.

But only if we stop pretending that “digital” is automatically sustainable.

The hardware behind AI is less smart than AI itself

AI runs on hardware from a detour

AI runs largely on GPUs: graphics processing units. The name already tells you where they come from. They were originally designed to display fast-moving 3D images on a screen. First for games. Later for professional graphics applications.

That worked surprisingly well for AI. Not because a GPU was specifically designed for language models, but because language models, just like 3D graphics, perform enormous amounts of calculations in parallel. Running an LLM on a GPU is therefore much faster than on a classic CPU.

However: faster is not the same as more efficient.

GPUs were the logical fast track to market. They were available, programmable, and powerful enough to suddenly make AI practically usable. Without GPUs, the current wave of AI would likely have taken years longer to materialize. That’s why vendors started building on them. Not because it was the perfect solution, but because it was the best solution available.

Now, there’s a second problem lurking behind this: lock-in. A massive amount of software, tooling, and expertise has been built around GPUs. Modern AI GPUs have also been further adapted with features like Tensor Cores for deep learning, so it’s no longer fair to pretend they’re just graphics cards. But that doesn’t automatically make them the most energy-efficient choice for every AI task. NVIDIA explicitly positions Tensor Cores as accelerators for AI and matrix computations; Apple also uses a separate Neural Engine as a specialized unit for AI workloads on the device itself.

Apple is showing us where things are headed: handling AI tasks on hardware specifically built for that purpose. Closer to the user. With less data transfer, less central processing load, and less energy loss.

That’s no small detail.

If AI is to become sustainable, we must take an honest look at the energy consumption of the machine running AI. And at the alternatives.

Using AI for cheese fondue is not innovation

The power of AI is underutilized

AI is marketed as a breakthrough technology. As an accelerator. As a new layer of intelligence within your organization.

And then we ask for a cheese fondue recipe.

That’s the uncomfortable part. AI can do impressive things. Recognize patterns in large amounts of data. Make connections that humans miss. Rewrite texts in a specific tone of voice. Help explore scenarios. Explain code. Break down complex information into actionable insights.

But a large part of daily AI use doesn’t fall into that category.

Many prompts are actually old Google queries with a new twist. An address. A simple explanation. A recipe. A list. Handy, certainly. Asking for everything in one place feels convenient. The question is whether you really need to run a heavy language model for that.

You see the same thing with AI agents. Someone proudly builds an agent that retrieves invoices from a folder, checks the bank, and automatically sends reminders. Sounds modern. Until you realize that accounting software has been able to do this for decades. Often more reliably, more cheaply, and with far fewer technical detours.

The agent is mainly just pretending to be smarter. In reality, it’s taking an expensive, energy-intensive detour around a problem for which a direct solution already existed. Extra API calls. Extra infrastructure load. Extra costs. Extra complexity.

For AI vendors, that’s wonderful. Every trivial use is consumption. Every prompt is revenue. Every agent is a story that sells.

For organizations, it’s a different story.

What matters there isn’t whether something was done “with AI.” What matters is whether it has become better, faster, cheaper, safer, or more scalable. If the answer is no, then AI isn’t innovation. Then it’s waste with a pretty interface.

The real power of AI doesn’t lie in replacing simple software or search queries. It lies in work where existing systems fall short: interpretation, context, pattern recognition, complexity, and speed of analysis.

So use AI primarily where it makes a difference.

Not where it just feels more expensive than common sense.

Sustainable AI starts with a smaller model

How do you use AI sustainably?

AI doesn’t always have to run with the full weight of a large language model.

That is perhaps the most important step toward sustainable AI: choose the smallest model that performs the task well.

For many applications, you don’t need an LLM that has read half the world. A Small Language Model or Medium Language Model tailored to your field of expertise can often do the same job. Sometimes even better, because it contains less noise and is closer to the context.

And above all: much more efficient.

There are now plenty of open-source models available that you can run locally. Not for every problem. Not immediately plug-and-play for every organization. But certainly for far more applications than is often thought.

On top of that, local AI hardware is becoming rapidly more accessible. Modern Macs, new Windows PCs, and compact machines with dedicated AI acceleration are making it increasingly realistic to run AI close to the user or the organization.

That saves energy, data traffic, and token costs. But just as important: you retain control over your data. You know where your data stays. You don’t have to run every internal query through an external AI service.

Yes, it requires a bit more work than creating an account with Grok, Gemini, or ChatGPT.

But that work is a one-time effort.

And the benefits keep coming: lower costs, less dependency, less energy waste, and more control.

Sustainable AI therefore starts with the question: how small can the model be without losing value?

Make your AI approach lighter, sharper, and more sustainable

AI can add value. A lot of value, in fact. But only if you use it thoughtfully.

Not every problem requires a large language model. Not every application belongs in the cloud. And not every AI tool automatically deserves a place in your IT landscape just because it says “AI” on the box.

Do you want to use AI without unnecessarily inflating your costs, energy consumption, and data risks? Then it starts with insight. Which tasks are suitable? Which model fits the bill? Which data goes where? And which infrastructure makes it smart, secure, and sustainable?

Schedule a no-obligation consultation with me.

You’ll gain a clear picture of the possibilities, risks, and practical choices for sustainable AI in your specific situation.