# AI FinOps: Why Falling Token Prices Make Costs Explode > In episode 9 of Prompt & Proper, Steffen Ochsenreither and I talk about AI FinOps. The price per token keeps falling, yet the bill keeps growing, because agentic workflows consume far more than the price drop can absorb. We cover the right model for the right task, why the model choice has to be automated, why cost alone is no KPI, and why companies will soon buy tokens the way they buy electricity. - Date: 2026-09-29 - Source: https://romanoroth.com/en/podcast/e09-ai-finops/ - Tags: AI, Podcast, Prompt & Proper, FinOps, Token Costs, Agentic Coding The price per token has been falling for months. The bill keeps rising anyway. In episode 9 of **[Prompt & Proper](https://promptandproper.ai/)**, Steffen Ochsenreither from Swiss Post and I compare what we see in our organisations, and it is the same pattern on both sides: the more we develop agentically, the faster consumption grows, and it grows faster than model prices drop. The episode is in German, and this post captures the core of the conversation.
## Why Consumption Grows Exponentially For everyone listening, this is the mechanism that matters. When you develop software agentically, the agent often works through a problem in several loops. With every loop, the previous context goes back in. So token consumption follows an exponential curve. Steffen named the second half of the problem. In traditional IT you ordered a licence and knew what it cost. You knew what a virtual machine cost, you knew what a server cost. With tokens, your input no longer determines how much consumption comes out the other end. Once an agent works autonomously, you lose almost all control over what it actually does. I see that with my own agent, which has been building a game for me autonomously. Over one weekend it easily burns a billion tokens. That is the moment you start looking at cost per result instead of cost per token. ## The Right Model for the Right Task Steffen checked which models the developers at Swiss Post actually use for coding. Around 70 percent of usage goes to frontier models, the most expensive ones. He does not put that down to laziness. People forget to switch, or they reason that if the best model is available, they want the best answer. For writing documentation, that makes no sense at all. At Zühlke we store for every task which model runs it. The rule of thumb that has emerged: - Analysis and specification get the frontier model, the strongest one you have. This is where the problem gets broken down. - For planning the implementation a smaller model is often enough, although some people stay on the frontier model here. - The implementation only executes the plan, so the smallest and cheapest model will do. Steffen added the counterweight. An expensive model that runs through once can be cheaper than a cheap model that needs three or four iterations for the same task. What you have to measure is the cost per completed task, not the price per million tokens. According to Steffen, we all still have a steep learning curve ahead of us on that. His point on world knowledge also stuck with me. Specifying documentation does not need the problem-solving capability of a frontier model, and it certainly does not need its world knowledge. You pay for everything the model knows, even when the task uses none of it. ## Automate the Choice, or It Will Not Happen > "One should not try to hammer everything with the best model." Steffen's conclusion from the data is sobering: if you do not automate the model choice, it gets forgotten, ignored or simply not done. And that leads straight to high costs. My answer to that is an AI gateway between the teams and the providers. It does three things. It controls which models get used. It decouples us from the individual providers. And it lets us break token consumption down to the project and the unit. Without that last part, AI FinOps stays a guess. Right now much of this still runs through training. In our AI Foundation training and the AI Augmented Software Engineering training we show explicitly which model belongs to which step. Next, I expect a local model to analyse each incoming task and decide which model gets it. That can go further: the same step can detect whether a request is confidential and route it to local or on-premise infrastructure instead of a frontier provider. ## The Consulting Stretch For a consulting company this creates a real problem. With a fixed-price project running over a year, how do you estimate the token costs? That is extremely hard to build into an offer or a business case, because it is so variable. With time and material it is no easier. Do we charge the developer's hourly rate as a base price and add a surcharge for tokens? Or do we invoice the token costs directly? I do not have a final answer to that yet, and I suspect many in our industry do not either. ## Cost Is No KPI on Its Own Steffen made the point that turns this from a cost discussion into a management discussion. You always look at the cost, but what you need to set against it is the benefit. And the benefit is even harder to measure, because you no longer do the same job without AI. There is no direct comparison left, only a historical one. His view: the next competitive advantage for companies using AI comes from better AI economics. How do I track it, and where do I use it to the best effect? > "Otherwise I just equip lots of people with licences, or in this case tokens, and get nothing in return." I fully agree. In the end you do not look at the output, meaning the lines of code or the tokens consumed, but at the outcome. Did we solve a problem? Did behaviour change? That is what drives the impact, the revenue or the margin. And it raises a question many skip: does it make sense to put a non-deterministic system into this process at all? Most people take the process they have been running for ten years, sprinkle AI on top and wait for the magic. You have to rethink the process from scratch. ## Buying Tokens Like Electricity My expectation for the next years: token costs, or AI costs in general, will be managed the way we manage electricity costs. Companies will have an AI FinOps team that looks daily at where to buy the cheapest tokens and closes the contracts. If I think that through a bit further, a market emerges, similar to a stock exchange, where you secure tokens at a discounted price for a whole year. > "Then you buy the tokens for a whole year upfront, the way you buy electricity today." Steffen took the electricity comparison one step further. Energy is cheaper at times when there is more of it and demand is low. Data centres could offer cheaper tokens in the evening and early morning, and then you shift the tasks that are not time-critical into those windows. Not every request has to be answered immediately. ## Open-Weight and Local Models The last part of the episode was about the cheap end of the spectrum. Steffen sees a misconception out there: a model trained on huge amounts of data is not automatically better at a specific task. For many coding tasks, open-source or open-weight models work well. I recently tested a Qwen model locally and was fascinated. It comes close to a Sonnet or an Opus. They removed layers so it fits into your memory. It is far slower than the frontier models, because your Mac or PC gives you less throughput in tokens per second, but the answer was good. For a lot of tasks that is already enough. And the next step is already visible: the first models cast directly into silicon, so small requests will run straight on your own hardware. ## The Take-Away Steffen's wrap-up at the end of the episode, and I share every point of it: - You need FinOps for AI. If you do not measure, you do not know what is happening, and the nasty surprise arrives with the invoice. - Track, look at what happens, raise awareness among your people. - The most important lever: the right model for the right task. - And if you set it up properly, look beyond cost to the benefit and the actual business value. If you enjoy the episode, subscribe to **Prompt & Proper** on [Spotify](https://open.spotify.com/episode/5ZWb7qvPqvHghupPpovoGv), [Apple Podcasts](https://podcasts.apple.com/us/podcast/prompt-proper/id6792786850), [Amazon Music](https://music.amazon.com/podcasts/b8a56367-a595-4865-8e73-97c706fa4743) or [YouTube](https://youtu.be/82jxgz5Nasc) and tell us which AI topic you are wrestling with right now. All episodes: [promptandproper.ai](https://promptandproper.ai/).