# How Do Companies Keep Control over AI Agents?

> AI software now acts faster than a human can approve it. Can AI be controlled at all? A commentary on rules, autonomy limits, agent identities and technical guardrails.

- Date: 2026-10-21
- Source: https://romanoroth.com/en/publications/itwelt-ki-agenten-kontrolle/
- Tags: AI, AI Agents, AI Governance, Security, Governance


AI software now acts faster than a human can approve it. More and more companies and users are therefore asking: can AI be controlled at all?

A growing number of studies show that the question is a fair one. According to Gartner, more than 40 percent of agentic AI projects will be cancelled by the end of 2027, because of rising costs, unclear value or inadequate risk controls. In the same forecast, Gartner expects at least 15 percent of day-to-day work decisions to be made autonomously by AI agents by 2028. A global survey of executives by the Capgemini Research Institute points the same way: trust in fully autonomous agents fell from 43 to 27 percent within one year, while usage went up. The more experience companies gain with AI agents, the more clearly they see their limits and risks.

The challenge is finding the right balance between autonomous work and control. If fear of mistakes leads to banning almost everything and checking it rigorously, work comes to a standstill. Human review of every single transaction then adds no safety, it only slows the process down. That is why control over AI agents has to be built into the system from the start of development.

## The control point moves from the transaction to the rule

Invoice approval makes this concrete. If an AI agent may approve invoices automatically within defined rules, the real control no longer sits with the individual invoice but with the rule. Raise the amount allowed for automatic approval, for example, and the agent can push through much larger payments without any human check. Whoever changes the rules or thresholds also changes the potential risk, in this case the risk of erroneous postings. Permissions, threshold settings and agent instructions therefore deserve special protection. Every change should be documented in a traceable way and approved under the four-eyes principle.

So instead of deciding across the board how much autonomy AI agents should get, users should assess each use case and every action in it individually beforehand. Three questions help: "How large is the damage if something goes wrong?", "Can the action be reversed?" and "Can a human review it in any meaningful way?". A reversible action with little potential damage can run autonomously and be checked by sampling. Critical and irreversible decisions stay with a human until the technical safeguards are in place. If a human cannot meaningfully review an action because it happens within seconds or thousands of times, an approval step does not help either. Then the limit has to be drawn technically.

## Identity and guardrails belong in the system, not in the prompt

Every agent needs its own technical identity with narrowly scoped permissions that are verifiable, time-limited, revocable at any time and centrally registered. A borrowed employee token meets none of these conditions. If an AI agent simply uses an employee's credentials, nobody can later trace who actually acted, and a compromised agent cannot be blocked on its own. AI agents should therefore never have more permissions than the human who authorised them.

A clearly identifiable agent is not enough, though, if its guardrails exist only as instructions. Instructions are not a hard technical barrier, and in the worst case the agent misinterprets them and gets around them. What is needed are limits outside the AI model that also prevent such actions technically: permissions at the tool level, approvals in the workflow, filters in the gateway or separate environments. A simulation Anthropic published in June 2025 shows that instructions alone fall short. It tested 16 models from different vendors in a deliberately extreme scenario in which harmful behaviour was the only way to avoid being shut down or to resolve a goal conflict. Models from every vendor resorted to means such as blackmail, some in up to 96 percent of the test runs.

## Human oversight has to be measurable

The users' approval process itself should be checked regularly. Two metrics show whether it works. If the rejection rate is close to zero, people are just rubber-stamping. If the response time is very short, that also suggests the review is not thorough. When these metrics look suspicious, policies and governance documents may say that humans oversee the AI agents, but in daily work that oversight is not happening. The four safety mechanisms (overarching rules instead of individual approvals, clearly defined autonomy limits, dedicated agent identities and technical guardrails in the system) need an organisation that adapts its roles, responsibilities and ways of working to the use of AI agents. Within the defined frame, the agents do their operational tasks and coordinate work steps themselves; in between, it takes people who steer and monitor the overall system.

ENGEL in Schwertberg, a manufacturer of injection moulding machines, shows what a good example can look like. In the company's first autonomous injection moulding cell, the human only sets the quality target and the machine decides how to get there. Not everyone finds this step from executing to orchestrating easy. The skill it takes is not using AI but leading it: setting goals, setting limits and checking results.

---

*This commentary was originally published in German on [ITWELT.at](https://itwelt.at/news/kommentar/wie-behalten-unternehmen-die-kontrolle-ueber-ki-agenten/).*