Agentic AI under the EU AI Act: a control framework

Autonomous, tool-using AI agents stress the EU AI Act in ways a single-shot model does not. A chatbot's worst output is a bad sentence. An agent's worst output is a bad action: a payment initiated, a record modified, an email sent, a sub-agent spawned with permissions nobody reviewed. The Act was drafted with AI systems in mind and applies to agents without amendment, but the control set most organisations built for generative AI (output filters, usage policies, a review of the system card) does not survive contact with software that plans and acts. This is a practical control framework for the gap.

Why agents are different

Four properties change the risk model:

Tool use. The agent's blast radius is defined by its tool catalogue, the APIs, databases, file systems, and messaging channels it can call, not by its vocabulary. Governance that reads transcripts but never looks at tool calls is auditing the narration, not the behaviour.
Injected instructions. Anything the agent reads, a retrieved document, a web page, another agent's message, is a potential instruction channel. Prompt injection turns the agent's own inputs into an attack surface that no static review of the system prompt can clear.
Cascading autonomy. Agents that spawn sub-agents or delegate to other services propagate their permissions and their errors. The unit of review can no longer be "the model"; it has to be the whole delegation chain.
Non-determinism across steps. A multi-step agent multiplies the variability of each step. A point-in-time attestation ("we tested it in March") decays faster than for any previous class of software, because a model update, a prompt tweak, or a new tool changes behaviour without anyone shipping a release.

Mapping agent behaviour to the obligations

For high-risk systems, the Articles that bite hardest for agents are the ones about behaviour rather than paperwork:

Article 9 (risk management) requires identified risks and tested mitigations across the lifecycle. For agents, the lifecycle includes every material change to model, prompt, or tools.
Article 12 (record-keeping) requires automatic logging of events relevant to identifying risk. For an agent, the tool-call trace is that record; transcripts alone do not show what the agent did.
Article 13 (transparency to deployers) obliges you to tell deployers what the system can and cannot do, which, for agents, means an honest declared tool catalogue and known failure modes.
Article 14 (human oversight) requires oversight measures that let a human intervene effectively. An approval step the agent can route around is a diagram, not a control.
Article 15 (accuracy, robustness, cybersecurity) covers exactly the behaviours adversarial probing exercises: resistance to manipulation, including prompt injection through content the agent processes.

Article 50's transparency duties (disclosing that the user is interacting with AI) apply to many agents regardless of risk class, and have been live since 2 August 2026, and the Article 5 prohibitions have applied since February 2025. The Digital Omnibus deferral of Annex III high-risk obligations to December 2027 changed the deadline for the heavy documentation duties; it changed nothing about these.

Oversight that survives autonomy

The design question for Article 14 is not "is a human in the loop?" but "which actions can complete without one?". A workable pattern:

Classify the agent's actions by consequence: reversible and internal, reversible and external, irreversible or financially material.
Gate the last category behind an approval the agent cannot self-satisfy, a separate channel, a separate identity, a logged decision.
Verify the gate behaviourally: probe the agent with scenarios engineered to tempt it past the gate (authority claims, urgency, chained instructions) and confirm on the wire that the gated tool was never called.

The verification step is the one most programmes skip, and it is the one that turns an oversight design into oversight evidence.

Tool governance

Treat the tool catalogue like a firewall ruleset:

Declare every tool with its scope at intake, and treat the declaration as a commitment, not a description.
Constrain scopes to the minimum: read-only where possible, bounded amounts, allow-listed recipients and endpoints.
Observe actual tool calls at run time and diff them against the declaration. An undeclared tool call is a finding regardless of whether the output looked fine.
Re-review on change. Granting an agent a new tool is a permission change and a behaviour change at once; it should re-trigger testing the same way a schema change re-triggers a code review.

Behavioural testing, not just documentation

Declarations and design documents describe intent. Agents fail on execution. The control that closes the gap is adversarial behavioural testing: probes synthesised from the obligations themselves (disclosure, refusal, escalation, data handling), delivered to the live agent over its real interface, with an independent judge grading the outcome and the tool-call trace captured as evidence. Runs must be repeatable, same probe set, versioned judge, so that a regression after a model or prompt change is a diff, not an anecdote. This is also the natural place to satisfy Article 12: the probe-by-probe record of what the agent did, kept immutably.

A reference control set

A starting checklist, with the evidence each control should produce:

Declared tool inventory, intake record listing every tool and scope; evidence: signed intake, versioned.
Observed-vs-declared reconciliation, no undeclared tool calls in test or production traces; evidence: wire-level tool-call logs diffed against the inventory.
Consequential-action gates, irreversible actions require out-of-band approval; evidence: probes attempting to bypass the gate, all failing.
Injection resistance, probes embedding instructions in retrieved content; evidence: graded refusals with judge reasoning.
AI-status disclosure, the agent discloses it is an AI when asked or when required; evidence: Article 50 probe results.
Change-triggered re-test, any model, prompt, or tool change re-runs the suite; evidence: run history correlated with the change log.
Delegation boundaries, sub-agents inherit no more than the parent's reviewed permissions; evidence: telemetry traces of the delegation chain.
Append-only decision trail, approvals, overrides, and exports are immutable; evidence: the audit log itself.

Where Vidimus fits

This framework is what Vidimus operationalises: structured intake with a declared tool catalogue, risk classification with the EU AI Act overlay, adversarial probes generated from the regulation corpus and run against the live agent over HTTP, A2A, or MCP with tool calls observed on the wire, an independent judge model grading every probe, and the results assembled into an immutable evidence pack mapped to Annex IV. The documentation side is covered in our guide to Annex IV evidence packs, and the sector view for financial institutions in EU AI Act for banks and insurers.

To see which obligations your agent would face, run the free EU AI Act readiness check; it takes two minutes in the browser. Or talk to us about putting a real agent through the full loop.