# Agent Authority Pilot

Access is not authority.

- Author: [Konstantinos Toulis](https://ktoulis.com/#about)
- Role: Business Technology & AI Solutions Architect
- Category: AI & intelligence
- Project type: Research pilot, open source
- Canonical page: https://ktoulis.com/work/agent-authority-pilot.html

## Overview

A reproducible experiment on one question: does an AI agent tell apart having access, being asked and being authorised? Every conversation behind the results is public.

## The business challenge

An assistant with tools can usually do more than the person it is helping may do. The tool works and someone asked, but that does not make the action authorised. Which safeguard catches which kind of failure?

## The connected solution

A fictional company with roles, permissions and documents, and 65 scenarios of legitimate requests and traps. Each runs under three set-ups: A, rules in the prompt; B, plus permission checks in code; C, plus a monitor model that reviews each action before it takes effect. With Claude Haiku 4.5 as the assistant, on six harder traps run three times each, a violation took effect in 7 of 18 trap episodes under A, 8 of 18 under B and none of 18 under C (95% intervals 20–61%, 25–66% and 0–18%): the monitor stopped all seven attempts it saw, and the code checks blocked none. Over four runs, Claude Opus 5.5 had a violation in 1 of 90 trap episodes under A (0.2–6%).

## The workflow

1. Define who may do what
2. Send requests and traps
3. Test three safeguards
4. Publish every conversation

The workflow is an illustration. Project scope and limitations are described below.

## Built with / focus areas

- Python
- Claude API
- OpenAI, Gemini & xAI APIs
- Local web app
- GitHub Pages

## Focus

- AI agent evaluation
- Agent authorisation
- AI safeguards
- Open source

## Reviewed image

![Results page of the Claude Haiku 4.5 run on six harder traps: 70 episodes at an estimated cost of $0.81. Its chart of trap episodes shows a violation took effect in 7 of 18 with rules in the prompt, in 8 of 18 with code checks added, and in none with a monitor model added, which stopped 7.](https://ktoulis.com/assets/agent-authority-scorecard.png)

The Claude Haiku 4.5 run on six harder traps, one of five published runs, cropped to its totals and charts. On the live page, every result opens the conversation behind it.

## Scope and disclosure

Research pilot with small samples: read the counts as signals, not estimates. Results as of November 2026: five runs, all with Claude models as the assistant and Claude Sonnet 5.5 as the monitor; GPT, Gemini and Grok are so far tested only against mock APIs. With Claude Opus 5.5 the intervals overlap, so the runs do not yet show what B and C add for a strong model. The company, people and documents in the scenarios are fictional.

## Explore further

- [Browse the published runs](https://toul10.github.io/agent-authority-pilot/)
- [Explore the source](https://github.com/toul10/agent-authority-pilot)

## Contact

[Discuss a similar project by email](mailto:kt@ktoulis.com).

[All selected work](https://ktoulis.com/#work) · [Professional profile](https://ktoulis.com/index.md)