architecture

An approval queue for AI agents that the agent cannot approve

Risky AI agent actions wait for a person in a queue the agent's own database role cannot decide. Expiry is never a yes; the kill switch fails closed.

An AI agent that can take a booking or move money will, sooner or later, be asked to by someone it should not obey. The usual answer is "ask for confirmation". That only means something if whatever asks for confirmation cannot also give it.

In PANTHEON the consequential tier of actions does not resolve to "allowed". It resolves to queued: a row in a database table, an approval id, and a person on the other end. This post is about what makes that queue a gate rather than a formality.

The agent can ask. It cannot decide.

Requests live in approval_requests. The role the application connects as, pantheon_app, has INSERT and SELECT on it, and nothing else: UPDATE and DELETE are revoked. So the code path an agent drives can put a request in the queue and read it back, but it cannot mark one approved, even if a prompt injection steers it to try. Decisions run under a separate, privileged role used only by the owner's decide path.

The test is test_app_role_cannot_decide: it connects as the app role, inserts a request (allowed), then tries UPDATE approval_requests SET status='approved' and expects the database to refuse.

The queue is also per business. It sits under the same forced row-level security as every tenant table (test_rls_isolates_approval_requests_across_tenants), so one business's owner cannot see or decide another's.

Approving resumes the work

A request that came from a running pipeline carries the run's id. Approving it signals that run's wait, so the parked work resumes the moment a person says yes, across restarts and workers (test_approve_resumes_parked_run). Rejecting signals the same wait with "denied", so the run can abort the action cleanly (test_reject_resumes_with_denied_payload). A request can be decided once (test_double_decide_raises).

Nobody looking is not a yes

A queue nobody clears fills up. In September I measured 39 pending requests, the oldest 76 days old, with a real one buried under months of dead ones. So pending requests older than seven days now move to expired, never to approved. A gate that says yes because nobody looked is not a gate. Expired rows are relabelled, never deleted, so the trail can still answer who asked for what and what became of it. The totals by outcome are on /what-is-pantheon.html, recounted from the database before each deploy.

The kill switch fails closed

One level up from approvals is a pause: global, or per business. With it on, a consequential action is refused before it is even queued. The switch is read on every turn through a five-second cache, and if that read errors it returns "paused" rather than "carry on" (test_killswitch_fails_closed_when_store_raises). A kill switch that a database blip silently ignores is not a kill switch. The app role can read it but cannot set it (test_app_role_cannot_pause).

What this does not prove

It is my own design, checked by my own tests and self-run audits, not by an independent review. It proves the agent's role cannot approve; it does not prove every action is put in the right tier. That classification is policy, and a tool marked low-risk when it should not be would skip the queue. Bookings and deposits are built and switched off in production: what the queue has carried is announcements, enquiry replies, notes, slot holds, shift-scheduling actions from a prototype, and my own test requests. No money has passed through it.

You can watch it work on /prove.html: a tier-3 action is parked at the approval gate in a sandbox, and the kill switch is there to try.

← all posts