Why Your AI Agent Needs a Compliance Layer

I added an AI agent to a client's booking inbox in February 2024. The demo went well. The agent read new requests, checked the calendar, created bookings, and sent confirmations by SMS. Three weeks after launch, the client's auditor sent six questions about customer data. I answered five. The sixth asked for proof of which records the agent had touched, who had approved each outbound message, and when transcripts with phone numbers would be deleted. I had no clean answer. The launch paused for three weeks while I rebuilt the data layer.

The audit I failed once

The client ran a home-services company with sixty staff and around four hundred booking requests per week. The agent triaged the inbox with four tools: read_inbox, check_calendar, create_booking, and send_sms. I logged prompts and raw tool outputs into one Postgres table called agent_logs, stored as raw text. Half the rows missed a session id. Phone numbers and email addresses sat in plain text. The team kept everything forever for possible training use.

An external auditor arrived before a partnership deal and asked three questions. First: list every customer record the agent read on March 12. Second: show who approved the forty SMS messages the agent sent that week. Third: state the deletion date for transcripts with phone numbers.

My answers were weak. I reconstructed the March 12 reads from timestamps, and the work took two days. The SMS messages had no approval field, because the agent sent them alone. No deletion date existed, because I had never set one. The partnership slipped by a month. I spent the next two weeks deleting twelve thousand raw transcripts with a script and writing a one-page retention note. The client kept me on.

Auditors ask for records. Intentions do not satisfy them. Every agent I ship now carries a small compliance layer from day one, before it touches customer data. The layer has three parts.

The three-part layer: tool-call log, retention pin, PII scrub

Tool-call log

Every tool call lands in a Postgres table named agent_audit. The columns: id uuid, created_at timestamptz, tenant_id text, session_id text, actor text, tool_name text, args_hash text, args_redacted jsonb, result_code text, policy_decision text, delete_after date. I store a SHA-256 hash of the full arguments plus a redacted copy with PII replaced by labels. Raw customer text never enters this table. Two indexes carry the query load: one on tenant_id plus created_at, one on session_id.

Actor takes three values: user, agent, approver. Every outbound message needs a row with actor set to approver, or policy_decision set to auto_approved with the rule id attached. The send tool refuses to run when that row is missing. When the auditor asks which records the agent touched, one query answers: filter agent_audit by session_id and order by created_at.

Retention pin

Every row carries delete_after, and the default equals created_at plus thirty days. A pg_cron job runs nightly at 03:00 and deletes expired rows. Tenant config can shorten the window to seven days for sensitive transcripts, or extend booking records to ninety days with a written note from the data owner. The default stays thirty days. Deletion gets logged as counts per tenant: rows deleted, oldest row remaining. When the auditor asks when transcripts disappear, I show the config line and the counts from the previous month.

PII scrub pipeline

Two passes run before anything reaches disk. Pass one uses regex for email addresses, phone numbers, card numbers, IBANs, and national ID patterns for the countries I serve. Pass two sends the text through a small language model that tags names, street addresses, and birth dates in free text, including misspelled ones the regex misses. Scrubbed text goes to logs and to agent memory. Originals go to an encrypted vault with a seven-day TTL behind a break-glass role, and every read through that role writes its own audit row.

I test the pipeline with a fixed set of two hundred sample messages before each deploy. The regex class must show zero misses. The model class must show a logged quality score for comparison with the previous release. Any new PII pattern found in production becomes a new test sample the same day.

What it costs vs what a breach costs

On a new agent I build this layer in two to three working days: half a day for the audit table and indexes, half a day for the retention job and config, one day for the scrub pipeline and its two-hundred-sample test set, half a day for the approval gate and drill. On an existing agent with messy logs, budget one full week including backfill and deletion of old raw data. Running costs stay small: one extra table, one cron job, and one model call per inbound message, priced in cents per thousand at current rates.

Compare that with the bill I watched a forty-person services firm pay after a data scare smaller than a breach: around $35,000 across legal review, customer notification, and three weeks of rework. Staff quietly returned to manual entry after trust in the tool had dropped. The layer costs about $2,500 of build time. The scare cost fourteen times that, before counting lost trust.

Checklist you can copy

Run this list against your current agent this week. Tick each line or open a ticket for it.

1. Create agent_audit with the eleven columns above and both indexes.
2. Give every session a stable session_id from the first message.
3. Log every tool call. Allow no silent calls.
4. Store args_hash plus redacted args. Keep raw customer text out of the log.
5. Pin delete_after on every row. Hold the thirty-day default.
6. Schedule the nightly delete job. Alert when it skips a run.
7. Add the regex scrub pass with its two-hundred-sample test.
8. Add the model scrub pass for names and addresses.
9. Vault originals with a seven-day TTL behind break-glass access.
10. Gate outbound sends on an approval row.
11. Drill monthly: pull one random session and answer the three audit questions in under ten minutes.
12. Keep a one-page retention note: what you store, where it lives, how long it stays, who approves changes.

I run this drill on the first Monday of each month. It takes twenty minutes. It would have saved my February 2024 launch.