How long does a review take?

It is scoped to be short — typically days, not weeks — because the value is in the findings and the fix list, not in the length of the engagement. Larger or more entangled systems take longer, and I will say so up front rather than pad a small one.

Do you need access to our code and infrastructure?

Read access to the code, the configuration and the logs, and a conversation with whoever runs it. I do not need write access to anything, and the review does not change your system. Access is arranged in whatever way your security policy requires.

Is this a sales exercise for a rebuild?

No. The deliverable is written so that any developer can act on it. Some clients ask me to do the fixes; many do them in-house. The review is the same either way, and it says so when the right answer is 'this is fine, leave it alone'.

Can you review a vendor's AI product we use, rather than something we built?

Partly. I cannot see inside a vendor's system, but I can review what you have granted it, what data flows to it, what it can do in your accounts, and whether the vendor's answers about all of that hold up. That is often the part that matters most.

What do we get at the end?

A written report: the failure modes and their blast radius, the cost accounting, the permission and data-access map, and a prioritised fix list with verification steps. Plus a call to walk through it, in plain language, with whoever needs to make decisions.

You already have AI in production and are not sure what it is doing.

AI systems review

A short, blunt assessment of an existing AI system: where it will fail, what it costs, what it can reach, and what to fix first.

The situation

Plenty of businesses now have AI in production that was built quickly, by a contractor who has moved on, or by an enthusiastic team member, or by a vendor whose answers to 'what can it access' are vague. It works, mostly. Nobody is quite sure what it costs, what it could break, or what would happen if it went wrong at 3am.

An AI systems review is a short, fixed-scope engagement that answers those questions in writing. It is the same review I run on my own systems before I let them near anything that matters, applied to yours, and it ends with a prioritised list of fixes that any developer can pick up — not a proposal to rebuild everything with me.

How I approach it

Failure modes and blast radius

For each thing the system does, we ask: what happens when the model is wrong, what happens when the upstream service is down, what happens when the input is malformed or hostile, and how far does the damage travel before something stops it. The answer to that last question — the blast radius — is usually the most important finding in the review.

Cost and token accounting against real usage

Not the vendor's estimate: your logs, your invoices, and the actual calls the system makes. This regularly finds a single prompt or retry loop responsible for most of the spend, and it always produces a number you can put in a budget. If the system has no way to measure its own spend, that is a finding on its own.

Permission and data-access audit

What can the system read, write, send and delete? Which credentials does it hold, and what else do those credentials open? Where does customer data go, and does anyone know? The output is a one-page map of reach that you can hand to a client, a security questionnaire or an insurer.

A fix list in priority order

Every finding is rated by likelihood and consequence and written as a task: what to change, why, and how to verify it is fixed. The first few items are usually cheap and close most of the risk. You can give the list to your own developer, to a new contractor, or to me — the review is designed not to depend on the answer.

What gets delivered

Failure-mode and blast-radius analysis
Cost and token accounting against actual usage
Permission and data-access audit
A prioritised fix list you can hand to any developer

What it costs to get wrong

A monthly AI bill that has become a line item nobody questions, where one retry loop accounts for most of it.
A system holding credentials far broader than its job, one prompt injection away from an incident.
A client security questionnaire you answer optimistically because nobody actually knows.
Finding out what the system could reach only when it reaches it.

Why you can believe this

The same review I run on my own systems before I let them near anything that matters.

Explore the work Read the ai-os case study

Questions buyers ask

How long does a review take?: It is scoped to be short — typically days, not weeks — because the value is in the findings and the fix list, not in the length of the engagement. Larger or more entangled systems take longer, and I will say so up front rather than pad a small one.
Do you need access to our code and infrastructure?: Read access to the code, the configuration and the logs, and a conversation with whoever runs it. I do not need write access to anything, and the review does not change your system. Access is arranged in whatever way your security policy requires.
Is this a sales exercise for a rebuild?: No. The deliverable is written so that any developer can act on it. Some clients ask me to do the fixes; many do them in-house. The review is the same either way, and it says so when the right answer is 'this is fine, leave it alone'.
Can you review a vendor's AI product we use, rather than something we built?: Partly. I cannot see inside a vendor's system, but I can review what you have granted it, what data flows to it, what it can do in your accounts, and whether the vendor's answers about all of that hold up. That is often the part that matters most.
What do we get at the end?: A written report: the failure modes and their blast radius, the cost accounting, the permission and data-access map, and a prioritised fix list with verification steps. Plus a call to walk through it, in plain language, with whoever needs to make decisions.