AI Agent Production Readiness: A Practical Checklist

A convincing agent demonstration answers one question: can this system complete this example? A production decision asks something harder: can we tell when it failed, contain the consequences, and recover without creating a second problem?

My recommendation is to start with one business workflow, one accountable owner, and written acceptance checks. Do not begin by granting access to every tool the agent might eventually use. Expand access when the workflow has earned it through evidence, not because the demo felt capable.

This checklist reflects a September 2026 review of my own projects. It distinguishes tested behavior from design intentions. It is an engineering starting point, not a security certification, an exhaustive audit, or a claim that every item belongs in every small pilot.

What changed in the conversation about agents

The useful unit of evaluation is no longer just the model's answer. It is the model working inside software that gives it tools, preserves state, applies permissions, and decides when a task is finished. Anthropic's January 2026 discussion of agent evaluations distinguishes the recorded interaction from the actual outcome in the environment. A statement that work is complete is not equivalent to evidence that the required state exists.

That distinction matters to a buyer. A drafted reply, a queued reply, a delivered reply, and a correctly delivered reply are four different outcomes. If the dashboard calls all four “success,” it conceals the information you need to decide whether automation helps.

1. Define the finished task before choosing the framework

Write a small contract for the first workflow. For an enquiry-triage assistant, it might say: read a permitted inbox, identify the request, draft a response with supporting references, and place it in a review queue. Sending the response is outside scope until a named person approves it.

Specify the input, the artifact, and the stopping point. Then name the conditions that count as failure. A plausible response with an invented deadline is a failure. A missing reference is a failure. An unclear request placed in an exception queue might be the correct outcome rather than a failure at all.

Avoid a target such as “automate support.” It is too large to price, verify, or debug. A bounded contract helps an engineer estimate the work and helps you recognize scope expansion before it becomes an expensive surprise.

2. Map what the agent can reach

List each account, credential, directory, database, and external service available to the workflow. For each one, distinguish read, write, delete, and send permissions. Record which access is required now and which access is merely convenient for a possible future feature.

Also distinguish model instructions from enforced controls. A prompt asking an agent not to inspect a directory is not the same as an operating-system boundary preventing that access. A confirmation screen is not a substitute for credentials restricted to the right account and operation.

The OWASP Top 10 for Agentic Applications for 2026 provides a useful risk-review reference. Use it to challenge assumptions about an agent's authority, not as a badge suggesting that reading a checklist makes a product secure.

3. Test the approval boundary, including the unglamorous paths

In my Sentinel project, all 167 original tests passed. Inspection still found that the controller's default command-approval setting was not consistently enforced. The command-line interface could also silently approve familiar command names.

That was a concrete implementation defect, not a hypothetical model failure. Recognizing a command name did not establish that its arguments were harmless. The repair applied default approval checks in the CLI, controller, and executor, then added regression cases. The resulting 179-test suite passed locally.

For your system, ask an engineer to demonstrate denied actions without launching the real operation. Test the internal execution function as well as the visible button. Include malformed requests, cancelled approvals, dry runs, and a caller attempting to bypass the usual interface. A permission check that exists only in the screen can disappear when another integration invokes the same action.

Sentinel still is not an OS sandbox. Passing these tests proves the checked approval behavior, not that an approved process is contained against every filesystem or network action.

4. Build a small evaluation set with clear outcomes

Start with representative, authorized examples from the workflow. Remove private details where possible and obtain the necessary permission before copying production material into a test environment. Synthetic examples are useful for deliberate edge cases, but should be labeled and should not stand in for all real-world variation.

Include ordinary tasks, incomplete requests, conflicting information, unavailable dependencies, and inputs that should be escalated. For each example, record both the expected result and the evidence required to accept it. Keep a separate set of cases that you do not repeatedly use to tune the prompt.

A test can check multiple dimensions without hiding them inside one score. A response might be structurally valid but factually wrong. It might be accurate but sent to the wrong place. Display those failures separately so a strong score in presentation cannot compensate for a broken permission boundary.

5. Check the meaning of evidence, not just its presence

My AI Operations Lab makes extracted tasks and drafted responses inspectable against their sources. That is useful, but the existence of a citation does not establish that the cited sentence supports the claim. A model can quote a real source and still misinterpret who is responsible or when an action is due.

Use the review interface to test that difference. Put a conflicting date into a fixture. Include an instruction quoted by a customer that is not actually an authorized instruction to the system. Ask whether the application can show the supporting material without presenting uncertain interpretation as established fact.

For a pilot, make uncertainty actionable. “Needs review because two dates conflict” gives the operator something to resolve. A generic confidence percentage may not. If you use confidence scores, validate what they mean before making business decisions from them.

6. Make retries and duplicate actions visible

Suppose an external request times out after the receiving service has already accepted it. Repeating the request blindly can create duplicate records or send the same message twice. Treat that ambiguity as a workflow state, not merely an exception to hide with another attempt.

Ask what identifies a unique task, where its delivery state is stored, and how an operator checks an uncertain result. Use idempotency mechanisms where the receiving service supports them. Where it does not, design a reconciliation step appropriate to the action before authorizing unattended retries.

Set explicit retry and time limits. An agent should have a defined point at which it stops, preserves useful context, and asks for intervention. An endless loop is neither persistence nor reliability from the person paying the bill's perspective.

7. Verify that monitoring describes what actually happened

In NexusTerm, a transcript parser used the current clock while reading historical entries. Old work could therefore look active. The watcher also advanced past partially written records, which could permanently lose an event.

The fixes use recorded timestamps and commit file offsets only through complete records. The macOS Xcode build and 24 tests passed, including a real filesystem-notification test. That is specific evidence about ingestion behavior; it does not make every dashboard indicator authoritative.

For a business workflow, ask whether timestamps describe creation, receipt, processing, or completion. Show delayed data and unknown states explicitly. A stale heartbeat should not be rendered as fresh activity because someone refreshed the page. Test monitoring failures as carefully as the work it monitors.

8. Measure cost per accepted outcome

Model usage is only part of the cost. Add retrieval, external tools, retries, hosting, operator review, and exception handling where they apply. Compare that total with the number of outcomes accepted under the workflow's quality criteria.

Here is illustrative arithmetic, not a client result: if a pilot costs $60 to run and produces 30 accepted tasks, its measured cost is $2 per accepted task. If only 15 tasks are accepted, it is $4. Cheap individual calls do not rescue a system that requires repeated repair.

Keep the manual baseline honest too. Measure the human time needed for the same output and quality bar. Time theoretically released is not automatically cash saved. It may represent capacity, faster response, or less backlog, and those should be described separately.

9. Rehearse stopping and recovery

Before a live pilot, stop a run deliberately. Confirm what happens to queued actions, outstanding requests, held credentials, and the operator's view. If cancellation cannot withdraw an already accepted external action, make that limitation explicit.

Then rehearse recovery. The next operator should be able to determine what was completed, what is uncertain, and what can safely be retried. A log that requires the original developer to interpret every line is not much of a handover.

Keep only the operational data necessary for that job. Raw prompts and full documents may contain sensitive material. Decide retention, redaction, access, and deletion deliberately rather than collecting everything because storage is inexpensive.

10. Release a measured pilot, not a sweeping promise

Agree on a limited cohort or task volume, allowed actions, budget, review responsibility, and a rollback trigger. Test the conditions before enabling the live workflow. Record the model and configuration used so later changes can be compared against the same baseline.

A useful handoff includes the workflow contract, permission map, evaluation cases, known failures, run instructions, and recovery procedure. If a supplier cannot explain what remains unverified, the absence of that explanation should itself be a finding.

You do not need an elaborate platform to begin. You need a workflow small enough to understand and evidence strong enough to decide whether it should grow. If you already have an agent and want that evidence, scope an AI systems review. If you are choosing an implementer, use the AI developer hiring checklist and bring one real task to the conversation.