AWS Production Readiness Checklist for AI Workloads

An AI workload on AWS is ready for production when every check below is backed by evidence you can point to: a configuration, an alarm, a test result, a restore record. The checklist has 44 checks across eight areas, from Bedrock quotas and agent permissions to restores, rollbacks and on-call. It combines four sets of published AWS guidance into one pass: the Operational Readiness Review questions Amazon developed to confirm a team can operate a workload before it launches, the AWS Startup Security Baseline, and the Well-Architected Generative AI and Agentic AI lenses. Each check links to the AWS best practice behind it.

Why an AI Workload Needs Its Own Checklist

Most production checklists were written for services whose dependencies behave predictably. A model call differs in four ways, and each one changes what “ready” means:

• It is a quota-bound remote dependency. Bedrock meters inference against per-model token quotas, and requests beyond them are throttled. Launch traffic is often the first time a workload meets those limits.
• Its output varies. The same input can produce a different answer, so quality is measured by evaluation rather than asserted by a unit test.
• It is metered per token. A looping agent or an abusive user turns directly into spend, which makes cost ceilings a reliability control as much as a finance one.
• It handles sensitive data. Prompts and retrieved context carry user records, and model invocation logging, when enabled, records full request and response data.

The first two areas below cover these differences. Areas 3 to 8 apply to any production workload on AWS, and an AI product needs them just as much.

How to Use the Checklist

Rate every check in one of five ways:

Rating	Meaning
Ready	Evidence seen: the configuration, alarm or record exists and does what the check describes
Blocker	Must close before launch traffic arrives
Fix within 30 days	A real risk the workload can carry at launch volume
Accepted risk	The team chooses to carry it, with a named owner and a written reason
Not applicable	Recorded with the reason, such as a workload with no agents

Two rules keep the ratings honest. Rate a check ready only on evidence, since a runbook someone plans to write is still a gap. And treat an accepted risk without an owner as a blocker, because nobody will revisit it.

The areas are independent of each other, so they can be split across a team and worked in parallel.

1. Model Layer

For workloads calling Bedrock, SageMaker endpoints or another model API.

Check	Ready means	How to verify
Backend-only model access	The model is called from the application’s backend under an IAM role scoped to the models it uses, and no provider key or AWS credential ships in a browser or mobile bundle	Search the built client bundle for API keys and model endpoints, then read the resources on the role’s bedrock:InvokeModel permission
Quota headroom	Requests-per-minute and tokens-per-minute quotas cover projected peak with margin, for every model and Region the workload calls	Compare the Bedrock quotas in Service Quotas with a peak estimate expressed in tokens as well as requests
Throttling path	Throttled calls retry with backoff, then degrade deliberately: a second model, a queued response, or a clear message to the user	Force a throttle in staging and observe what the user sees
Prompt versioning	Prompts live in version control or a prompt catalog and reach production through a reviewed release, the same way code does	Identify the production prompt’s version and the change that shipped it
Evaluation gate	A fixed evaluation set runs against any prompt or model change before release, with a pass threshold agreed in advance	The pipeline step or release record showing the last result
Input and output controls	User input is validated and size-limited before it reaches the model, and output on user-facing paths passes a filter such as Bedrock Guardrails	Guardrail configuration, tested against a set of prompt injection attempts
Invocation logging decision	Model invocation logging is either on, with access and retention set for the user data it records, or off by a recorded decision	aws bedrock get-model-invocation-logging-configuration in each Region the workload uses

Cross-Region inference profiles belong under quota headroom rather than the throttling path. They raise the throughput available to a model by routing requests across Regions within a geography, and AWS describes them as a capacity mechanism rather than a failover mechanism. A workload using one still needs its own answer for what happens when a call is throttled.

The evaluation gate is also what makes a later model change safe. Model evals covers how to build the evaluation set.

2. Agents and Tools

Applies where a model calls tools or takes actions on a user’s behalf.

Check	Ready means	How to verify
Tool permissions	Each tool runs under a role scoped to the actions it performs, so the agent’s reach is the sum of narrow grants	Read the IAM policy behind each tool
Loop bounds	Step limits and a per-session token or cost ceiling end a run that has stopped making progress	Trigger a looping task in staging and confirm it terminates at the limit
Human approval	Irreversible or external actions, such as payments, outbound email or deletes, wait for explicit approval	Walk the workflow for each such action
Traces	Every run records its reasoning steps, tool calls with inputs and outputs, and model invocations, kept on a defined retention policy	Reconstruct one past session end to end from its trace

Loop bounds and the per-session ceiling are the same control seen from two sides: one ends the run, the other limits what the run can spend before it ends. What an agent costs per conversation explains why that spend grows faster than the number of steps. For agents hosted on AgentCore Runtime, AgentCore Observability captures tool invocations, model calls and error paths as traces in CloudWatch.

3. Account and Access

Check	Ready means	How to verify
Production isolation	Production runs in its own AWS account, separate from development and staging	The account list in AWS Organizations
Root user	MFA registered, no access keys, and account contact details current	AccountMFAEnabled and AccountAccessKeysPresent in aws iam get-account-summary
Human access	People sign in through IAM Identity Center or another federated identity provider, and any remaining IAM user keys are listed with a reason	The IAM credential report
Pipeline access	CI/CD assumes a role through OIDC federation and receives temporary credentials, and that role can change only what the pipeline deploys	aws iam list-open-id-connect-providers, then the deploy role’s trust and permission policies
Audit trail	A multi-Region CloudTrail trail delivers to a bucket that the workload’s roles cannot modify or delete from	IsMultiRegionTrail in aws cloudtrail describe-trails, then the bucket policy
Threat detection	GuardDuty is enabled in every supported Region, and its findings reach someone who acts on them	aws guardduty list-detectors per Region, plus the notification target

4. Network and Data

Check	Ready means	How to verify
Exposure	Only intended entry points accept internet traffic, databases and internal services sit in private subnets, and S3 Block Public Access is on at account level	Security groups open to 0.0.0.0/0, and aws s3control get-public-access-block --account-id <account-id>
Secrets	Credentials and API keys are held in Secrets Manager or Parameter Store and read at runtime	Task definitions and function configuration, plus a secret scan of the repository
Encryption (at rest, in transit)	Data stores encrypt at rest, and every public endpoint serves TLS only	Encryption settings on each store, and load balancer listeners
Backups	Every stateful store is backed up automatically, on a schedule derived from its recovery point objective	aws backup list-backup-plans, plus native backup settings on RDS and DynamoDB
Restore tested	A restore has been carried out, timed, and checked for usable data	The restore record: date, duration and what was checked
Deletion safeguards	Deletion protection on production databases, and versioning on buckets that hold data you cannot regenerate	DeletionProtection in aws rds describe-db-instances, and aws s3api get-bucket-versioning --bucket <name>

The restore check is the one most often rated ready on assumption. A backup job that reports success shows that data was copied. Only a restore shows that the copy is usable and how long recovery takes, which is the number the recovery time objective in area 5 depends on.

5. Failure and Recovery

Check	Ready means	How to verify
Failure model	Each component and dependency is listed with its failure type, service impact and customer impact, including the loss of an Availability Zone	The written table
Availability Zone loss	Compute and stateful tiers span at least two Availability Zones, and the workload keeps serving through the loss of one without manual changes	Subnet placement and Multi-AZ settings, then a failover test
Timeouts and retries (timeouts, retries)	Connection and request timeouts are set deliberately on SDK and HTTP clients; synchronous paths retry sparingly, and asynchronous work backs off exponentially with jitter	Client configuration in code
Tenant fairness	Per-customer rate limits throttle one customer’s surge before it exhausts capacity shared with everyone else	API Gateway usage plans or the application’s rate limiter
Recovery objectives	RTO and RPO are stated for the workload, and the backup and deployment design can meet them	Compare the targets with backup frequency and measured restore time
Restart runbook	Written steps bring the service back from fully stopped, with no circular dependency between components	The runbook, exercised at least once

AWS’s own guidance on retries is more specific than most teams expect. For a dependency called inside a synchronous request, the ORR example questions recommend retrying once immediately and then giving up. Exponential backoff with jitter is for work requested asynchronously. Applying backoff to a synchronous path holds the user’s request open while the dependency is already struggling.

6. Release and Change

Check	Ready means	How to verify
Infrastructure as code	Production resources are defined in Terraform, CDK or CloudFormation and change through a pipeline, with console changes rare and back-ported	Compare the infrastructure code with the live account inventory
Second reviewer	Every production change, including infrastructure and configuration, is approved by someone other than its author	Branch protection and pipeline approval settings
Rollback	A failed deployment can be reverted in minutes, and the path has been used	Deployment history showing a completed rollback
Load test	Tested at projected peak and at a surge from a single customer, with the breaking point and first bottleneck known	Load test results
Environment parity	Staging is built from the same infrastructure code as production, with different inputs	Infrastructure modules and per-environment variables

For an AI workload, the load test is also the quota test. Run it against the same model, Region and inference profile production uses, so the first throttling event happens in a test.

7. Monitoring and On-Call

Check	Ready means	How to verify
Symptom alarms	Alarms fire on what customers experience, such as error rate, p99 latency and failed jobs, in addition to resource metrics	CloudWatch alarms and the metrics behind them
Synthetic check	A canary runs the main user flow on a schedule and alarms within minutes of a failure	CloudWatch Synthetics canaries and their alarms
Dependency dashboard	The operational dashboard shows critical dependencies, including model latency, throttles and token usage	The dashboard itself
On-call	Every alarm reaches a named person on a rota with a defined response, and that person has production access	Alarm actions, the rota and the access list
Runbooks	The top failure modes in the failure model each have a runbook	Runbooks mapped to rows of the failure model
Logs	Application logs are centralised, searchable, access-controlled and kept on a set retention	retentionInDays in aws logs describe-log-groups

8. Spend at Launch Traffic

A bill that multiplies in the first week of launch is an operational incident, so four cost checks belong in a readiness review.

Check	Ready means	How to verify
Unit cost	Cost per request or per conversation is estimated at launch volume, model tokens included	The estimate and the assumptions behind it
Budget and anomaly alerts	AWS Budgets and Cost Anomaly Detection notify a named person	Alert subscriptions in the Billing and Cost Management console
Cost allocation tags	Tags split spend by environment and service, and are activated for cost allocation	Tag coverage in Cost Explorer
Spend ceilings	Per-customer token or request caps stop one customer from driving the bill	Rate limiter and quota configuration

Why your Bedrock bill makes no sense covers building per-feature and per-user cost visibility from data you already have.

What a Completed Review Produces

A finished pass through the checklist produces two documents:

1. The readiness report. Every check rated, with the evidence behind each rating, and the failure model from area 5.
2. The action plan. Blockers first, each with the change that closes it, followed by the items to fix within 30 days, each with an owner and a date.

The checklist is also meant to be run again. AWS recommends repeating readiness reviews through a workload’s life to catch drift from best practice (OPS07-BP02), so rerun it before each major launch and after significant architecture changes.

Want us to run this review on your AWS account? Our AWS production readiness review works through every check on this list, and you get the readiness report and the action plan.

Key Takeaways

• Rate a check ready only on evidence you have seen, and treat an unowned accepted risk as a blocker.
• Bedrock quotas are per model and per Region; size them in tokens against projected peak, and run the load test against the same model and Region production uses.
• Cross-Region inference adds capacity, and the throttling path still needs its own fallback.
• Agents need scoped tool permissions, loop and spend limits, approval for irreversible actions, and traces that can reconstruct a session.
• A backup only counts as ready once a restore has been carried out and timed.
• Rerun the checklist before each major launch; readiness drifts as the workload changes.