基于 Amazon Bedrock AgentCore 的可插拔 Agentic AI 框架：七层可组合企业级智能体参考架构

“How do we run our AI agents in production at scale?” is the question every customer eventually asks. It comes bundled with several questions. How do we handle thousands of concurrent user requests securely and confidently? How do we make sure the agents stay reliable under real traffic instead of failing silently? How do we know if an agent still meets the quality and safety standards it passed during testing? What are agent security best practices? Should that be deterministic or non-deterministic? And many more. A proof-of-concept answers none of these, because it is built to show the agent model can do the task, not to survive production traffic, cost pressure, and security review.

That gap between a working prototype and a system that runs reliably, securely, frugally and responsibly at scale, is where most agentic AI initiatives stall, and it is rarely the model’s fault. Every team building an agent tends to rebuild the same scaffolding from scratch: its own routing logic, its own guardrails, its own observability, all applied after the fact rather than built in from the start. The teams need a shared, production-ready architecture that every agent can build on.

This post walks through a seven-layer composable reference architecture designed to take multi-agent AI applications from prototype to production scale.

Composable architecture

The Pluggable Agentic AI Framework is a reference architecture for multi-agent applications organized into seven layers, each with a distinct responsibility and a stable contract. The services and technologies shown in each layer reflect the reference implementation, not mandatory choices. Organizations can substitute alternatives that align with their existing infrastructure and requirements. Together, these layers offer a blueprint for building, securing, and operating agentic AI applications at production scale.

Figure 1: The seven layers of the Pluggable Agentic AI Framework

Data and Knowledge is the foundation of Agentic AI. An agent is only as good as the data it can access. Whether an agent is checking out a healthcare formulary, tracking inventory, or interpreting a compliance rule, it needs relevant, up-to-date information to answer correctly. This layer owns retrieval, embeddings, and access to structured and unstructured data through a clear contract and protocol: accept a query and return the most relevant information that acts as context for an agent. Amazon Bedrock Knowledge Base, Vector Engine for Amazon OpenSearch Serverless, Amazon S3 Vectors and Amazon DynamoDB are example building blocks for this foundation.

Model and Inference choices should be driven by configuration, not hardwired into agent or application logic. Use smaller, faster models for simple tasks and larger models for requests that require more nuanced reasoning. It is important to conduct a thorough model evaluation for the use case you are building with realistic and representative business data. A model routing layer keeps AI agents decoupled from the model hosting environment and provider-specific APIs by relying on a consistent request-and-response contract. A proxy such as LiteLLM provides a common interface across providers, so you can switch models based on your evaluations without rewriting application logic.

Orchestration determines how work is distributed, and choosing the right approach is critical. For straightforward tasks, use a single agent to handle the work from end to end. For complex, cross-domain workflows, use a supervisor to delegate specialist agents, each scoped to a specific capability and tied to a specific set of tools. Avoid building one agent that does everything. As tools, instructions, and context accumulate, the context window grows, latency increases, and accuracy can suffer. Tool sprawl sets in, which can push agents beyond their intended boundaries, leading to unintended actions and security breaches. Splitting responsibilities keeps each agent’s context focused and makes latency more predictable. You can also test each agent’s behavior, scale it, and define its security model independently. Enforce bounded autonomy: specialists should execute strictly within the tools they have been granted, and nothing more.

In addition to decomposing into specific specialists and setting its boundary, decouple the specialist agents from the supervisor agents. Allow the supervisor to discover the specialists from a centralized agent registry and invoke them through a standardized protocol like MCP (Model Context Protocol) and A2A (Agent to Agent). Maintain a centralized agent registry that catalogs every agent’s tools, skills, and owner. This gives your organization the visibility it needs for governance at scale, increases the velocity of AI agent implementation through reusability, and avoids the cost of repetitive development. Consider using Strands Agents for specialists, Amazon Bedrock AgentCore harness for the orchestrator, AWS Agent Registry and Amazon Bedrock AgentCore Gateway for centralized discovery and routing.

Security and Guardrails should be a primary concern, built into the design from day one. Agentic AI exposes organizations to a new class of risks that conventional security frameworks were not built to address, from prompt injection and tool misuse to memory and data poisoning. Treat all external content as untrusted by default. Authenticate every request, whether it comes from a user, another agent, or an automated workflow, through a service like Amazon Bedrock AgentCore Identity. Then use fine-grained access control to determine whether each tool call is permitted before execution. Grant agents temporary, scoped credentials instead of persistent secrets. On the data-protection side, mask sensitive data like PHI (Personal Health Information) and PII (Personally Identifiable Information) in agent and tool requests and responses, so that sensitive customer information does not appear in agent outputs or lingers in memory. For accountability, capture every significant decision in a tamper-evident audit trail, route high-impact actions through a human approval gate, and deploy circuit breakers to halt anomalous behavior before it can escalate. Consider using Amazon Bedrock Guardrails for user inputs, Amazon Bedrock AgentCore Identity and Policy in Amazon Bedrock AgentCore for authentication and access control, along with AWS Security and governance controls to bring defense-in-depth security.

Application is where the end-user experience lives, whether that is a chat interface, an API surface, or an embedded widget. Keep this layer intentionally thin. Its only responsibilities should be presenting the conversation and passing requests to the orchestration layer beneath it. Resist the urge to embed business logic here. The cleaner the separation, the more easily you can iterate on the user experience without destabilizing the backend. A decision model can sit at this boundary as a fast, inexpensive check: in one call it scores whether a request is in scope, needs a human, or is urgent, returning probabilities instead of free text. Your code acts on those scores with simple thresholds, answering off-topic requests directly and flagging escalations, so the full agentic loop runs only when it adds value. When building the application layer, think about what would be needed for another AI agent to consume and interact with your AI agents along with humans. Build your external interfaces accordingly.

Observability should watch every layer from day one, not after the first incident. Along with the logs and metrics, end-to-end agent traces are critical to measure whether the agent performed its task correctly. Along with the trace, collect cost and token usage per request, time to first token, whether the call is secure, and surface dashboards for latency, error rates, and spend before you go live. Proactive monitoring is cheaper than reactive firefighting. Consider using Amazon CloudWatch with OpenTelemetry along with Amazon CloudWatch Omni for tracing and metrics to get comprehensive visibility across your stack.

Agentic Operations closes the loop with continuous evaluation, so set it up from day one. Score every production session for groundedness, accuracy, and safety, and put a regression gate in place to catch quality drops before they reach customers. Without this, you only know the agent worked at launch. With it, you can prove it still works after every subsequent release. Consider using Amazon Bedrock AgentCore Evaluations for scoring and Amazon Bedrock AgentCore optimization for continuous performance tuning.

Reference implementation for a retail customer support agent

Figure 2: Functional flow of the retail customer support reference implementation

To bring the architecture to life, the framework includes a ready-to-run reference implementation based on a retail customer support scenario covering order status and refund eligibility. We chose this use case because it exercises every layer of the stack while remaining intuitive enough for a team to follow and adapt to their own needs.

A supervisor agent receives a customer’s refund request and delegates to an Order agent and a Refund agent, each a specialist with a scoped set of tools. The Order agent looks up delivery and purchase details from Amazon DynamoDB. The Refund agent retrieves tier-based policy rules from a knowledge base and calculates eligibility using the delivery date, the current date, and the customer’s loyalty tier. Every one of these calls passes through the AgentCore Gateway, which enforces fine-grained access control before a tool executes with Policy in AgentCore, and every call is traced end-to-end through the observability layer, so when a calculation goes wrong, the trace shows exactly which layer and which decision point caused it.

Anyone building a similar supervisor-and-specialist pattern can clone the reference implementation and swap in their own domain logic layer by layer.

Why pluggability matters as much as the layers themselves

The value of a seven-layer architecture lies not only in how it separates responsibilities, but in how quickly each layer can evolve. Pluggability makes that possible. Stable contracts at layer boundaries, from identity checks and access policies to telemetry interfaces, let you change the underlying technology without redesigning the rest of the stack. Essentially, take an architecture-first approach to building agentic AI systems rather than a technology-driven approach.

The framework puts this into practice by demonstrating pluggability through integrations with third-party providers across the layers: model and hosting providers at the inference layer, search capabilities at the orchestration layer, decision APIs at the application layer, guardrails at the security layer, and agent tracing and monitoring solutions at the observability layer alongside business value and cost tracking. It supports industry-standard providers at each layer, so you can compose solutions that meet your specific requirements.

For enterprises, flexibility protects existing investments. Teams can retain the observability, security, and inference tools they already trust, fill gaps where needed, and adopt new capabilities incrementally. The architecture provides a consistent foundation without requiring every technology decision to come from the same vendor.

Conclusion

A working prototype proves what an agent can do. A well-designed architecture determines whether your enterprise can depend on it. Moving to AI operations means designing security, governance, visibility, cost controls, reliability, and adaptability into the design from day one, not adding them after the system is built.

The Pluggable Agentic AI Framework brings those concerns into an architecture with a clear principle: build a stable foundation without locking every technology choice in place. Models will change. Tools will evolve. Business requirements will shift. Your architecture should make those changes manageable, not turn each one into a rebuild.

Put that principle into practice. Start with the Workshop Studio experience, explore the public sample repository, and use the hands-on implementation as a starting point for your own architecture. It is not a substitute for production security hardening, testing, and review.

Build the capabilities you need today and the flexibility you will need tomorrow.

About the authors

Sindhura Palakodety

Sindhura is a Senior Solutions Architect at Amazon Web Services (AWS) and Single-Threaded Leader (STL) for ISV Generative AI, where she is dedicated to empowering customers in developing enterprise-scale, Well-Architected solutions. She specializes in generative AI and data analytics domains, enabling organizations to leverage innovative technologies for transformative business outcomes.

Sujatha Kuppuraju

Sujatha is a Principal Solutions Architect at AWS, specializing in cloud and agentic AI solutions. She partners with software company executives to architect secure, scalable products on AWS and guide strategic product development.

Mythili Annamalai Sekar

Mythili is a Solutions Architect at Amazon Web Services (AWS) working with Independent Software Vendors (ISVs), where she guides customers in designing secure, enterprise-scale architectures built on AWS best practices. Her work centers on serverless and generative AI, helping software companies turn new cloud capabilities into real, production-ready outcomes for their businesses.

Madhu Samhitha Vangara

Madhu is a Worldwide generative AI Specialist Solutions Architect at AWS, focusing on Agentic AI technical GTM for Amazon Bedrock AgentCore and Strands Agents. She brings deep enterprise experience translating emerging AI capabilities into measurable customer outcomes. Madhu is a speaker at AI conferences and specializes in production-grade Agentic AI.