Observability and Evaluation

Overview

CloudWatch GenAI metrics, traces, and golden evaluation sets.

Key ideas

• LangGraph owns orchestration: explicit state, nodes, and conditional edges.
• AgentCore owns production concerns: runtime hosting, memory, identity, gateway, observability.
• MCP standardizes tool surfaces so workers do not hard-code every backend integration.

Walkthrough

1. Model the workflow as a graph: who plans, who executes tools, who summarizes.
2. Attach MCP tools through Gateway (or local mocks in early labs).
3. Add Memory for session continuity and Identity before calling protected APIs.
4. Instrument traces and run golden-set evals before promoting changes.

References

• Amazon Bedrock AgentCore
• LangGraph docs
• Model Context Protocol