How the parts fit together
Follow one request. A user sends a message to an agent. The agent reads its instructions and asks the model for a reply. If it needs information, it calls a tool, such as a lookup in your order system. The reply goes back to the user. Tracing records each step. Evaluations test answers before real users see them. Publishing gives the agent a stable endpoint for real use. The rest of this lesson takes each part in order.
Agents: three types
Foundry Agent Service supports three agent types. The type decides how much code you write and how much Foundry runs for you.
- Prompt agents: you define the model, instructions, and tools, and Foundry runs the agent. There is no code or container to manage. Start here.
- Voice-based prompt agents (preview): prompt agents for real-time spoken conversations. You connect over a WebSocket with Voice Live.
- Hosted agents: your own code or framework, packaged as a container image or a zip file. Foundry runs it with a managed endpoint, automatic scaling, and a dedicated Microsoft Entra identity.
Move to a hosted agent when your logic needs custom code, multi-step workflows, or a framework such as LangGraph.
Models and deployments
A model in the catalog does nothing for your app until you deploy it. Foundry prefers serverless API deployments. Managed compute (preview) runs open-source and custom models on dedicated GPU capacity that Foundry manages.
Learn says you can swap models without changing your agent code. Agent Service works with many models, such as GPT-4o, Llama, and DeepSeek.
Tools, MCP, and toolboxes
A model can only write text. Tools let an agent act.
- Built-in tools: web search, file search, a code interpreter, and memory.
- Custom tools: functions you write, OpenAPI specs that describe an API, and MCP servers.
- MCP, the Model Context Protocol, is a standard way for an agent to connect to tools.
A toolbox groups tools into one reusable unit. You set it up once, and Foundry exposes it behind one managed MCP-compatible endpoint that any agent can use. Authentication, governance, and versioning sit in one place, so you update a tool once, not in every agent.
Tracing: see what the agent did
A trace is the full record of one agent run. It is made of spans, and each span is one step with a start time, an end time, and details. The main spans are invoke_agent for the whole run, chat for a model call, and execute_tool for a tool call.
Server-side tracing needs no code changes. Connect an Application Insights resource to your project, and Foundry logs traces for prompt agents and hosted agents. Open the agent, then the Traces tab, and select a trace to step through it. Foundry keeps traces for 90 days.
Trace Replay has two views. The User view shows the conversation the way the user saw it. The Trajectories view shows the step tree, with timing or token cost for each step.
Evaluations: score quality before users do
An evaluation runs an agent or model on test data, then scores the answers. The steps are: choose a target, choose your data, choose evaluators, run, and review.
- Target: an agent, a model, a dataset of existing answers, or traces that are already captured.
- Data: test questions in CSV or JSONL, existing conversations, or generated test data.
- Prerequisites: the Foundry User role on the project. For AI-assisted quality scores, you also need an Azure OpenAI connection with a deployed GPT model, such as gpt-4.1-mini.
| Group | Examples | What it checks |
|---|---|---|
| Agent | Intent Resolution, Task Adherence, Tool Call Accuracy | Whether the agent understood the request, followed its instructions, and used its tools correctly |
| Quality | Coherence, Groundedness, Relevance, Fluency | Whether answers are logical, grounded in the provided context, relevant, and natural |
| Safety | Violence, Sexual, Self-harm, Hate/Unfairness | Whether responses contain harmful content |
Run evaluations before you deploy, then keep running them on production conversations to catch regressions. That habit matters more than any single score. The agent optimizer (preview) evaluates agent behavior and generates candidate improvements to instructions, skills, tool descriptions, and model choices. Full-conversation scoring is also in preview.
Governance and security
- Identity: Microsoft Entra. Each hosted agent can have its own identity, so it reaches resources without shared passwords.
- Access: Azure role-based access control decides who can create, invoke, and manage agents. The roles from Lesson 2 apply here.
- Safety: content filters help reduce unsafe outputs and mitigate prompt injection risks.
- Network: private networking. Prompt agents support it, and hosted agents can run in your own virtual network.
- Policy: Azure Policy, which the Foundry overview lists among its governance controls.
Publishing and versions
Foundry keeps versions of an agent automatically, so you can compare two versions or roll back. When an agent is ready, publish it to get a stable endpoint. You can then share it through Microsoft Teams and Microsoft Copilot.
Preview features to check
Voice-based prompt agents, the agent optimizer, and full-conversation evaluations are in preview. Preview capabilities come without a service-level agreement, and Microsoft does not recommend them for production workloads until you review their support, security, and availability requirements.
- Start with the overview of Foundry: What is Microsoft Foundry?
- For how agents are built and run: Foundry Agent Service overview
- For how to run an evaluation in the portal: Run evaluations in Foundry
- For how traces are recorded and read: Agent tracing overview