Evaluation and red teaming

Evaluation measures the quality, safety, and reliability of models and agents while you build them and after they are live. In Microsoft Foundry, an evaluation applies one or more evaluators to your data or application. Evaluators return metrics and, when supported, explanations so you can compare versions, find recurring failures, and decide whether an app is ready to deploy.

Concept 8 introduced the same four-step rhythm and Prompt Optimizer. This expert concept goes deeper: how to run evaluations from the Microsoft Foundry portal, how to pick evaluators and datasets for regression testing, how to wire evaluations into a release pipeline, and how to red-team agents before real users probe weak spots. Concept 13 showed how traces in Application Insights capture real conversations; here you turn those signals into scored test runs and adversarial scans.

• Score answers already stored in a dataset without calling the model again.
• Generate fresh responses from test inputs, then score them.
• Evaluate individual turns or complete conversations, depending on where the risk sits.
• Run adversarial safety scans with the AI Red Teaming Agent in addition to metric-based evaluation.

Most evaluation workflows in Foundry follow the same sequence. You choose what to evaluate, select evaluators that match your goal, run the job, then review aggregates and individual failures before you change prompts, tools, or deployments.

• Choose what to evaluate: a single turn, a complete conversation, an existing dataset, generated responses, simulated conversations, or production traces.
• Select evaluators: quality and safety metrics such as task completion, relevance, groundedness, similarity, or content safety.
• Run the evaluation: once during development, inside a CI/CD pipeline, or on a recurring schedule for production traffic.
• Review and improve: compare versions, read cluster-style failure patterns when available, and fix issues that repeat across rows or conversations.

Before your first run, review permissions for evaluation workflows, supported regions and rate limits, and virtual network requirements listed in the evaluation documentation. You need the Foundry User role on the Foundry project. AI-assisted quality evaluators also need an Azure OpenAI connection with a deployed GPT model, such as gpt-4.1-mini, acting as the judge model.

The portal walks you through target, scope, data, evaluators, and submit. You can start from the left pane by selecting Evaluation, then Create. You can also open the Evaluation tab on a model or agent and select Create, or open an agent playground, select Metrics, then Run full evaluation.

• Step 1 — Select evaluation target: Agent scores your agent’s output; Model scores a deployment or instant access model; Dataset scores preexisting outputs; Traces pulls agent interactions from Application Insights for a chosen time range.
• Step 2 — Select evaluation scope (agent and dataset targets): Full conversations (preview) scores the whole thread; Individual turns scores one response at a time. Model evaluations always use individual turns.
• Step 3 — Select data source: options include Simulated data, Existing conversations, Synthetic data, Existing dataset (CSV or JSONL), or Existing traces for agents.
• Step 4–6 — Configure agents (agent runs only), map dataset fields when you use existing data, then select testing criteria from built-in or custom evaluators.
• Step 7 — Review and submit: name the run, confirm settings, select Submit, then watch Status on the Evaluation list (In Progress, Completed, Partial, or Failed).

Evaluations usually finish within a few minutes, depending on dataset size and how many conversations are simulated. If a run is slow or times out, reduce rows or simulated turns, and confirm the judge model has enough quota. Multimodal image and audio content in evaluations is in preview; audio currently supports WAV in the documented JSONL shapes.

Pick the evaluation unit before you pick evaluators. Turn-level evaluation scores one response. Conversation-level evaluation scores a full multi-turn interaction and fits outcomes that span several steps, such as finishing a support case. Each evaluator supports specific levels; conversation-level scoring needs message history in chat format, not a simple query-and-response column alone.

The portal preselects recommended evaluators from your target and scope. For full conversations it suggests customer satisfaction, task completion, coherence, and groundedness. For individual turns with existing data it adds agent evaluators plus quality and safety checks. You can add or remove evaluators and attach custom evaluators defined in the project. Many agent-focused evaluators and rubric evaluators are in preview; treat their scores as directional until you validate them on your data.

An evaluation dataset is a reusable set of test cases, usually JSONL with one JSON object per line. Create one when you want a stable set to rerun against new model, prompt, or agent versions—for regression testing, pipeline quality gates, or side-by-side comparisons.

• Upload curated JSONL or CSV as a versioned dataset when you already know the questions and expected shape.
• Generate synthetic evaluation data from an agent definition, inline prompt, or reference file when you want fresh cases before you commit them to a dataset.
• Convert production traces into a dataset when you want failing real sessions to become repeatable tests.
• Simulate multi-turn scenarios with a seed dataset of scenario descriptions, then evaluate the generated conversations.
• Skip building a file when you evaluate by response ID or evaluate Application Insights traces directly in the portal flow.

Keep columns consistent with the evaluators you run. For individual turns, map query and response at minimum; add context for RAG and tool fields when agent evaluators need them. For conversation evaluation, messages and tool_definitions are required. If a field shows as Unassigned during field mapping, assign the column manually—required fields left empty cause evaluator failures, not low scores.

Put evaluations in your deployment pipeline so a regression can stop a release before it reaches production. Foundry supports offline agent evaluation in GitHub Actions and in Azure DevOps; both integrations are in preview and come without a service-level agreement, so treat them as pre-production gates you review before relying on them for every customer-facing change.

• Provide a JSON data file with a name, evaluator list, and an array of query rows (plus optional ground_truth or context fields).
• Point the task at your Foundry project endpoint, a deployment name for the judge model, agent IDs in agent-name:version form, and the data file path.
• Authenticate with Microsoft Entra ID—GitHub workflows commonly use the Azure Login action with OpenID Connect; Azure DevOps uses an Azure Resource Manager service connection with AzureCLI@2 before AIAgentEvaluation@2.
• Read the pipeline summary: per-metric scores, confidence intervals, and pairwise statistical tests when you compare multiple agents against an optional baseline agent ID.

The AI Red Teaming Agent automates adversarial probing of models and Foundry agents. It uses Microsoft’s PyRIT attack strategies plus Foundry Risk and Safety Evaluations to scan for content risks, score each attack–response pair, and produce reporting you can track over time. The headline metric is Attack Success Rate (ASR): the share of attacks that succeeded out of the total tried.

• PyRIT strategies include encoding and obfuscation patterns—Base64, character swaps, jailbreak and indirect jailbreak, and multi-turn escalation such as Crescendo—so filters see harder cases than a plain harmful prompt.
• Cloud red teaming for agentic categories runs in East US 2, France Central, Sweden Central, Switzerland West, and US North Central; harmful inputs are redacted from results shown to developers.
• Supported Foundry targets today include hosted prompt agents and hosted container agents with Azure tool calls; workflow agents, non-Foundry agents, and function or browser automation tool calls are not supported.
• Runs use synthetic data and mock tools, can be non-deterministic, and may false-positive—review results before you change production guardrails.

Use red teaming during design, development, and pre-deployment, and schedule continuous runs on synthetic adversarial data after deployment if your process calls for it. For agentic cloud runs against a Foundry hosted agent, the service treats the run as transient so harmful chat completions are not stored by the Foundry Agent Service. Exercise agents in a purple environment—non-production but production-like—before you trust the scores. Red-team findings feed directly into guardrail choices in Concept 15.

You now have a full loop: stable datasets and portal runs for everyday quality, pipeline gates for regressions, and red teaming for adversarial safety. Keep a baseline agent version, rerun the same evaluators after each change, and promote only when metrics and red-team ASR fit your bar. Concept 15 turns those findings into guardrails, security settings, and network isolation so production traffic meets the standard you measured.

• Workflow overview, approaches, and where to run evaluations. Evaluation in Microsoft Foundry
• Portal steps, targets, scopes, and evaluator selection in detail. Run evaluations from the Microsoft Foundry portal
• Catalog of built-in evaluators, levels, and preview families. Built-in evaluators reference
• Automated adversarial scans, risk categories, and PyRIT strategies. AI Red Teaming Agent