The Prompt Playground is where you try changes safely. Load a saved prompt (or start blank), compare up to three panels, swap models, fill template variables, and run. Each result shows latency, input and output tokens, and an estimated cost so you can promote with evidence, not guesswork.

Where To Open It

In the app, go to Evaluate → Prompt Playground. You can also open it from a prompt's detail page, or from a trace span that includes prompt content via Replay in Playground.

Playground Vs Prompts Library

Prompts is the versioned library: immutable versions, version notes, tags, and Promote To Production. The Playground is the try-before-you-promote surface. When a panel looks right, save it as a new version back into the library, then promote when you are ready.

Provider Keys

Runs use your provider API keys. Open Keys in the Playground toolbar and paste keys for the providers you need (OpenAI, Anthropic, Gemini, and others). Keys stay in your browser for this session. They are never stored on Traccia servers.

Shared With Demo Portal

OpenAI keys you add here are shared with the Demo Portal in the same browser. Removing a key clears it for both. Other providers apply to the Playground only.

Compare Panels

Start with one panel. Use Compare to add a second or third (up to three).
Each panel has its own model, temperature, max output, messages, Variables, and optional Tools schema.
Run a single panel, or Run All to compare outputs side by side.
After a run, colored chips show latency, input/output tokens, and estimated cost (when pricing is available for that model).

Different Variable Values Per Panel

Open Variables on each panel separately. Compare mode can use different values for the same {{placeholder}} so you can stress-test wording against real inputs.

Models And Parameters

Pick from popular models or search the catalog. Switching which saved prompt a panel loads does not reset the model you already chose.
Temp (0 to 1) controls randomness. Lower is more focused; higher is more varied.
Max out caps how long the model reply can be (max output tokens). Leave blank for the provider default.

Policy Sandbox

After a run, each panel can show a Policy Sandboxsummary for this run only (offline, detect-only). It checks active org, workspace, and linked-agent policies against the run's cost, latency, tokens, tool calls, or model.

Detections show the policy name, what would happen (Flag, Soft block, or Hard block), which agent or scope it applies to (name plus id when available), and the rule that fired (for example latency or cost vs threshold).
Policies that need live traffic windows (aggregates over time) are skipped with a short reason. Expand the skipped list when present.
Link agents on the prompt detail page so the sandbox can prefer those agents' policies when you open the prompt in Playground.

Save As A New Version

When a panel is linked to a prompt (or you name a new one), use Save As Prompt / save as new version with a short version note. That appends an immutable version in the Prompts library. Promote separately when you want production traffic to pick it up.

Run On A Dataset

Expand the optional Run On A Dataset panel (collapsed by default so single-run Playground stays primary). Attach a dataset, choose scorers, then Run And Save Experiment. Results save as an experiment you can attach when promoting. The panel supplies the prompt and model; each dataset row supplies the variables. Panel variable fields are used only for Run All, not for dataset runs. Runs show progress while they complete.

From Traces

On spans that include prompt content (for example llm.prompt or linked traccia.prompt.* attributes), the span detail panel shows:

Replay in Playground: opens the Playground with messages and model prefilled from that span.
Save As Prompt: creates a new prompt from the span's messages (requires write access). Optionally enable redaction to replace common PII patterns (email, phone, SSN-like) in the saved body before it lands in your library.

Spans without prompt content (for example bare HTTP client spans) do not show these actions.

Next Steps

Scorers

Built-in rules, LLM-as-judge, and custom code for grading.

Datasets

Curate test cases and run them from the Playground.

Prompts

Version, note, and promote production labels in the library.