From ea007b82a454e354a4db6b13c4cc7c218d8a6b60 Mon Sep 17 00:00:00 2001 From: JonathanChavezTamales Date: Tue, 10 Mar 2026 20:48:25 -0600 Subject: [PATCH 1/4] x --- autotune/introduction.mdx | 43 +- autotune/prompts/models.mdx | 10 - autotune/reference.mdx | 281 ++++- autotune/sdks/python.mdx | 176 +++ autotune/sdks/typescript.mdx | 149 +++ autotune/setup.mdx | 233 ---- docs.json | 38 +- feedback/api-reference.mdx | 222 ++++ feedback/introduction.mdx | 47 + feedback/python.mdx | 145 +++ feedback/typescript.mdx | 150 +++ tracing/api-reference.mdx | 489 ++++++++ tracing/introduction.mdx | 53 + tracing/manual-instrumentation.mdx | 1497 ------------------------- tracing/opentelemetry.mdx | 219 +++- tracing/quickstart.mdx | 25 - tracing/reference.mdx | 138 --- tracing/sdks/python/reference.mdx | 132 ++- tracing/sdks/python/setup.mdx | 180 +++ tracing/sdks/typescript/reference.mdx | 49 + tracing/sdks/typescript/setup.mdx | 76 +- tracing/sessions.mdx | 202 ---- tracing/signals.mdx | 199 ---- tracing/tagging.mdx | 93 -- 24 files changed, 2282 insertions(+), 2564 deletions(-) delete mode 100644 autotune/prompts/models.mdx create mode 100644 autotune/sdks/python.mdx create mode 100644 autotune/sdks/typescript.mdx delete mode 100644 autotune/setup.mdx create mode 100644 feedback/api-reference.mdx create mode 100644 feedback/introduction.mdx create mode 100644 feedback/python.mdx create mode 100644 feedback/typescript.mdx create mode 100644 tracing/api-reference.mdx create mode 100644 tracing/introduction.mdx delete mode 100644 tracing/manual-instrumentation.mdx delete mode 100644 tracing/quickstart.mdx delete mode 100644 tracing/reference.mdx delete mode 100644 tracing/sessions.mdx delete mode 100644 tracing/signals.mdx delete mode 100644 tracing/tagging.mdx diff --git a/autotune/introduction.mdx b/autotune/introduction.mdx index c10a81e..7c8911f 100644 --- a/autotune/introduction.mdx +++ b/autotune/introduction.mdx @@ -1,36 +1,43 @@ --- title: "Introduction" -description: "Run evaluations on models and prompts to find the best variants for your agents" +description: "Version, track, and optimize every prompt your agent uses" --- -Prompt optimization is a different approach to the traditional evals experience. Instead of setting up complex eval pipelines, we simply ingest your production traces and let you optimize your prompts based on your feedback. +Prompts are the instructions that drive your agent's behavior. Small changes in wording can dramatically affect output quality, but without tracking, you have no way to know which version works best -- or even which version is running in production. + +ZeroEval Prompts gives you version control for prompts with a single function call. Every change is tracked, every completion is linked to the exact prompt version that produced it, and you can deploy optimized versions without touching code. + +## Why track prompts + +- **Version history** -- every prompt change creates a new version you can compare and roll back to +- **Production visibility** -- see exactly which prompt version is running, how often it's called, and what it produces +- **Feedback loop** -- attach thumbs-up/down feedback to completions, then use it to [optimize prompts](/autotune/prompts/prompts) and [evaluate models](/autotune/prompts/models) +- **One-click deployments** -- push a winning prompt or model to production without redeploying your app ## How it works - - Replace hardcoded prompts with `ze.prompt()` calls in Python or `ze.prompt({...})` in TypeScript + + Swap string literals for `ze.prompt()` calls. Your existing prompt text becomes the fallback content. - - Each time you modify your prompt content, a new version is automatically created and tracked + + Each unique prompt string creates a tracked version. Changes in your code produce new versions without any extra work. - - ZeroEval automatically tracks all LLM interactions and their outcomes + + When your LLM integration fires, ZeroEval links each completion to the exact prompt version and model that produced it. - - Use the UI to run experiments, vote on outputs, and identify the best prompt/model combinations - - - Winning configurations are automatically deployed to your application without code changes + + Review completions, submit feedback, and generate improved prompt variants -- all from real traffic. +## Get started + - - Learn how to integrate ze.prompt() into your Python or TypeScript codebase + + `ze.prompt()` and `ze.get_prompt()` for Python applications - - Run experiments and deploy winning combinations + + `ze.prompt()` for TypeScript and JavaScript applications - diff --git a/autotune/prompts/models.mdx b/autotune/prompts/models.mdx deleted file mode 100644 index 4fa58a3..0000000 --- a/autotune/prompts/models.mdx +++ /dev/null @@ -1,10 +0,0 @@ ---- -title: "Models" -description: "Evaluate your agent's performance across multiple models" ---- - -