Opik Agent Optimization is a suite of tools designed to improve prompts and agents, you can read more about it in the Opik Agent Optimization documentation.
In this repository we are going to focus on optimizing real-world prompts and agents rather than academic benchmarks. The goal is to help you find similar projects to what you are working on to better understand how Agent Optimization can help.
Note
We are continously adding new benchmarks, if you have a project you would like us to add just create a Github issue with a link to the project and we will take a look.
Current projects:
Convex evals: Convex is an open-source, reactive database that's the best platform for full-stack AI coding. These evals are used to tailor the Convex AI rules you can find in their documentationJSON Schema: LangChain includes a prompt to generate JSON outputs that follow a specific schema that can be used for models that don't support tool calls or structured outputs natively. We optimize the prompt using JSON Schema Bench that is a real world dataset of JSON schemas