Feature
Test your AI agents before your customers do
Build a set of real questions, say what a good answer looks like, and run it on each version. You see what passed, what failed and what changed, before anything goes live.
Create my first agentWhy evaluate your agents
Real questions, expected behaviour
For each question, describe the expected answer and what the agent should do: call a tool, consult the knowledge base, hand over to a person.
Graded automatically
Each answer is graded against your criteria by a judge model, with the reason for every verdict.
Compare two runs
Put two runs side by side to see which questions improved or regressed between two versions or two models.
Costs under control
Set a token budget for a run; it stops cleanly when the budget is reached.
Evaluate an agent in three steps
Write the test set
Add the questions your users really ask, with the answer and the behaviour you expect.
Run it
Launch the set on the draft or a published version and follow each case as it is graded.
Compare and fix
Compare with the previous run, adjust the instructions or the knowledge, and run again.
When teams run evaluations
Before publishing
Check a new version answers at least as well as the one in production.
When changing model
Measure whether a cheaper or newer model keeps the same quality on your questions.
After updating knowledge
Make sure new documents did not break answers that used to be right.
Frequently asked questions
What does an evaluation check?
The answer itself and the behaviour: which tools were called, whether the knowledge base was consulted, whether the conversation was handed over, which rules were triggered.
Who grades the answers?
A judge model compares each answer to what you expected and explains its verdict. You can read every case.
Does running an evaluation cost money?
It uses your AI provider key like any conversation. A token budget caps how much a run can spend.
Can I compare two models?
Yes. Run the same test set on two versions that use different models, then compare the two runs.
Ready to put your first agent to work?
Create your account, describe what your agent should do and publish it on your first channel today.
Create my first agent