How this app works
When you use AI, you write instructions called a prompt. This app checks how good the answers to your prompt are, across several AIs and many examples, so you don't have to guess.
Your first test, step by step
For example: "Summarize this complaint in one sentence: {{input}}". The {{input}} part is where your example text goes.
Paste one real example, such as a customer email. In an evaluation, add many.
Ones marked "free" cost nothing. Comparing different companies is the most interesting.
Read each AI's answer, its grade from 1 to 10, and why it got that grade.
How to read the results
Grade (1–10)
7 or more counts as a pass. Green is good, amber is so-so, red is wrong or off-task. Read the "Why" line under each grade.
Speed and cost
How long each AI took, and what the answer cost. "free" means it used a free tier.
Better or worse
When you re-run an evaluation after editing your prompt, it lists which answers improved and which broke, side by side.
Tips for good results
- Give an ideal answer when you can. The grader is much more accurate with one.
- Say what "good" means, for example "one sentence, no invented facts". Put it in the grading criteria box.
- Test realistic and awkward examples: long, short, angry, or ones that try to trick the AI.
- Change one thing at a time, and keep the same test inputs, so before and after are comparable.
- Don't over-read one flipped answer. The grader varies a little. Look for patterns.
Glossary
- Prompt
- The instructions you give an AI, for example "Summarize this complaint in one sentence."
- Input
- The text the prompt works on, for example one customer email. Write {{input}} in your prompt to mark where it goes.
- Model
- A specific AI that answers your prompt, such as Gemini, GPT or Llama. Different models give different answers at different speeds and prices.
- Grader
- A second AI that reads each answer and scores it from 1 to 10, with a short reason. It saves you from reading every answer yourself.
- Ideal answer
- What a perfect answer looks like, if you know it. Giving one makes grading much more accurate.
- Pass mark
- The lowest score that counts as a good answer. It is 7 out of 10 by default.
- Dataset
- A list of test inputs, such as 30 real customer emails, saved so you can reuse it every time you change your prompt.
- Regression
- Something that used to work but broke after a change. This app lists exactly which answers got better and which got worse.
- Tokens
- The small chunks of text that AI providers count and charge for. 100 tokens is roughly 75 English words.
Questions
Do I need to know how to code?
No. If you can write instructions for an AI, you can use this. Type a prompt, pick some AIs, and press Run.
Is it free?
It can be. Free AI options (Google Gemini, Groq, OpenRouter's free models and Ollama on your own computer) are marked "free". Paid models show their price, and every result shows what it cost.
Which AI should I use?
That's exactly what this tool helps you decide. Run the same prompt on two or three models and compare the grades, speed and cost side by side.
Why should I test many inputs instead of one?
One good answer can be luck. Testing 10 to 30 realistic inputs shows how a prompt behaves in general, including the awkward cases.
Why did a grade change when I changed nothing?
The grader is an AI too, and it isn't perfectly consistent. One answer moving a point or two is normal. Look for patterns across several answers.
Is my data safe?
Your inputs are sent to the AI providers you pick. Some free tiers may use what you send to improve their models, so use made-up or anonymised examples there, or Ollama, which never leaves your computer.
Are quick tests saved?
No. Quick tests are for trying ideas. Use "Evaluate many inputs" when you want results saved and compared over time.
How do I connect a free AI?
Get a free key from Google AI Studio (aistudio.google.com/apikey) or Groq (console.groq.com/keys). Paste it into the backend settings file (backend/.env) as GEMINI_API_KEY=… or GROQ_API_KEY=…, then restart the backend. The README has full steps.