Eval Table
A model evaluation comparison table with prompts as rows and model outputs as columns, pass/fail and score badges, aggregate pass-rate bars, a word diff between two models and a failures-only filter.
Preview
Support assistant · regression set
5 prompts · 3 models
- Pebble 3Top100%5/5 · avg 0.94
- Pebble Mini60%3/5 · avg 0.64
- Pebble 70B60%3/5 · avg 0.69
5 of 5 shown
| Prompt | Pebble 3 | Pebble Mini | Pebble 70B |
|---|---|---|---|
#01Convert 3.5 cups of flour to grams.Expect: About 440 g (125 g per cup) | Pass0.96 3.5 cups of all-purpose flour is about 440 grams, at 125 g per cup. | Pass0.81 Changes against Pebble 3: 3.5 cups of | Fail0.32 That's roughly 500 grams of flour. |
#02Summarise the return policy in one sentence.Expect: 30 days, unused, receipt required | Pass0.94 You can return unused items within 30 days if you have the receipt. | Fail0.55 Changes against Pebble 3: You can return | Pass0.90 Unused items can be returned within 30 days with a receipt for a full refund. |
#03Reply to a customer whose parcel arrived damaged.Expect: Apologise, offer replacement or refund, no blame | Pass0.92 I'm so sorry your parcel arrived damaged. I can send a replacement today or refund you in full, whichever you prefer. | Pass0.86 Changes against Pebble 3: | Fail0.41 Sorry about that. Couriers can be rough with parcels. Let us know if you want a refund. |
#04Extract the due date from: “Payment is due 14 days after the 2 March invoice.”Expect: 16 March | Pass1.00 The payment is due on 16 March. | Fail0.10 Changes against Pebble 3: The payment is due on | Pass0.98 Payment is due on 16 March. |
#05Name three hardy herbs for a shady balcony.Expect: Any three of mint, parsley, chives, lemon balm | Pass0.90 Mint, parsley and chives all cope well with partial shade. | Pass0.88 Changes against Pebble 3: Mint, parsley and | Pass0.85 Mint, chives and lemon balm do fine with a few hours of light. |
Install
Pro source requires Pebble Pro. Try the full preview above.
Props
| Prop | Type | Default | Description |
|---|---|---|---|
| models | EvalModel[] | — | Columns: { id, name }. Colours follow the chart tokens in order. |
| cases | EvalCase[] | — | Rows: { id, prompt, expected?, results }. results maps model id to { output, pass, score, reason? }. |
| defaultCompare | [string, string] | first two models | Base (A) and comparison (B) model ids for the diff. |
| defaultDiff | boolean | false | Start with the word diff shown in the B column. |
| defaultFailuresOnly | boolean | false | Start filtered to rows with at least one failure. |
| title | string | "Evaluation" | Heading above the aggregate scores. |
Pro component
Try the full preview above. Editable source and build guides require Pebble Pro.