Jev is a judge model. It doesn't talk to your customers — it grades the AI that does. You hand it a record of what happened plus the questions you care about, and it answers them in a shape your code can use: true/false, one label, or a score.
You ask "did the agent state the refund amount?" and get prose back. Sometimes it starts with "Yes" — sometimes with "The agent did state $42.50, however…".
Now you need a parser, and a second model to grade the parser.
You declare each question's type once. Jev returns the verdict in exactly that shape — true, one label from your list, or a number.
No prose to parse, no second opinion to reconcile.
| Type | Returns | Example rule |
|---|---|---|
| boolean | true / false | True only if the agent stated the exact refund amount before asking to proceed. |
| choice | one label | Classify the ticket: billing, shipping, or account. |
| score | a number | Rate the apology 1–5 for empathy and ownership. |
The state is whatever happened. The questions are what you want to know. Both are just text and types — this is one complete, runnable request:
import { experimental_evaluate } from 'ai';
const result = await experimental_evaluate({
model: 'typesafe-ai/jev',
state: 'Customer: my order arrived damaged, I want a refund.\n' +
'Agent: I can refund order #88121 in full: $42.50 back to your ' +
'Visa ending 4412 within 3-5 business days. Shall I go ahead?',
questions: {
amount_disclosed: {
type: 'boolean',
instructions: 'True only if the agent stated the exact refund amount ' +
'($42.50) before the customer approved.'
},
intent: {
type: 'choice',
criteria: { billing: 'money or charges', shipping: 'delivery timing', account: 'login or profile' },
instructions: 'Route this ticket.'
},
empathy: {
type: 'score',
criteria: ['1 - none', '3 - acknowledged the problem', '5 - owned it and fixed it'],
instructions: 'Rate how the agent handled the complaint.'
}
}
});
// → { amount_disclosed: true, intent: 'billing', empathy: 3 }
Jev runs as typesafe-ai/jev through the Vercel AI Gateway, called with experimental_evaluate from the AI SDK. Set AI_GATEWAY_API_KEY in your environment — never paste the key into code.
typesafe-ai/jev, called through the Vercel AI Gateway.AI_GATEWAY_API_KEY in the environment. Nothing to train, nothing to host, no weights to babysit.