Voting on Jev's probabilities instead of on thresholded answers
Moneyby fibs2026-09-19
Voting on Jev's probabilities instead of on thresholded answers
Context
We develop a framework, polygent, for building deterministic companions for AI agents. A model interprets an artifact into a small typed vocabulary; everything downstream is a replayable check.
,
Our Guard polygent audits a coding agent's tool policy (for instance Claude Code settings + MCP servers). The one thing it asks a model: does this unknown MCP tool read untrusted input? send bytes out? read proprietary source?
,
That maps straight onto noul questions: 24 yes/no, one call, ~0.5 s.
,
The problem
Our vote was built for LLMs that write an answer: ask 3 times independently, take the majority per label. That's how you catch a model that isn't sure.
,
With Jev that logic breaks. Asked whether an internal sync tool reads proprietary source, three calls returned 0.50, 0.48, 0.45 (Jev has a little run-to-run jitte