Summary Jev Rerank Experiment for code context The eval: ReadyBase ( - my tool to provide meaningfully viable per turn
Moneyby PromptForcePrime2026-09-16
Summary Jev Rerank Experiment for code context
The eval: ReadyBase (https://www.promptforce.ai/readybase - my tool to provide meaningfully viable per turn context to coding agents) already has a 28-scenario gold-labeled regression harness each scenario is a real file + a real task description, with a hand-authored list of which facts (dependencies, size/fragility warnings,ownership, tests, etc.) an engineer would actually need to see before making that change, some marked critical. We reused it as-is: no new labels, no synthetic data.
Baseline = readybase context builds a candidate pool of structural facts about the file, then its hand-tuned ranker (weightScaleByKind) scores and picks which ones fit the token budget. This is what ships in production. On the 28 scenarios: 72.6% recall, 95.2% critical-fact retention.
Plain Jev (as a reranker): same candidate pool, but ranked by one TypeSafe choice API call per scenario asking
Links