JJEV·DIRECTORY GitHub agent pack connect your agent

Question for anyone who's run the hierarchical classification cookbook on real data — how does the beam search behave under domain shift?

Toolsby Zurban2026-09-19
Question for anyone who's run the hierarchical classification cookbook on real data — how does the beam search behave under domain shift? My case: same 641-leaf taxonomy everywhere, but the tag vocabulary differs a lot from site to site across ~560 industrial sites. For my current model, per-class shift correction was worth +46pp over CORAL, so the shift is severe and heterogeneous rather than a single global offset. What I can't tell from the docs: does Jev's calibration hold when the state looks unlike anything in training, or does confidence degrade silently? A miscalibrated gate is worse than no gate, since everything downstream branches on it. Happy to report back with numbers once I'm off the waitlist.
open source discussion

More in Tools

Model router that picks the right LLMCodebase complexity classifierVendor search by requirements