Locally-hosted open models frequently botch function calls in ways cloud APIs hide. This post diagnoses every failure mode and walks through the concrete reliability stack — constrained decoding, correct chat templates, validate-retry loops, and measurement — that actually fixes it.
Structured-Output
-
Making Tool Calling Reliable on Local Models -
Constrained Generation: Outlines, JSON Mode, and Structured Output That Works How regex-constrained sampling and grammar-guided decoding actually work at the token level, why prompt-only JSON mode fails 5–20% of the time in production, and the full toolchain for guaranteed structured output: Outlines, XGrammar, vLLM guided decoding, and Instructor.