Your grammar can be wrong and still compile

Constrained decoding guarantees your model conforms to the grammar as written. It doesn't guarantee the grammar says what you meant. And it does nothing at all for text produced without it.

GBNF is the notation llama.cpp uses for constrained decoding. At each step the sampler masks every vocabulary token that would leave the grammar's language, so the model physically can't emit malformed output. llama.cpp reads it, and so do XGrammar (and therefore vLLM and SGLang), KoboldCpp, LocalAI and node-llama-cpp.

In every one of those the grammar runs inside generation and nowhere else. None of them can answer “does this string match my grammar?” without spinning up a model. The ecosystem's one offline checker, llama.cpp's llama-gbnf-validator, is a C++ example binary that answers accept or reject and nothing else: no library form, no AST, no structured errors.

That gap is what this fills.

Try it here

Both boxes below are editable and the verdicts re-run as you type. The grammar is compiled by the same package you would install, running in this page.

Every character counts, because GBNF is scannerless: yes  (90%) is rejected for the second space. Break the grammar itself and you get a compile error in place of verdicts, which is the distinction the tool exists to keep. Widen it and watch a rejected sample turn green.

Start here

Give it a grammar and a sample. The exit code alone is the answer, so this drops straight into a shell script or a CI step:

# npm i -g @tabnas/gbnf
$ gbnf-check json.gbnf --text '{"a": 1}' --text '{"a": 1,}'
grammar json.gbnf: ok
sample text#0: accept
sample text#1: REJECT — [tabnas/unexpected]: unexpected character(s): , (line 1, column 8)
$ echo $?
1

From Python, where models are usually driven:

import gbnf

with gbnf.Grammar.from_file("json.gbnf") as g:
    g.accepts('{"a": 1}')      # True
    g.accepts('{"a": 1,}')     # False

    v = g.check('{"a": 1,}')
    v.accept                   # False
    v.error["message"]        # '[tabnas/unexpected]: unexpected character(s): ,'

From TypeScript and Go, the same grammar on a parser instance:

const { Tabnas } = require('@tabnas/parser')
const { gbnf } = require('@tabnas/gbnf')

const tn = new Tabnas({ plugins: [gbnf] })
tn.gbnf(src)

tn.parse('{"a": 1}')  // => the AST

tn := tabnas.Make()
if _, err := gbnf.Install(tn, src, nil); err != nil {
    // the grammar is broken
}

if _, err := tn.Parse(`{"a": 1}`); err != nil {
    // outside the language, err says where
}

Four things you'd use it for

The first is the one people underestimate. A grammar is code. It can be wrong in ways that never throw. Nothing else in the ecosystem lets you assert what it accepts:

def test_response_grammar():
    with gbnf.Grammar.from_file("response.gbnf") as g:
        for good in GOLDEN_OUTPUTS:
            assert g.accepts(good)      # the grammar admits what you expect
        for bad in NEAR_MISSES:
            assert not g.accepts(bad)  # and excludes what it should

Both directions matter. A grammar that accepted everything would pass a suite that only ever checked the happy path. That's why this repo's own corpus is graded in both directions.

Three outcomes, not two

A rejection is an answer, not a failure. An out-of-language string comes back as a falsy verdict and never raises. Invalid calls and operational failures do raise, as they should: a grammar that won't compile, a closed handle, a bad argument type, an unreadable file, a shared library that can't be found.

Collapsing the first two is the mistake worth avoiding. It tells you your model's output was wrong when the real problem was your grammar. So you go and debug the wrong thing.

The report is machine-readable

--json writes a stable document to stdout, in the shape tooling and agents consume, so nothing has to parse prose:

$ gbnf-check json.gbnf --text '{"a": 1,}' --json
{
  "tool": "gbnf-check",
  "version": "0.1.4",
  "ok": false,
  "exit": 1,
  "grammar": {
    "source": "json.gbnf",
    "ok": true,
    "error": null
  },
  "samples": [
    {
      "source": "text#0",
      "ok": false,
      "length": 9,
      "error": {
        "name": "SyntaxError",
        "message": "[tabnas/unexpected]: unexpected character(s): ,",
        "code": "unexpected",
        "line": 1,
        "column": 8
      }
    }
  ],
  "caveats": [
    {
      "code": "rejection-may-be-engine-limit",
      "message": "a rejection is this engine's answer, not always the grammar's: a grammar that needs backtracking can reject here and still constrain a sampler correctly",
      "url": "https://github.com/tabnas/gbnf/blob/main/ts/doc/known-gaps.md"
    }
  ]
}

That caveats entry is deliberate, and it's worth reading. A rejection here is this engine's answer, not always the grammar's: a grammar that needs backtracking can reject offline and still constrain a sampler correctly. The report says so, rather than let you decide your output was invalid.

Compile once, validate anywhere

Your inference workers don't need the GBNF front-end at all. Compile the grammar at build time into a pure-data recognition spec, ship that, and check against it with the engine alone:

spec = gbnf.compile_spec(open("json.gbnf").read(), as_text=True)
# ship `spec` to a service that has the engine and has never heard of GBNF

So how does it read the notation?

Skip ahead if you only wanted the tool. This part matters if you care whether the answers can be trusted.

GBNF is scannerless. Its grammar describes the input one character at a time, while the engine underneath is a tokenising parser that ships JSON-shaped matchers and an ignore set. Left alone, root ::= "a" would happily accept " a ", and a # in your input would vanish as a comment.

So the compiled grammar carries an empty ignore set and switches every default matcher off. That configuration is part of the accepted language. It looks like a performance knob and it isn't one. That's why dropping it from the serialized form was a correctness bug.

The front-end that reads GBNF is itself a tabnas grammar: a rule table the engine runs, in both TypeScript and Go, rather than a hand-written parser in each. One definition of the notation. And the engine proves itself on its own front-end.

What it's graded against

• All eight grammars from llama.cpp's grammars/ directory, copied verbatim, compiled and parsed in both directions: accept and reject.
• All seventy expected outputs of llama.cpp's JSON-schema-to-grammar converter, sampled in both directions.
• The eight grammars and their samples are graded in all three runtimes: TypeScript, Go and Python. So if they disagree, the runtimes disagree. It isn't one binding going wrong. The seventy live grammars are graded in full by TypeScript and compiled, not sampled, by Go. Python doesn't run them at all.

The corpus grammars are upstream bytes, never tidied to make a test pass. Exactly one sample is recorded as an expected failure: chess.gbnf with Nf3, which needs backtracking. If it ever starts working the suite goes red, because an expected failure that quietly turns into a pass is a documentation bug.

What it won't do

• It's not a sampler. It won't make your generation constrained and it won't make your model faster. llama.cpp still does that. This answers questions either side of it.
• It's not a reason to adopt GBNF. If you're not already using GBNF, this changes nothing for you.
• A rejection isn't always the grammar's verdict. A grammar that needs backtracking can reject here and still constrain a sampler correctly. The JSON report flags it.
• And a pass isn't proof that llama.cpp will accept it. Rule boundaries here use a two-token NM ::= lookahead. That's exact, but it doesn't care about line breaks. llama.cpp does: it ends a rule at a top-level newline. So this accepts a superset. A grammar that runs on across lines without | can pass here and still be refused by the sampler. Run llama-gbnf-validator as well if that's the contract you need.
• Tokenizer-token terminals are refused, on purpose. <think>, <[1000]>, !</think> and friends are sampler-level: what they mean depends on a model's tokenizer, and an offline checker hasn't got one. They parse, so the error can name the rule, and then compilation fails rather than guessing they were literal text and quietly changing the accepted language.
• The Python package isn't on PyPI yet. There's no pyproject.toml either, so import gbnf needs py/ on PYTHONPATH once you've built the shared library. A wheel matrix is the obvious next step and it isn't done yet.

Where the rest of it lives

Install

# the CLI and the TypeScript library
npm i @tabnas/gbnf

# the Go module
go get github.com/tabnas/gbnf/go

# the Python binding: build the shared library, then put py/ on the path
# (no packaging yet – see “what it won't do”)
git clone https://github.com/tabnas/gbnf && cd gbnf
(cd go/clib && ./build.sh)
PYTHONPATH=$PWD/py python3 -c 'import gbnf; print(gbnf.version())'