Extend embedded ABNF grammar to parse tree-sitter2abnf @ annotations #4

New issue

Open

opened 2026-08-07 14:23:20 +00:00 by hum3 · 0 comments

hum3 commented

2026-08-07 14:23:20 +00:00

Owner

Problem

The embedded tree-sitter ABNF grammar (from upstream tree-sitter-abnf) only handles standard RFC 5234/7405 ABNF. It cannot parse the @ extension annotations emitted by tree-sitter2abnf (@token, @pattern, @prec, @field, @grammar, @extras).

This breaks the development feedback loop: grammar → tree-sitter2abnf → ABNF text → tree-sitter ABNF parser → syntax highlighting. The ABNF parser returns a fully-errored tree, so highlighting produces 0 ranges and the output renders as plain text.

Evidence

entry := grammars.DetectLanguage(".abnf")
parser := gotreesitter.NewParser(entry.Language())
tree, _ := parser.Parse([]byte(generatedABNF))
tree.RootNode().HasError() // true — entire tree is ERROR

Simple standard ABNF like foo = "bar"\n parses and highlights correctly.

Requested

Extend the ABNF grammar to handle the @ annotations so that tree-sitter2abnf output parses and highlights correctly. The formal spec for the extension syntax is being defined in hum3/tree-sitter2abnf#2.

This likely means forking or extending the upstream tree-sitter-abnf grammar and embedding the extended version.

Highlight query additions

Once the grammar handles the extensions, the highlight query in abnf_register.go will need entries for the new node types (e.g. annotation names as keywords, pattern strings as strings).

Migrated from Codeberg: originally #4, opened 2026-03-18.

## Problem The embedded tree-sitter ABNF grammar (from upstream tree-sitter-abnf) only handles standard RFC 5234/7405 ABNF. It cannot parse the `@` extension annotations emitted by [tree-sitter2abnf](https://codeberg.org/hum3/tree-sitter2abnf) (`@token`, `@pattern`, `@prec`, `@field`, `@grammar`, `@extras`). This breaks the development feedback loop: grammar → tree-sitter2abnf → ABNF text → tree-sitter ABNF parser → syntax highlighting. The ABNF parser returns a fully-errored tree, so highlighting produces 0 ranges and the output renders as plain text. ## Evidence ```go entry := grammars.DetectLanguage(".abnf") parser := gotreesitter.NewParser(entry.Language()) tree, _ := parser.Parse([]byte(generatedABNF)) tree.RootNode().HasError() // true — entire tree is ERROR ``` Simple standard ABNF like `foo = "bar"\n` parses and highlights correctly. ## Requested Extend the ABNF grammar to handle the `@` annotations so that tree-sitter2abnf output parses and highlights correctly. The formal spec for the extension syntax is being defined in [hum3/tree-sitter2abnf#2](https://codeberg.org/hum3/tree-sitter2abnf/issues/2). This likely means forking or extending the upstream tree-sitter-abnf grammar and embedding the extended version. ## Highlight query additions Once the grammar handles the extensions, the highlight query in `abnf_register.go` will need entries for the new node types (e.g. annotation names as keywords, pattern strings as strings). --- *Migrated from Codeberg: originally [#4](https://codeberg.org/hum3/gotreesitter/issues/4), opened 2026-03-18.*

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

hum3/gotreesitter#4

No description provided.

Rows
Columns