hum3/tree-sitter-yabnf
Watch
1
0
Fork
You've already forked tree-sitter-yabnf
 0
Code Issues 2 Pull requests Projects Releases Packages Wiki Activity Actions

YABNF grammar fails to parse UTF-8 chars in %s literals and escaped quotes in @pattern #2

New issue
Open
opened 2026-08-07 14:24:32 +00:00 by hum3 · 0 comments
hum3 commented 2026-08-07 14:24:32 +00:00
Owner
Copy link

Problem

The tree-sitter YABNF grammar cannot parse valid YABNF output from tree-sitter2abnf. The entire goluca grammar YABNF output parses as a single ERROR node.

Two specific issues:

1. UTF-8 characters in %s"..." literals

arrow = (%s"->" / %s"//" / %s"→" / %s">")

The multi-byte → (U+2192) inside %s"→" breaks parsing. Highlighting stops at this point.

2. Escaped double quotes inside @pattern() arguments

description = @pattern("\"[^\"]*\"")
option-key = (@pattern("\"[^\"]*\"") / @pattern("[a-z][a-z-]*"))

The \" escape sequences inside pattern strings are not handled correctly — the grammar treats them as string terminators.

Reproduction

tree-sitter2abnf path/to/tree-sitter-goluca/src/grammar.json > goluca.yabnf
# Parse goluca.yabnf with the tree-sitter-yabnf grammar — entire file is ERROR

Expected

The grammar should correctly parse both cases, enabling full syntax highlighting of YABNF files.

Migrated from Codeberg: originally #2, opened 2026-03-24.

## Problem The tree-sitter YABNF grammar cannot parse valid YABNF output from `tree-sitter2abnf`. The entire goluca grammar YABNF output parses as a single ERROR node. Two specific issues: ### 1. UTF-8 characters in `%s"..."` literals ```abnf arrow = (%s"->" / %s"//" / %s"→" / %s">") ``` The multi-byte `→` (U+2192) inside `%s"→"` breaks parsing. Highlighting stops at this point. ### 2. Escaped double quotes inside `@pattern()` arguments ```abnf description = @pattern("\"[^\"]*\"") option-key = (@pattern("\"[^\"]*\"") / @pattern("[a-z][a-z-]*")) ``` The `\"` escape sequences inside pattern strings are not handled correctly — the grammar treats them as string terminators. ## Reproduction ```bash tree-sitter2abnf path/to/tree-sitter-goluca/src/grammar.json > goluca.yabnf # Parse goluca.yabnf with the tree-sitter-yabnf grammar — entire file is ERROR ``` ## Expected The grammar should correctly parse both cases, enabling full syntax highlighting of YABNF files. --- *Migrated from Codeberg: originally [#2](https://codeberg.org/hum3/tree-sitter-yabnf/issues/2), opened 2026-03-24.*
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
v0.1.2
v0.1.1
v0.1.0
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
hum3/tree-sitter-yabnf#2
No description provided.