Conversation
|
simolus3
left a comment
There was a problem hiding this comment.
This is really neat, it's nice that we're able to find parser bugs this way.
I can take a look at remaining issues for the new compiler.
| - This grammar is used when config.sync_config_compiler = true (and edition >= 2) | ||
| */ | ||
|
|
||
| SyncStreamsCompilerSql ::= CompilerStreamQuery | CompilerSubquery | CompilerCteSubquery |
There was a problem hiding this comment.
Should we remove CompilerSubquery here? It's not a top-level block that could be parsed, right?
This adds grammar for our supported SQL syntax, in W3C EBNF syntax. The idea is to have a formal reference on what SQL syntax we support.
The goal is to eventually have public documentation on the exact syntax we support, similar to the SQLite documentation. The grammar in the current format isn't quite ready to use as-is for that, but it's a step in that direction.
These grammar definitions are not used anywhere in the actual code - this is just for documentation and validation.
This grammar definitions is split between our three different parsers:
To confirm that our implementation matches the grammar, this adds tests that runs queries through both our parser and a parser generated purely from the grammar. These do not test the behavior/output of the queries at all - just checks whether the queries pass the parsing stage or not.
The test queries fall into three different categories:
We don't currently distinguish between syntax and semantic errors in our parser, so we can't test "rejected_semantic" actually fails with a semantic error rather than syntax error, but it's good enough for now.
Note: The majority of the grammar syntax and test fixtures were generated using Codex.
Fixes
The tests picked up an error in the parsing of BETWEEN statements and in
!=operators in the new compiler, due to the location not being set. This fixes it.There is a pending issue still -
NOT IN [json string]works in the alpha sync streams parser, but not the new compiler.