grammarinator-process fails for SQLite and MySQL grammars

[mrigger]

I just tried grammarinator on the SQLite and MySQL grammars from https://github.com/antlr/grammars-v4/tree/master/sql (commit 8dca3622acbea8fce8726c73364af232cb6eacce), but for both of them, the latest version of grammarinator-process failed.

grammarinator-process SQLiteLexer.g4 SQLiteParser.g4 -o out/
Traceback (most recent call last):
  File "/home/manuel/.local/bin/grammarinator-process", line 8, in <module>
    sys.exit(execute())
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 743, in execute
    FuzzerFactory(args.language, args.antlr, args.out).generate_fuzzer(args.grammar, options=options, encoding=args.encoding, lib_dir=args.lib, actions=args.actions, pep8=args.pep8)
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 650, in generate_fuzzer
    graph = build_graph(actions, lexer_root, parser_root)
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 604, in build_graph
    graph.calc_min_depths()
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 236, in calc_min_depths
    assert all(min_depths[node.id] < inf for node in self.vertices[ident].out_neighbours), '{ident} has an alternative that isn\'t reachable.'.format(ident=ident)
AssertionError: 739 has an alternative that isn't reachable.

grammarinator-process MySqlLexer.g4 MySqlParser.g4 
Traceback (most recent call last):
  File "/home/manuel/.local/bin/grammarinator-process", line 8, in <module>
    sys.exit(execute())
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 743, in execute
    FuzzerFactory(args.language, args.antlr, args.out).generate_fuzzer(args.grammar, options=options, encoding=args.encoding, lib_dir=args.lib, actions=args.actions, pep8=args.pep8)
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 650, in generate_fuzzer
    graph = build_graph(actions, lexer_root, parser_root)
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 602, in build_graph
    build_rules(root)
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 584, in build_rules
    build_rule(*rule_args)
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 529, in build_rule
    build_expr(node, rule.id)
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 524, in build_expr
    build_expr(child, parent_id)
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 411, in build_expr
    build_expr(child, alternative_id)
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 415, in build_expr
    build_expr(node.alternative(), parent_id)
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 428, in build_expr
    build_expr(child, parent_id)
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 456, in build_expr
    build_expr(node.children[0], quant_id)
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 524, in build_expr
    build_expr(child, parent_id)
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 524, in build_expr
    build_expr(child, parent_id)
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 400, in build_expr
    build_expr(children[0], parent_id)
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 428, in build_expr
    build_expr(child, parent_id)
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 447, in build_expr
    build_expr(node.children[0], parent_id)
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 509, in build_expr
    build_expr(child, parent_id)
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 513, in build_expr
    graph.add_edge(frm=parent_id, to=str(node.TOKEN_REF()))
  File "/home/manuel/.local/lib/python3.7/site-packages/grammarinator/process.py", line 215, in add_edge
    assert to in self.vertices, '{to} not in vertices.'.format(to=to)
AssertionError: ADMIN not in vertices.

Is this expected?

[renatahodovan]

Hi @mrigger!

After investigating the processed grammars, it seems that both of them are buggy:

1. SQLiteParser contained a typo in the order_by_expr_asc_desc rule: instead of using expr_asc_desc it referred to itself, causing an infinite recursion issue. Although I'm not an SQLite expert, but it seems - at least from the ordering of the rules - that it was not the original intention (not mentioning that expr_asc_desc is not used in the current codebase at all). I've created a PR to fix this issue.

2. MySqlParser used two undefined tokens: ADMIN and OPTIONAL. I've created a fix for them too.

With the patches above the fuzzer generation should run without any errors.

Please let me know if it worked (or not, ofc).

Cheers,
Reni

[renatahodovan]

@mrigger Please note, that the SQLite grammars in the ANTLR grammars-v4 repository possibly have issues - that were mentioned in the PR too - that can negatively influence the effectiveness of test generation. For example, since window functions, aggregate functions, etc., are unreachable, they won't be generated and tested by Grammarinator either.

[renatahodovan]

@mrigger Is this working now with the latest grammar files? Can we close this issue?

[mrigger]

Thanks a lot for your investigation, explanation, and fixes to the grammar, I really appreciate it! I haven't had time yet to re-check, but it is on my TODO list, and I'll reopen the issue or create a new one if necessary. Thanks again!