Parser gets stuck in an endless loop with an unmatched closing parenthesis

[LhKipp]

Describe the bug

Hi,

thank you for developing this tree-sitter grammar 🙏🏻 👍🏻 . It works great, except for one annoying case.

To Reproduce

The parser runs into an endless loop for the following code (note the unmatched closing ) after int:

type MyType =
    { v: int }

    member this.MemberFunc x : int) list = [ 1 ]

Debug logs return the following:

> npm run generate
> npx tree-sitter parse [path-to-bad-code] -d 2> debug_out.txt
<CTRL-C> # parse will never end as it runs into an endless loop
> cat debug_out.txt
...
recover_to_previous state:6095, depth:5
process version:1, version_count:3, state:5646, row:3, col:34
lex_internal state:158, row:3, column:34
  consume character:')'
lexed_lookahead sym:), size:1
reduce sym:long_identifier_or_op, child_count:1
reduce sym:identifier_pattern, child_count:1
reduce sym:_pattern, child_count:1
detect_error
process version:2, version_count:3, state:6095, row:3, col:35
lex_external state:43, row:3, column:35
  skip character:' '
lex_internal state:145, row:3, column:35
  skip character:' '
  consume character:'l'
  consume character:'i'
  consume character:'s'
  consume character:'t'
  consume character:'l'
lexed_lookahead sym:identifier, size:5
detect_error
resume version:0
skip_token symbol:)
process version:0, version_count:2, state:0, row:3, col:35
lex_external state:1, row:3, column:35
lexed_lookahead sym:_dedent, size:0
recover_to_previous state:6095, depth:5

The parser recovers to state 6095, runs into an error, recovers to state 6095, runs into an error ...

Expected behavior

The parser does not run into an endless loop.

** Environment (please complete the following information):**

OS: Linux
Grammar version: Commit ad824de

[LhKipp]

The issue seems to be with the error handling in scanner.h. The parser doesn't get stuck, when not returning DEDENT/PREPROC_END tokens in the error_recovery case.

@@ -132,18 +132,6 @@ static inline bool is_bracket_end(TSLexer *lexer) {
 
 static bool scan(Scanner *scanner, TSLexer *lexer, const bool *valid_symbols) {
   if (valid_symbols[ERROR_SENTINEL]) {
-    if (scanner->indents.size > 1) {
-      array_pop(&scanner->indents);
-      lexer->result_symbol = DEDENT;
-      return true;
-    }
-
-    if (scanner->preprocessor_indents.size > 0) {
-      array_pop(&scanner->preprocessor_indents);
-      lexer->result_symbol = PREPROC_END;
-      return true;
-    }
-
     return false;
   }