[Python] Regression: Descriptions consisting of just a comment in a Scenario or Scenario Outline block throws error

[jsa34]

👓 What did you see?

This commit: #334
Appears to have broken a number of unit tests for pytest-bdd due to the gherkin parser throwing errors on parsing feature files.
The cause is that the parser throws an error if the description if just a comment inside a Scenario or Scenario Outline. It appears this was a missed test case scenario in the PR.

See CI job https://github.com/pytest-dev/pytest-bdd/actions/runs/13252365760/job/36993581825?pr=765 for errors.

Backtrace example:

test_scenario_comments.py:5: in <module>
    @scenario("comments.feature", "Comments")
/pytest-bdd/src/pytest_bdd/parser.py:480: in parse
    gherkin_doc: GherkinDocument = self._parse_feature_file()
/pytest-bdd/src/pytest_bdd/parser.py:476: in _parse_feature_file
    return get_gherkin_document(self.abs_filename, self.encoding)
/pytest-bdd/src/pytest_bdd/gherkin_parser.py:316: in get_gherkin_document
    gherkin_data = Parser().parse(feature_file_text)
/.venv/lib/python3.10/site-packages/gherkin/parser.py:96: in parse
    state = self.match_token(state, token, context)
/.venv/lib/python3.10/site-packages/gherkin/parser.py:231: in match_token
    return state_map[state](token, context)
/.venv/lib/python3.10/site-packages/gherkin/parser.py:795: in match_token_at_11
    self.end_rule(context, 'Description')
/.venv/lib/python3.10/site-packages/gherkin/parser.py:120: in end_rule
    self.handle_ast_error(context, rule_type, self.ast_builder.end_rule)
/.venv/lib/python3.10/site-packages/gherkin/parser.py:2800: in handle_ast_error
    self.handle_external_error(context, True, argument, action)
/.venv/lib/python3.10/site-packages/gherkin/parser.py:2813: in handle_external_error
    return action(argument)
/.venv/lib/python3.10/site-packages/gherkin/ast_builder.py:54: in end_rule
    self.current_node.add(node.rule_type, self.transform_node(node))
/.venv/lib/python3.10/site-packages/gherkin/ast_builder.py:272: in transform_node
    last_non_empty = next(
E   StopIteration

Feature file examples:

Feature: Comments
  Scenario: Comments
    # Comment
    Given I have a bar

Feature: Outline Test

  Scenario Outline: Outline example
    # Just print the string so that we can assert later what it was by reading the output
    Given I print the <string>

    Examples:
      | string       |
      | bork         |

✅ What did you expect to see?

No errors with comments in Scenarios or Scenario Outlines!

📦 Which tool/library version are you using?

[email protected]

🔬 How could we reproduce it?

1. Create a feature file with a comment inside the scenario, starting with a '#' char (pythonic comment syntax)
2. Parse the feature file using: Parser().parse(feature_file_text)

📚 Any additional context?

No response