Error when file has end-spec string (`]]]`) somewhere outside the cog spec

[ro-oliveira95]

Hi,

First of all, thanks for this super useful lib!

So I caught a bug while using cog in a python module related to the end-spec string (by default ]]]) being present in some part of the file outside the cog spec block.

In my case I had a python type annotation with four levels of nesting, something like var: list[list[tuple[int, int]]], somewhere else in the file that had the cog spec, and because of that type annotation cog was not able to run, with the following error message:

my_module.py(2): Unexpected ']]]'

The workaround for me in that case was to split that nested type annotation into multiple types aliases, but that was far from ideal.

I did look at the cog code and the fix seems very straightforward (see "possible fix" section below). If confirmed, I would be happy to submit a PR with it.

How to reproduce

1. Create a file with a cog spec + some occurrence of the end-spec string (by default ]]])

def my_func(param: list[list[tuple[int, int]]]) -> None:  # <-- end-spec string here
    ...


# [[[cog
#   import cog
#   fnames = ['DoSomething', 'DoAnotherThing', 'DoLastThing']
#   for fn in fnames:
#       cog.outl("void %s();" % fn)
# ]]]
# [[[end]]]

2. Run cog, it will fail

$ cog my_module.py

my_module.py(2): Unexpected ']]]'

cog version: 3.5.1
Python version: 3.10.13

Possible fix

My guess is that the check that currently is triggering the error should not exist. It is this one:

cog/cogapp/cogapp.py

Lines 449 to 454 in 0ff1d7c

	if self.is_end_spec_line(line):
	raise CogError(
	f"Unexpected {self.options.end_spec!r}",
	file=file_name_in,
	line=file_in.linenumber(),
	)

I think it should not be there because at that point the current line is not inside a cog-spec block, so actually any occurrence of an end-spec string in that line (as in my case) should be allowed.

[nedbat]

I'm interested to see how you are using cog in a Python file. I've never had need, since Python's introspection usually solves those problems for me.

I guess it seems a bit obsessive to raise an error if we find the end spec outside a cog block... Hmm.

[ro-oliveira95]

I'm interested to see how you are using cog in a Python file. I've never had need, since Python's introspection usually solves those problems for me.

Some use cases I stumbled on:

• generating boilerplate code, when code reuse in the classical form (inheritance, composition) or dynamically achieved (with introspection or metaprogramming) doesn't fit well, mainly to avoid overly complex or untyped code
• managing imports of binded C++ code
• keeping things in sync and/or explicit and under version control, like updating the list of parameters of a test when new test files are added -- this was my original use case, though it is more rare because usually its ok to just call the generator function at runtime

In fact, cog is part of our build pipeline, both locally and in CI.

I guess it seems a bit obsessive to raise an error if we find the end spec outside a cog block... Hmm.

Yeah I guess so, because otherwise it would be doing assumptions on non-cog code, as in my case.

The simple solution of removing the is_end_spec_line check was OK in my manual tests and at surface seems fine to me, but I wonder, can you think in possible cases in which this could break cog's parsing logic?