How to properly type methods that return Generators with subclasses?

[DanielNoord]

I asked on gitter, but didn't get a response yet. Perhaps somebody here can help me. I have the following code:

from typing import Callable, Generator

class Parent:
    def simple_method(self) -> "Parent":
        ...

    def my_method(self, param: str) -> Generator["Parent", None, None]:
        ...

class Child(Parent):
    simple_method: Callable[["Child"], "Child"]
    my_method: Callable[["Child", str], Generator["Child", None, None]]

class OtherChild(Parent):
    simple_method: Callable[["OtherChild"], "Child"]
    my_method: Callable[["OtherChild", str], Generator["Child", None, None]]

The Callables are there because these methods are defined in another file which we can't really move just yet. However, to avoid mypy complaining about the attribute/method not being defined when we assign them in the other file we'd like to add Callable typing on the definition of the class.

For simple_method mypy allows returning Child in both Child and OtherChild as they are subclasses of Parent which is the return value of the parent class definition of the method. (Or that's how I interpret why this is allowed).
However, for my_method I get the following errors:
Incompatible types in assignment (expression has type "Callable[[str], Generator[Child, None, None]]", base class "Parent" defined the type as "Callable[[str], Generator[Parent, None, None]]")

I'm not sure why this isn't allowed. Isn't it the same as returning Child in simple_method? If not, please tell me what mypy is warning me about and what should be changed to allow this pattern.

[erictraut]

The problem is that Callable parameters are position-only, since they are nameless, but the method my_method that you are attempting to override accepts a parameter named param that can be specified as positional or as keyword. If you change my_method in Parent to accept position-only parameters, mypy will no longer generate an error.

def my_method(self, param: str, /) -> Generator["Parent", None, None]:
    ...

or

def my_method(self, __param: str) -> Generator["Parent", None, None]:
    ...

The error message provided by mypy is quite misleading and could be improved. By comparison, here's the error emitted by pyright:

Method "my_method" overrides class "Parent" in an incompatible manner
  Parameter 2 name mismatch: base parameter is named "param", override parameter is named "__p1"

--
Eric Traut
Contributor to Pyright & Pylance
Microsoft

[DanielNoord]

Thanks Eric for that explanation. I looked online and the only other option seems to be to define a Callable Protocol. I'd like to let the parent signature stay unmodified so, although good for future reference, sadly I can't use your solution.
Do you know if there is any discussion about a allowing specifying argument names in Callable? I saw that #264 was closed, but wondered if there are any WIP PEPs or discussion that I could start to follow.

[JelleZijlstra]

It came up in pre-discussions for PEP 677 but it didn't make it into the submitted version of that PEP.

[erictraut]

Yeah, you can use a callback protocol for these (relatively rare) situations where parameter names are required.