Skip to content

Conversation

alfonsoSR
Copy link
Contributor

@alfonsoSR alfonsoSR commented Oct 3, 2025

Rationale

The goal of this PR is to replace mypy.stubgen and pybind11-stubgen with custom stub generators for Python and C++ extensions.

Bad features of current stub generator

  • pybind11-stubgen is not particularly well maintained, it does not have documentation, it is not good at processing enumerations, and it produces rough stubs that require significant post-processing.
  • mypy.stubgen is very slow, and the stubs it produces also require a lot of post-processing

Nice features of the new stub generators

  • We can discourage users from using deprecated functions/classes/modules by not generating stubs for them. This means that deprecated code still works, but it no longer gets syntax highlighting, so new people will not find it.
  • We can raise custom warnings to help us find code that does not follow some rule. For example, we can raise a warning when a function does not have documentation, or when it has C++ types in its signature.
  • We can use a logger to control what gets printed to the terminal while generating the stubs. For example, we can use log.warning for warnings that matter to users, and log.debug for information that is only relevant to developers, and then decide to only print developer information if we call the build.py script with the --debug flag.
  • Instead of having to define post-processing functions to fix the stubs produced by pybind11-stubgen, we can adapt the stub generators to our needs and simplify the implementation.

TODO

- Replaced stubgen with our own stub generator: much faster, more
  control, and much simpler
- Introduced logger to color stdout (e.g. info, warning...) and be able
  to hide pybind11-stubgen "errors" from users (--debug flag)

BACKWARDS COMPATIBLE
Initial implementation. Requires polishing and some refactoring

- Implemented a logger to have more control over stdout
- Implemented statement processors to generate stubs from Python code
  using the ast module
- Replaced the use of mypy.stubgen with our custom stub generator
First working iteration of a custom stub generator for C++ extension
modules, which is meant to replace pybind11-stubgen. The previous stub
generation setup is still used because the implementation of the new one
is still not robust.
@alfonsoSR alfonsoSR self-assigned this Oct 3, 2025
@alfonsoSR alfonsoSR added priority: low Nice to have or long-term consideration type: enhancement Improvements to existing functionality labels Oct 3, 2025
@alfonsoSR alfonsoSR moved this to In Progress in Tudat development board Oct 3, 2025
@DominicDirkx DominicDirkx removed the priority: low Nice to have or long-term consideration label Oct 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: enhancement Improvements to existing functionality

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

[F&E]: Module level docstrings through stubs [BUG]: Missing types during stub generation [BUG]: Exposing functions with nullptr as default argument

2 participants