Description
Hi,
One thing that I have to patch in my local install of dbt is {{ debug() }}, which allows me to use any debugger I want with {{ debug() }}, e.g it'll work with PyCharm, VS Code, PyDev, etcetc.
Right now, in base.py, it basically only supports ipdb, a simple change would also be supporting other methods of debugging by generic means, such as breakpoint().
Options
breakpoint()
This would just call breakpoint() instead of ipdb, which would allow pretty much any debugger to attach.
PYTHONBREAKPOINT Environment Variable
Using the PYTHONBREAKPOINT environment variable would allow custom breakpoints.
Examples: https://github.com/search?q=PYTHONBREAKPOINT%3D+language%3APython+&type=code
Debugger
I started working on the idea of a dbt debugger a while ago, which basically extended {{ debug() }} to be way more useful, as debugging complicated macros is a PITA, to put it lightly. I kind of lost steam, because trying to get certain changes in dbt is an uphill battle.
This would have allowed you to get a nicer view when you are debugging via a debugger, such a fixed frame with relevant variables/project information, or even export the entire stack to view in the web, etc.
Focusing on the user experience of "what is the current state of my dbt run at the time of {{ debug() }}" is a really nifty idea, and developers who would want do this are usually a bit confused by both ipdb and the way dbt/jinja2 does sandboxing/variables, etc.
I would be happy to contribute this, as I believe IDE integration/tracing/etc, as well as dumping more contextual would be a killer feature for opendbt.
python dbt-debugger.py dbt ...
This would basically do some setup before, then pass straight off to dbt.
-
Using
dbtRunner+dbt.invoke
References / other reading
- Wiki addition: How does Jinja Template debugging work microsoft/debugpy#381
- pydevd jinja debug
- https://peps.python.org/pep-0553/
tl;dr: {{ debug() }} is a bit tooo user unfriendly, especially given how tedious it is to get information about the executing sandboxed environment. https://docs.getdbt.com/blog/guide-to-jinja-debug.