[Bug]catch_jinja should capture type errors during the rendering phase and display the node information

[devmessias]

Is this a new bug in dbt-common?

• I believe this is a new bug in dbt-common
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

As discussed in the following comment by @jeremyyeo dbt-labs/dbt-core#8276 (comment) on another issue , when there is a type error inside a Jinja template for just one model, the dbt pipeline fails without providing any useful information. I've encountered the same problem several times and spent too much time figuring out what caused it. We can see the same issue appearing in other issues, with users complaining about it:

dbt-labs/dbt-core#4596 (comment).
dbt-labs/dbt-labs-experimental-features#37
dbt-labs/dbt-core#9033

Expected Behavior

• The user should receive information about the node (model) where the error occurred.

Steps To Reproduce

# dbt_project.yml
name: my_dbt_project
profile: all
config-version: 2
version: 1.0
models:
  my_dbt_project:
    +materialized: table

-- models/foo.sql
{{ config(materialized=-'table') }}

select 1 id

Relevant log output

12:23:14  Running with dbt=1.9.0-b2
12:23:14  Registered adapter: duckdb=1.9.0
12:23:14  Encountered an error:
bad operand type for unary -: 'str'
12:23:14  Traceback (most recent call last):
  File "/home/devmessias/phd/pkgs/dbt-my/core/dbt/cli/requires.py", line 153, in wrapper
    result, success = func(*args, **kwargs)
  File "/home/devmessias/phd/pkgs/dbt-my/core/dbt/cli/requires.py", line 103, in wrapper
    return func(*args, **kwargs)
  File "/home/devmessias/phd/pkgs/dbt-my/core/dbt/cli/requires.py", line 235, in wrapper
    return func(*args, **kwargs)
  File "/home/devmessias/phd/pkgs/dbt-my/core/dbt/cli/requires.py", line 264, in wrapper
    return func(*args, **kwargs)
  File "/home/devmessias/phd/pkgs/dbt-my/core/dbt/cli/requires.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/home/devmessias/phd/pkgs/dbt-my/core/dbt/cli/requires.py", line 327, in wrapper
    setup_manifest(ctx, write=write, write_perf_info=write_perf_info)
  File "/home/devmessias/phd/pkgs/dbt-my/core/dbt/cli/requires.py", line 351, in setup_manifest
    ctx.obj["manifest"] = parse_manifest(
  File "/home/devmessias/phd/pkgs/dbt-my/core/dbt/parser/manifest.py", line 1998, in parse_manifest
    manifest = ManifestLoader.get_full_manifest(
  File "/home/devmessias/phd/pkgs/dbt-my/core/dbt/parser/manifest.py", line 308, in get_full_manifest
    manifest = loader.load()
  File "/home/devmessias/phd/pkgs/dbt-my/core/dbt/parser/manifest.py", line 413, in load
    self.parse_project(
  File "/home/devmessias/phd/pkgs/dbt-my/core/dbt/parser/manifest.py", line 710, in parse_project
    parser.parse_file(block)
  File "/home/devmessias/phd/pkgs/dbt-my/core/dbt/parser/base.py", line 499, in parse_file
    self.parse_node(file_block)
  File "/home/devmessias/phd/pkgs/dbt-my/core/dbt/parser/base.py", line 468, in parse_node
    self.render_update(node, config)
  File "/home/devmessias/phd/pkgs/dbt-my/core/dbt/parser/models.py", line 349, in render_update
    super().render_update(node, config)
  File "/home/devmessias/phd/pkgs/dbt-my/core/dbt/parser/base.py", line 444, in render_update
    context = self.render_with_context(node, config)
  File "/home/devmessias/phd/pkgs/dbt-my/core/dbt/parser/base.py", line 272, in render_with_context
    get_rendered(parsed_node.raw_code, context, parsed_node, capture_macros=True)
  File "/home/devmessias/phd/pkgs/dbt-my/core/dbt/clients/jinja.py", line 144, in get_rendered
    rendered = render_template(template, ctx, node)
  File "/home/devmessias/phd/pkgs/dbt-common/dbt_common/clients/jinja.py", line 585, in render_template
    return template.render(ctx)
  File "/home/devmessias/phd/pkgs/dbt-common/dbt_common/clients/jinja.py", line 174, in render
    return self.environment.handle_exception()
  File "/home/devmessias/phd/pkgs/dbt-my/.venv/lib/python3.9/site-packages/jinja2/environment.py", line 939, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
TypeError: bad operand type for unary -: 'str'

Additional Context

I created an issue on dbt-core, but I think here is the correct place dbt-labs/dbt-core#10856

[dbeatty10]

@devmessias This is a great idea to avoid that nasty stacktrace in a variety of scenarios 🏆

Reprex

In addition to the one you provided, here's another reprex (originally offered by @jtcohen6 in dbt-labs/dbt-core#4596 (comment) and you also included in dbt-labs/dbt-core#10856 that will give an error that says "'NoneType' object is not iterable":

Create this file:

models/my_model.sql

{% set mylist = None %}

{% for item in mylist %}

  select '{{ item }}' as my_item
  {{ 'union all' if not loop.last }}

{% endfor %}

Run this command:

dbt parse

[devmessias]

Hi @dbeatty10, thanks for pointing out this example. With my PR #207, here is the output:

I'Ve created a new unit test to test if it's correct handling a type error when iterating over a None value.

c9b8667

I'll wait for the #207 review

[devmessias]

Hi @dbeatty10 @gshank, any updates on the PR review for this issue?