-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Description
Is this your first time submitting a feature request?
- I have read the expectations for open source contributors
- I have searched the existing issues, and I could not find an existing issue for this feature
- I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion
Describe the feature
Background & Motivation
Large dbt deployments often encounter scenarios requiring the execution of the same template package multiple times, each with distinct parameter sets (e.g., for different regions, business units, or customers). The fundamental challenge arises from dbt's current package resolution mechanism, which enforces a strict one-to-one mapping between a package name and its loaded resources within a single project.
For BigQuery environments, this limitation is particularly pronounced because INFORMATION_SCHEMA
queries must be region-qualified (region-us
, region-asia-northeast1
, etc.). If a package is designed to expose metadata for a specific region like region-us
, leveraging it for another region like asia-northeast1
currently necessitates code duplication or complex workarounds. This is because specifying the same package name multiple times in packages.yml
triggers a duplicate-project error during dbt deps
.
Existing workarounds—such as using macro loops for dynamic resource generation, conditional logic with generate_schema_name
, implementing multi-repo Mesh architectures, or maintaining repo forks—introduce significant complexity, increase maintenance overhead, and can obscure lineage.
Example Use-Case
# packages.yml
packages:
- package: my-org/bq_information_schema
version: 0.7.0
name: bq_info_us # ← proposed new key
with:
region: us
dataset_suffix: us_meta
- package: my-org/bq_information_schema
version: 0.7.0
name: bq_info_tokyo # ← unique alias
with:
region: asia-northeast1
dataset_suffix: tokyo_meta
Related Issues
- Aliases for dbt-packages — [CT-3381] [Feature] Aliases for dbt-packages #9073
- Add namespacing for dbt resources — Add namespacing for dbt resources #1269
- dbt deps should resolve multiple packages from one repo — [CT-748] [Enhancement]
dbt deps
should resolve multiple packages with samegit:
repo and uniquesubdirectory:
#5374 - Duplicate dbt_utils folder on
dbt deps
— Duplicate dbt_utils folder when running dbt deps #4372 - Multi-project collaboration (discussion) — Multi-project collaboration #6725
- Multi-project collaboration – Milestone 1 — [CT-1915] [Epic] Multi-project collaboration - Milestone 1 #6747
- Dynamically running a dbt package for 2+ sources (discussion) — Dynamically running a dbt package for 2+ distinct sources and outputting to separate schemas dbt-technical-blog-writing#92
Describe alternatives you've considered
- Macro loops & dynamic SQL – hides lineage, harder to test.
generate_schema_name
– only separates database schemas, not logic.- Multiple root projects (Mesh) – operational overhead for small variations.
- Fork & rename package – brittle; every upstream release requires two merges.
Who will this benefit?
- No code duplication – one canonical repo, many configurations.
- Clear lineage – DAG nodes grouped by logical tenant/region.
- Simpler upgrades – a single version bump affects all aliases.
- Mesh-ready – downstream projects can depend on specific aliases via
dependencies.yml
.
Are you interested in contributing this feature?
I am happy to contribute.
Anything else?
No response