-
Notifications
You must be signed in to change notification settings - Fork 146
Description
Is this your first time submitting a feature request?
- I have read the expectations for open source contributors
- I have searched the existing issues, and I could not find an existing issue for this feature
- I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion
Describe the feature
BigQuery now allows users to assign a reservation to a query at runtime via the reservation
field in the job configuration. This feature enables more granular control over slot usage and can help isolate workloads, manage performance, and control cost.
The google-cloud-bigquery
lib already supports this parameter: googleapis/python-bigquery#2186
It would be helpful if dbt supported passing a reservation in the BigQuery adapter credentials (optional config), and forwarded this value to the query job config when executing queries. e.g
my-profile:
target: dev
outputs:
dev:
type: bigquery
method: oauth
project: abc-123
dataset: my_dataset
reservation: projects/{project}/locations/{location}/reservations/{reservation}
Docs reference: https://cloud.google.com/bigquery/docs/reservations-workload-management#flexible
Describe alternatives you've considered
- One alternative is to create separate projects that rely on default reservation assignments, but this limits flexibility and introduces operational overhead especially since a single project can be linked to multiple reservations.
- Another option is to use BigQuery scripting to explicitly assign the reservation at runtime via SQL like:
SET @@reservation='projects/project1/locations/US/reservations/test-reservation';
SELECT 42;
However, this approach runs in scripting mode and can introduce complexity, especially when conditionally assigning reservations.
Who will this benefit?
- Teams managing multiple BigQuery workloads with dedicated reservations
- Users optimizing cost/performance by manually routing dbt queries to specific reservations
- Organizations running multiple dbt jobs that need to isolate workloads more effectively especially for business-critical pipelines where avoiding slot contention is essential
Are you interested in contributing this feature?
Yes. I already have a working implementation and would open up a PR soon.
Anything else?
No response