Skip to content

[Feature] Microbatch batch_size should support metric intervals like '5 day' or '10 hour' #11638

@pransito

Description

@pransito

Is this your first time submitting a feature request?

  • I have read the expectations for open source contributors
  • I have searched the existing issues, and I could not find an existing issue for this feature
  • I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

Currently microbatch supports "hour", "day", "month", "year" instead of also things like "5 hour" to go in steps of 5 hours e.g. This was done already in custom materializations like e.g. here:

https://gist.github.com/jessedobbelaere/6fdb593f9e2cc732e9f142c56c9bac87

Describe alternatives you've considered

custom materializations but that is clunky

Who will this benefit?

Anyone who wants to make the microbatch batch size dynamic and sometimes go faster and sometime in smaller steps. E.g. the data for one data needs bigger steps than a country that has lots of data.

Are you interested in contributing this feature?

In principle yes. My skills may be limited. :-)

Anything else?

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions