Skip to content

Conversation

@vipulr8
Copy link

@vipulr8 vipulr8 commented Mar 23, 2024

just a poc to test usage of sqlachemy as required by use case

  • able to read a timestamp from snowflake table
  • needs mocking to local as next step , supported as seen in docs

@vipulr8
Copy link
Author

vipulr8 commented Mar 23, 2024

Not for mergeing, please discard, only for meetup and idea

@vipulr8
Copy link
Author

vipulr8 commented Mar 23, 2024

next steps

  • Implement ORM for the same code , start with timestamp
  • Add local mocking
  • Udpate hub.py to incorporate sql alchemy replacements

@michael-the1
Copy link
Collaborator

michael-the1 commented Mar 25, 2024

@vipulrai91 @hellowithchicken Thanks again for your efforts during the spring last Saturday! To summarize the sprint:

  • We set up a Snowflake account and learned the basics of Data Vault.
  • The overarching goal is to replace the use of string templates with SQLAlchemy. Hopefully this will pave the way forward to supporting backends other than Snowflake (e.g. BigQuery, postgres).
  • To that end, it would be nice if this PR can demonstrate a small example where hub_link_dml.py is expressed in terms of SQLAlchemy objects. Since this file consists of a select statement and a merge statement, I think it should be a simple select statement and to use those results in a MergeInto object.
    • For this minimal example, we don't need to worry about supporting other dialects yet so using MergeInto is totally fine.

Hope those guidelines will help you out and once again thank you for your contribution!

@michael-the1 michael-the1 self-requested a review March 25, 2024 15:53
@michael-the1 michael-the1 added the wip Work in progress label Mar 25, 2024
@michael-the1 michael-the1 changed the title poc to use sqlachemy: wip PoC: Replace string templating with SQLAlchemy Mar 25, 2024
@dlouseiro dlouseiro mentioned this pull request Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

wip Work in progress

Development

Successfully merging this pull request may close these issues.

3 participants