Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PoC: Replace string templating with SQLAlchemy #54

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

scriptdruid
Copy link

just a poc to test usage of sqlachemy as required by use case

  • able to read a timestamp from snowflake table
  • needs mocking to local as next step , supported as seen in docs

@scriptdruid
Copy link
Author

Not for mergeing, please discard, only for meetup and idea

@scriptdruid
Copy link
Author

next steps

  • Implement ORM for the same code , start with timestamp
  • Add local mocking
  • Udpate hub.py to incorporate sql alchemy replacements

@michael-the1
Copy link
Collaborator

michael-the1 commented Mar 25, 2024

@vipulrai91 @hellowithchicken Thanks again for your efforts during the spring last Saturday! To summarize the sprint:

  • We set up a Snowflake account and learned the basics of Data Vault.
  • The overarching goal is to replace the use of string templates with SQLAlchemy. Hopefully this will pave the way forward to supporting backends other than Snowflake (e.g. BigQuery, postgres).
  • To that end, it would be nice if this PR can demonstrate a small example where hub_link_dml.py is expressed in terms of SQLAlchemy objects. Since this file consists of a select statement and a merge statement, I think it should be a simple select statement and to use those results in a MergeInto object.
    • For this minimal example, we don't need to worry about supporting other dialects yet so using MergeInto is totally fine.

Hope those guidelines will help you out and once again thank you for your contribution!

@michael-the1 michael-the1 self-requested a review March 25, 2024 15:53
@michael-the1 michael-the1 added the wip Work in progress label Mar 25, 2024
@michael-the1 michael-the1 changed the title poc to use sqlachemy: wip PoC: Replace string templating with SQLAlchemy Mar 25, 2024
@dlouseiro dlouseiro mentioned this pull request Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wip Work in progress
Development

Successfully merging this pull request may close these issues.

3 participants