Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

reclassify gcs data storage class #2808

Draft
wants to merge 16 commits into
base: main
Choose a base branch
from
Draft

reclassify gcs data storage class #2808

wants to merge 16 commits into from

Commits on Nov 15, 2023

  1. first pass as notebook to reclassify data to cold storage - still nee… 

    …d to add code to change storage type
    @charlie-costanzo
    charlie-costanzo committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    42dc064 View commit details
    Browse the repository at this point in the history

  2. comment out lines that were supposed to be commented

    @charlie-costanzo
    charlie-costanzo committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    0188a13 View commit details
    Browse the repository at this point in the history

  3. cleaned up spacing, print statements, and some comment blocks

    @charlie-costanzo
    charlie-costanzo committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    45857cf View commit details
    Browse the repository at this point in the history

  4. fix failing precommit

    @charlie-costanzo
    charlie-costanzo committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    75b1cb0 View commit details
    Browse the repository at this point in the history

  5. added consistency to formatting

    @charlie-costanzo
    charlie-costanzo committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    9f61e4e View commit details
    Browse the repository at this point in the history

  6. added storage class change logic

    @charlie-costanzo
    charlie-costanzo committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    38348de View commit details
    Browse the repository at this point in the history

  7. not sure why notebook viewer action did't work, testing

    @charlie-costanzo
    charlie-costanzo committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    06bb382 View commit details
    Browse the repository at this point in the history

  8. added parallelization, refctored some for modularity

    @charlie-costanzo
    charlie-costanzo committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    4a385f2 View commit details
    Browse the repository at this point in the history

  9. parallelized file listing

    @charlie-costanzo
    charlie-costanzo committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    6c8a44b View commit details
    Browse the repository at this point in the history

  10. moved bucket calls, put blob call in the right place

    @charlie-costanzo
    charlie-costanzo committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    8401f05 View commit details
    Browse the repository at this point in the history

  11. looping through all files instead of just one, as previous

    @charlie-costanzo
    charlie-costanzo committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    153abd5 View commit details
    Browse the repository at this point in the history

  12. removed some unnecessary, commented out lines

    @charlie-costanzo
    charlie-costanzo committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    0f2116c View commit details
    Browse the repository at this point in the history

  13. attempt to get nbviewer to load

    @charlie-costanzo
    charlie-costanzo committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    76c7a83 View commit details
    Browse the repository at this point in the history

  14. revised parameters in parallelization function

    @charlie-costanzo
    charlie-costanzo committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    b3b93dc View commit details
    Browse the repository at this point in the history

  15. pulled writing of blob out of list paralleliztion function

    @charlie-costanzo
    charlie-costanzo committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    40a6e9a View commit details
    Browse the repository at this point in the history

  16. forgot to return the list in the parallelization function

    @charlie-costanzo
    charlie-costanzo committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    9127ac1 View commit details
    Browse the repository at this point in the history