Creating an ideal CLI for clone and add to an existing project

[quinnwai]

Description

• How do we manage state for a project?
• In existing g3t, we have used git but only for specific commands rather than in totality (pulling, resolving conflicts)
• We need to identify the ideal state of pulling a project and pushing up new project data without the assumption that you have an up-to-date project.

Ideal Steps

1. g3t clone: git clone-type UI to clone
   1. see below, maybe in file browser, maybe another app
   2. project IDs are unique (been patched out) -> need to double check
   3. Populates only metadata folders (and hidden dirs, .g3t and .git and config)
2. g3t ls: list of files and status on which files you have downloaded
   1. Calls the new ES table created in bucket config refactor
   2. Describes files that have been localized vs not
   3. Currently g3t status does the opposite (follows git status and looks at local compared to remote)
   4. Longer versions accessible via -vvv flag
3. g3t pull <file_selection>
   1. Need to be separate pulling from metadata vs from files
   2. Incremental pull of metadata exists(?) -> need to double check
   3. Conflict resolution
4. adding files, committing, and pushing

Misc Notes

• Create a git server instead of gen3 submission
  • or a git client on our servers?
• Setting up a gitlab referencing any ACC-created gitlab
• pull requests to merge?

[bwalsh]

https://github.com/ACED-IDP/aced_etl_pod/blob/main/etl-job/fhir_import_export.py#L461-L468

        # get the meta data file
        if _download_and_unzip(object_id, file_path, output, commit['meta_path']):
          
            # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # 
            # TODO : check prior zip files extract .git check to see if current .git can be merged, if not raise error
            # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # 

            # tell user what files were found
            for _ in pathlib.Path(file_path).glob('*'):
                output['files'].append(str(_))

            # load the study into the database and elastic search
            _load_all(project, f"{program}-{project}", output, file_path, schema)