Skip to content

GitHub files stored using Git LFS are not captured in our archives #680

@zaneselvans

Description

@zaneselvans

Overview

  • We archive several GitHub repositories, including EIA NEMS.
  • In some of those repositories, larger binary files are stored using Git LFS (large file storage).
  • However, it appears that the files stored with Git LFS are not captured by our archivers.
  • For example this spreadsheet of NEMS inputs is 11MB in the GitHub repo, but only 133 bytes in our archive of the repo

Probably we need to enable Git LFS in our git repo archiving runs and maybe do something other than downloading the prefabricated ZIP archives to capture these files -- like maybe we need to clone the repo with Git LFS installed and make our own zip archive locally.

The following archivers grab Github files:

  • eianems
  • epacamdeia

Tasks

  • Add git-lfs to dependencies and install using pixi bash script
  • Clone and run git lfs fetch inside the project repository
  • For each release, checkout and fetch again, then zip and upload

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

Status

In progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions