Skip to content

Fallback handling in case the StepLib (spec / source) is not available #181

Open
@viktorbenei

Description

@viktorbenei

Simply download the full spec (JSON) from s3 or from another Spec git repository, if the primary source can't be accessed (e.g. GitHub rate limiting)

Solution idea

  • Include the "alternatives" / "caches" / "mirrors" list in the StepLib's git repo spec, in the steplib.yml
  • The primary StepLib git repo still have to be downloaded at least once (e.g. from GitHub), but then the cache/mirror can be used in case the primary source (GitHub) is not available
  • Cache / mirror should be either an URL (ZIP, tar) or a git (clone) url; a list should be allowed with priority - similar to how download_locations work for steps
  • we should set up a mirror on GitLab for the main Bitrise StepLib - GitLab has an automatic GitHub mirroring feature
  • maybe also set up an S3 cache, just to be sure & to test
  • the ZIP/tar should not include the .git directory - to save space (.git is pretty much the largest dir, as it includes diffs and git history)

Notes

Only download the full spec.json, and do it only if there was a change. First send a HEAD request, to see when it was last modified, and store the modified date when downloading the spec.json, so that it can be used for comparison in subsequent checks.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions