Skip to content

Merge local and remote GPU shapes index, preferring Object Storage overrides #1176

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 13, 2025

Conversation

mrDzurb
Copy link
Member

@mrDzurb mrDzurb commented May 6, 2025

Description:

This PR enhances the load_gpu_shapes_index function to merge GPU-shape definitions from two sources—an OCI Object Storage bucket and a local JSON file, so that any shapes defined remotely override their local counterparts, while still falling back cleanly if the remote load fails.


What Changed

  1. Separate remote vs. local loads

    • We now load remote_data and local_data into two distinct dictionaries.
    • If the CONDA_BUCKET_NS environment variable is set, we attempt to read from Object Storage.
  2. Merge logic

    • After loading both sources, we merge the two on the "shapes". Any entry present in remote_data["shapes"] overrides the same key in local_data, while preserving local entries not present remotely.
  3. Return the merged index

@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label May 6, 2025
Copy link

github-actions bot commented May 6, 2025

📌 Cov diff with main:

Coverage-0%

📌 Overall coverage:

Coverage-58.62%

Copy link

📌 Cov diff with main:

Coverage-0%

📌 Overall coverage:

Coverage-58.62%

Copy link

📌 Cov diff with main:

Coverage-0%

📌 Overall coverage:

Coverage-58.62%

Copy link
Member

@VipulMascarenhas VipulMascarenhas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good. I was thinking if local shapes should override the remote ones and not the other way round. But probably makes sense to keep default as remote to avoid bad shape combinations that might lead to failures.

@mrDzurb mrDzurb merged commit ddb3094 into main May 13, 2025
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OCA Verified All contributors have signed the Oracle Contributor Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants