OCI: Downloading policy images from AWS private repository failure

[carabasdaniel]

Trying to use a policy image from an AWS private repository fails to download all image layers. Getting a 400 Bad Request when trying to download the blob.

Steps to reproduce:

1. Create configuration file:

services:
      ghcr:
        url: https://**.dkr.ecr.us-east-2.amazonaws.com
        type: "oci"
        response_header_timeout_seconds: 5
        credentials:
          bearer:
            token: "AWS:$TOKEN"
            scheme: "Basic"
    bundles:
      todo:
        service: ghcr
        resource: "**.dkr.ecr.us-east-2.amazonaws.com/testnamespace/test-repo:1.0.0"
        persist: false
        config:
          polling:
            min_delay_seconds: 60
            max_delay_seconds: 120

2. Set your TOKEN using the AWS cli aws ecr get-login-password --region us-east-2
3. Run opa run -c <config.yaml> -l debug

From my initial investigation it seems that only the tarball layer fails to download while the manifest and config layer are loaded.

I've also tried using the rest aws plugin but getting the same results.

Can someone please take a look at what might be the cause of this issue and if there is a possible workaround ?

[ashutosh-narkar]

I'm not very familiar with the oci downloader. @carabasdaniel you and @DerGut have more insight into this. @DerGut anything you see that can help to resolve this.

[carabasdaniel]

Hi @ashutosh-narkar, I'm not very familiar with the authorizer plugin mechanism used by the rest client. In this scenario I suspect the issue might be related to something like #6728.

I've tried to use the policy CLI with the private registry and that one seems to work without any issues using the default docker resolver in the OCI package. I suspect the rest client used by the docker resolver might be the cause behind this as I mentioned.

I hope @DerGut knows this better and can help us out.

[stale]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days. Although currently inactive, the issue could still be considered and actively worked on in the future. More details about the use-case this issue attempts to address, the value provided by completing it or possible solutions to resolve it would help to prioritize the issue.

[srenatus]

I suspect the rest client used by the docker resolver might be the cause behind this as I mentioned.

Perhaps you could join @bluebrown in their efforts on #7189 -- it's similar in nature, something goes wrong downloading OCI bundles, and it's probably got to do with auth headers of some sort.