Add support for pushing and pulling private images in the remote builder

[pingsutw]

This pull request adds support for using private base images when building and running Flyte tasks, allowing users to specify registry secrets for authentication. The main changes introduce a new workflow for handling private registries, update the Image class to accept and propagate registry secrets, and ensure secrets are used during image build and runtime. The example demonstrates how to use these features.

Support for private image registries:

• Added an example (examples/image/private_base_image.py) showing how to build and run a Flyte task using a private base image, including instructions for creating and using a registry secret.
• Updated the Image class (src/flyte/_image.py) to accept a registry_secret parameter in relevant methods (from_debian_base, from_uv_script, clone) and propagate it internally for use during image build/pull. [1] [2] [3] [4] [5] [6]
• Modified the remote image builder (src/flyte/_internal/imagebuild/remote_builder.py) to use the registry secret when building images and to properly construct the target image reference for private registries. [1] [2]

Secret handling improvements:

• Enhanced secret creation logic (src/flyte/remote/_secret.py) to set project/domain only for regular secrets, allowing image pull secrets to be global. [1] [2]

Internal code improvements:

• Added _image_registry_secret attribute to the Image class for tracking registry secrets and updated method signatures and docstrings to document the new parameter. [1] [2] [3]

import flyte
from flyte import Image

image = (
    Image.from_base("pingsutw/private:d1742efed83fc4b18c7751e53e771bbe")
    .clone(registry="docker.io/pingsutw", name="private", registry_secret="pingsutw")
    .with_apt_packages("vim")
    .with_local_v2()
)

env = flyte.TaskEnvironment(name="private-image", image=image, secrets="pingsutw")


@env.task
async def t1(data: str = "hello") -> str:
    return f"Hello {data}"


if __name__ == "__main__":
    flyte.init_from_config()
    run = flyte.run(t1, data="world")
    print(run.name)
    print(run.url)

[wild-endeavor]

call it pull_secret? or something more clear that this is for pulling? or is this for pushing? Is it possible to use one secret to pull and another secret to push?

Also this new field(s) should be in every from_ function right?

[pingsutw]

or something more clear that this is for pulling? or is this for pushing?

for both

[kumare3]

raise an error if project/domain are set

[pingsutw]

done

[wild-endeavor]

wait so you have to declare it twice?

[pingsutw]

one is for task, another one is for remote builder

[wild-endeavor]

this seems weird... why?

[pingsutw]

We don't want to use project/domain in the config by default. If it's none, we should create a global secret