diff --git a/docs-v1/content/en/docs/pipeline-stages/deployers/docker.md b/docs-v1/content/en/docs/pipeline-stages/deployers/docker.md index 5ab6d292456..86d0e147883 100644 --- a/docs-v1/content/en/docs/pipeline-stages/deployers/docker.md +++ b/docs-v1/content/en/docs/pipeline-stages/deployers/docker.md @@ -41,3 +41,129 @@ the application image `my-image` to the local Docker daemon: {{< alert title="Note" >}} Images listed to be deployed with the `docker` deployer **must also have a corresponding build artifact built by Skaffold.** {{< /alert >}} + +## Deploying with Docker Compose + +Skaffold can deploy your application using Docker Compose instead of individual containers. +This is useful when your application is already configured with a `docker-compose.yml` file +and you want to leverage Compose's features like service dependencies, networks, and volumes. + +### Configuration + +To deploy using Docker Compose, set `useCompose: true` in the `docker` deploy configuration: + +```yaml +deploy: + docker: + useCompose: true + images: + - my-app +``` + +### How it works + +When `useCompose` is enabled, Skaffold: + +1. Reads your `docker-compose.yml` file (or a custom file specified via environment variable) +2. Builds images as specified in your `build` section +3. Automatically replaces image names in the compose file with the built image tags +4. Creates a temporary compose file with the updated images +5. Runs `docker compose up -d` with a unique project name (`skaffold-{runID}`) +6. On cleanup, runs `docker compose down --volumes --remove-orphans` + +### Image name mapping + +**Important**: For Skaffold to correctly replace images in your compose file, the image names +in your `docker-compose.yml` must match (or be contained in) the image names specified in +the `build.artifacts` section. + +For example, if your `skaffold.yaml` has: + +```yaml +build: + artifacts: + - image: gcr.io/my-project/frontend-app + - image: gcr.io/my-project/backend-app +``` + +Your `docker-compose.yml` should use matching image names: + +```yaml +version: '3.8' +services: + frontend: + image: frontend-app # Matches the suffix of gcr.io/my-project/frontend-app + backend: + image: backend-app # Matches the suffix of gcr.io/my-project/backend-app +``` + +Skaffold will replace `frontend-app` with `gcr.io/my-project/frontend-app:latest-abc123` +and `backend-app` with `gcr.io/my-project/backend-app:latest-def456`. + +### Custom compose file location + +By default, Skaffold looks for `docker-compose.yml` in the current directory. +You can specify a custom location using the `SKAFFOLD_COMPOSE_FILE` environment variable: + +```bash +export SKAFFOLD_COMPOSE_FILE=path/to/my-compose.yml +skaffold dev +``` + +Or inline: + +```bash +SKAFFOLD_COMPOSE_FILE=docker-compose.prod.yml skaffold run +``` + +### Example + +Complete example configuration: + +**skaffold.yaml:** +```yaml +apiVersion: skaffold/v4beta13 +kind: Config +build: + artifacts: + - image: my-web-app + docker: + dockerfile: Dockerfile +deploy: + docker: + useCompose: true + images: + - my-web-app +``` + +**docker-compose.yml:** +```yaml +version: '3.8' +services: + web: + image: my-web-app + ports: + - "8080:8080" + environment: + - NODE_ENV=development + redis: + image: redis:7-alpine + ports: + - "6379:6379" +``` + +When you run `skaffold dev`, Skaffold will: +- Build `my-web-app` image +- Replace `my-web-app` in the compose file with the built tag (e.g., `my-web-app:latest-abc123`) +- Leave `redis:7-alpine` unchanged (not built by Skaffold) +- Deploy both services using `docker compose up` + +### Limitations and Notes + +- The compose file must have a valid `services` section +- Only images that are built by Skaffold will be replaced +- External images (like `postgres`, `redis`, etc.) are deployed as-is +- The compose project name is automatically generated as `skaffold-{runID}` to avoid conflicts +- Multiple Skaffold instances can run simultaneously without interfering with each other + +For a complete working example, see [`examples/docker-compose-deploy`](https://github.com/GoogleContainerTools/skaffold/tree/main/examples/docker-compose-deploy). diff --git a/docs-v1/content/en/samples/deployers/docker-compose.yaml b/docs-v1/content/en/samples/deployers/docker-compose.yaml new file mode 100644 index 00000000000..3faf65274cb --- /dev/null +++ b/docs-v1/content/en/samples/deployers/docker-compose.yaml @@ -0,0 +1,6 @@ +deploy: + docker: + useCompose: true + images: + - my-web-app + - my-api-app diff --git a/docs-v2/content/en/docs/deployers/docker.md b/docs-v2/content/en/docs/deployers/docker.md index 8d5db0677df..9f5b8230bd0 100644 --- a/docs-v2/content/en/docs/deployers/docker.md +++ b/docs-v2/content/en/docs/deployers/docker.md @@ -42,3 +42,129 @@ the application image `my-image` to the local Docker daemon: {{< alert title="Note" >}} Images listed to be deployed with the `docker` deployer **must also have a corresponding build artifact built by Skaffold.** {{< /alert >}} + +## Deploying with Docker Compose + +Skaffold can deploy your application using Docker Compose instead of individual containers. +This is useful when your application is already configured with a `docker-compose.yml` file +and you want to leverage Compose's features like service dependencies, networks, and volumes. + +### Configuration + +To deploy using Docker Compose, set `useCompose: true` in the `docker` deploy configuration: + +```yaml +deploy: + docker: + useCompose: true + images: + - my-app +``` + +### How it works + +When `useCompose` is enabled, Skaffold: + +1. Reads your `docker-compose.yml` file (or a custom file specified via environment variable) +2. Builds images as specified in your `build` section +3. Automatically replaces image names in the compose file with the built image tags +4. Creates a temporary compose file with the updated images +5. Runs `docker compose up -d` with a unique project name (`skaffold-{runID}`) +6. On cleanup, runs `docker compose down --volumes --remove-orphans` + +### Image name mapping + +**Important**: For Skaffold to correctly replace images in your compose file, the image names +in your `docker-compose.yml` must match (or be contained in) the image names specified in +the `build.artifacts` section. + +For example, if your `skaffold.yaml` has: + +```yaml +build: + artifacts: + - image: gcr.io/my-project/frontend-app + - image: gcr.io/my-project/backend-app +``` + +Your `docker-compose.yml` should use matching image names: + +```yaml +version: '3.8' +services: + frontend: + image: frontend-app # Matches the suffix of gcr.io/my-project/frontend-app + backend: + image: backend-app # Matches the suffix of gcr.io/my-project/backend-app +``` + +Skaffold will replace `frontend-app` with `gcr.io/my-project/frontend-app:latest-abc123` +and `backend-app` with `gcr.io/my-project/backend-app:latest-def456`. + +### Custom compose file location + +By default, Skaffold looks for `docker-compose.yml` in the current directory. +You can specify a custom location using the `SKAFFOLD_COMPOSE_FILE` environment variable: + +```bash +export SKAFFOLD_COMPOSE_FILE=path/to/my-compose.yml +skaffold dev +``` + +Or inline: + +```bash +SKAFFOLD_COMPOSE_FILE=docker-compose.prod.yml skaffold run +``` + +### Example + +Complete example configuration: + +**skaffold.yaml:** +```yaml +apiVersion: skaffold/v4beta13 +kind: Config +build: + artifacts: + - image: my-web-app + docker: + dockerfile: Dockerfile +deploy: + docker: + useCompose: true + images: + - my-web-app +``` + +**docker-compose.yml:** +```yaml +version: '3.8' +services: + web: + image: my-web-app + ports: + - "8080:8080" + environment: + - NODE_ENV=development + redis: + image: redis:7-alpine + ports: + - "6379:6379" +``` + +When you run `skaffold dev`, Skaffold will: +- Build `my-web-app` image +- Replace `my-web-app` in the compose file with the built tag (e.g., `my-web-app:latest-abc123`) +- Leave `redis:7-alpine` unchanged (not built by Skaffold) +- Deploy both services using `docker compose up` + +### Limitations and Notes + +- The compose file must have a valid `services` section +- Only images that are built by Skaffold will be replaced +- External images (like `postgres`, `redis`, etc.) are deployed as-is +- The compose project name is automatically generated as `skaffold-{runID}` to avoid conflicts +- Multiple Skaffold instances can run simultaneously without interfering with each other + +For a complete working example, see [`examples/docker-compose-deploy`](https://github.com/GoogleContainerTools/skaffold/tree/main/examples/docker-compose-deploy). diff --git a/docs-v2/content/en/samples/deployers/docker-compose.yaml b/docs-v2/content/en/samples/deployers/docker-compose.yaml new file mode 100644 index 00000000000..3faf65274cb --- /dev/null +++ b/docs-v2/content/en/samples/deployers/docker-compose.yaml @@ -0,0 +1,6 @@ +deploy: + docker: + useCompose: true + images: + - my-web-app + - my-api-app diff --git a/integration/delete_test.go b/integration/delete_test.go index 72e29583f45..375051df99d 100644 --- a/integration/delete_test.go +++ b/integration/delete_test.go @@ -18,6 +18,7 @@ package integration import ( "context" + "strings" "testing" "time" @@ -133,6 +134,67 @@ func getContainers(ctx context.Context, t *testutil.T, deployedContainers []stri return cl } +func getComposeContainers(ctx context.Context, t *testutil.T, projectNamePrefix string, client docker.LocalDaemon) []types.Container { + t.Helper() + + // List all containers + cl, err := client.ContainerList(ctx, container.ListOptions{ + All: true, + }) + t.CheckNoError(err) + + // Filter containers by project name prefix + var result []types.Container + for _, c := range cl { + if project, ok := c.Labels["com.docker.compose.project"]; ok { + // Check if project name starts with the prefix (e.g., "skaffold-") + if strings.HasPrefix(project, projectNamePrefix) { + result = append(result, c) + } + } + } + + return result +} + +func TestDeleteDockerComposeDeployer(t *testing.T) { + tests := []struct { + description string + dir string + args []string + }{ + { + description: "docker compose deployer", + dir: "testdata/docker-compose-deploy", + args: []string{}, + }, + } + + for _, test := range tests { + testutil.Run(t, test.description, func(t *testutil.T) { + MarkIntegrationTest(t.T, CanRunWithoutGcp) + ctx := context.Background() + + // Run skaffold to deploy with docker compose + skaffold.Run(test.args...).InDir(test.dir).RunOrFail(t.T) + + // Verify containers are running + client := SetupDockerClient(t.T) + containers := getComposeContainers(ctx, t, "skaffold-", client) + if len(containers) == 0 { + t.T.Fatal("Expected at least one container to be deployed") + } + + // Delete the deployment + skaffold.Delete(test.args...).InDir(test.dir).RunOrFail(t.T) + + // Verify containers are deleted + containers = getComposeContainers(ctx, t, "skaffold-", client) + t.CheckDeepEqual(0, len(containers)) + }) + } +} + func TestDeleteNonExistedHelmResource(t *testing.T) { var tests = []struct { description string diff --git a/integration/dev_test.go b/integration/dev_test.go index df7b1e177ea..fa636dc7722 100644 --- a/integration/dev_test.go +++ b/integration/dev_test.go @@ -187,6 +187,48 @@ func TestDevCancelWithDockerDeployer(t *testing.T) { } } +func TestDevCancelWithDockerComposeDeployer(t *testing.T) { + if runtime.GOOS == "windows" { + t.Skip("graceful cancel doesn't work on windows") + } + + tests := []struct { + description string + dir string + minContainers int + projectPrefix string + }{ + { + description: "interrupt dev loop in Docker Compose deployer", + dir: "testdata/docker-compose-deploy", + minContainers: 1, + projectPrefix: "skaffold-", + }, + } + + for _, test := range tests { + t.Run(test.description, func(t *testing.T) { + MarkIntegrationTest(t, CanRunWithoutGcp) + p, err := skaffold.Dev().InDir(test.dir).StartWithProcess(t) + if err != nil { + t.Fatalf("error starting skaffold dev process") + } + + if err = waitForComposeContainersRunning(t, test.projectPrefix, test.minContainers); err != nil { + t.Fatalf("failed waiting for containers: %v", err) + } + + p.Signal(syscall.SIGINT) + + state, _ := p.Wait() + + if state.ExitCode() != 0 { + t.Fail() + } + }) + } +} + func TestDevAPIBuildTrigger(t *testing.T) { MarkIntegrationTest(t, CanRunWithoutGcp) diff --git a/integration/examples/docker-compose-deploy/Dockerfile b/integration/examples/docker-compose-deploy/Dockerfile new file mode 100644 index 00000000000..e109b4785d1 --- /dev/null +++ b/integration/examples/docker-compose-deploy/Dockerfile @@ -0,0 +1,11 @@ +FROM golang:1.22-alpine AS builder +WORKDIR /app +COPY main.go . +COPY go.mod . +RUN go build -o app main.go + +FROM alpine:latest +WORKDIR /root/ +COPY --from=builder /app/app . +EXPOSE 8080 +CMD ["./app"] diff --git a/integration/examples/docker-compose-deploy/README.md b/integration/examples/docker-compose-deploy/README.md new file mode 100644 index 00000000000..6c4eb17a608 --- /dev/null +++ b/integration/examples/docker-compose-deploy/README.md @@ -0,0 +1,81 @@ +# Example: Deploying with Docker Compose + +This example demonstrates how to use Skaffold to build and deploy applications using Docker Compose. + +## Prerequisites + +- Docker and Docker Compose installed +- Skaffold installed + +## Project Structure + +- `skaffold.yaml` - Skaffold configuration with Docker Compose deployment +- `docker-compose.yml` - Docker Compose configuration +- `Dockerfile` - Simple application Docker image +- `main.go` - Simple Go web application + +## How it Works + +1. Skaffold builds the Docker image for the application +2. Skaffold updates the `docker-compose.yml` with the built image tag +3. Skaffold runs `docker compose up` to deploy the application +4. When you stop Skaffold, it runs `docker compose down` to clean up + +## Usage + +### Run the application + +```bash +skaffold dev +``` + +This will: +- Build the application image +- Deploy it using Docker Compose +- Watch for changes and rebuild/redeploy automatically + +### Deploy only + +```bash +skaffold run +``` + +### Clean up + +```bash +skaffold delete +``` + +Or simply press `Ctrl+C` when running `skaffold dev`. + +## Configuration + +The key part of the `skaffold.yaml` configuration is: + +```yaml +deploy: + docker: + useCompose: true + images: + - compose-app +``` + +- `useCompose: true` - Enables Docker Compose deployment +- `images` - List of images to build and deploy + +## Environment Variables + +You can customize the Docker Compose file location: + +```bash +export SKAFFOLD_COMPOSE_FILE=custom-compose.yml +skaffold dev +``` + +Default is `docker-compose.yml` in the current directory. + +## Notes + +- The Docker Compose project name will be `skaffold-{runID}` +- Skaffold automatically replaces image names in the compose file with the built tags +- Volumes and networks are automatically cleaned up on `skaffold delete` diff --git a/integration/examples/docker-compose-deploy/docker-compose.yml b/integration/examples/docker-compose-deploy/docker-compose.yml new file mode 100644 index 00000000000..a8d5603620b --- /dev/null +++ b/integration/examples/docker-compose-deploy/docker-compose.yml @@ -0,0 +1,8 @@ +version: '3.8' +services: + app: + image: compose-app + ports: + - "8080:8080" + environment: + - MESSAGE=Hello from Docker Compose! diff --git a/integration/examples/docker-compose-deploy/go.mod b/integration/examples/docker-compose-deploy/go.mod new file mode 100644 index 00000000000..bd2b04ed17a --- /dev/null +++ b/integration/examples/docker-compose-deploy/go.mod @@ -0,0 +1,3 @@ +module compose-app + +go 1.22 diff --git a/integration/examples/docker-compose-deploy/main.go b/integration/examples/docker-compose-deploy/main.go new file mode 100644 index 00000000000..794a1cc7974 --- /dev/null +++ b/integration/examples/docker-compose-deploy/main.go @@ -0,0 +1,27 @@ +package main + +import ( + "fmt" + "log" + "net/http" + "os" +) + +func main() { + message := os.Getenv("MESSAGE") + if message == "" { + message = "Hello, World!" + } + + http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { + fmt.Fprintf(w, "%s\n", message) + }) + + port := ":8080" + log.Printf("Server starting on port %s", port) + log.Printf("Message: %s", message) + + if err := http.ListenAndServe(port, nil); err != nil { + log.Fatal(err) + } +} diff --git a/integration/examples/docker-compose-deploy/skaffold.yaml b/integration/examples/docker-compose-deploy/skaffold.yaml new file mode 100644 index 00000000000..81776999392 --- /dev/null +++ b/integration/examples/docker-compose-deploy/skaffold.yaml @@ -0,0 +1,14 @@ +apiVersion: skaffold/v4beta13 +kind: Config +metadata: + name: docker-compose-deploy-example +build: + artifacts: + - image: compose-app + docker: + dockerfile: Dockerfile +deploy: + docker: + useCompose: true + images: + - compose-app diff --git a/integration/testdata/docker-compose-deploy/Dockerfile b/integration/testdata/docker-compose-deploy/Dockerfile new file mode 100644 index 00000000000..e109b4785d1 --- /dev/null +++ b/integration/testdata/docker-compose-deploy/Dockerfile @@ -0,0 +1,11 @@ +FROM golang:1.22-alpine AS builder +WORKDIR /app +COPY main.go . +COPY go.mod . +RUN go build -o app main.go + +FROM alpine:latest +WORKDIR /root/ +COPY --from=builder /app/app . +EXPOSE 8080 +CMD ["./app"] diff --git a/integration/testdata/docker-compose-deploy/README.md b/integration/testdata/docker-compose-deploy/README.md new file mode 100644 index 00000000000..6c4eb17a608 --- /dev/null +++ b/integration/testdata/docker-compose-deploy/README.md @@ -0,0 +1,81 @@ +# Example: Deploying with Docker Compose + +This example demonstrates how to use Skaffold to build and deploy applications using Docker Compose. + +## Prerequisites + +- Docker and Docker Compose installed +- Skaffold installed + +## Project Structure + +- `skaffold.yaml` - Skaffold configuration with Docker Compose deployment +- `docker-compose.yml` - Docker Compose configuration +- `Dockerfile` - Simple application Docker image +- `main.go` - Simple Go web application + +## How it Works + +1. Skaffold builds the Docker image for the application +2. Skaffold updates the `docker-compose.yml` with the built image tag +3. Skaffold runs `docker compose up` to deploy the application +4. When you stop Skaffold, it runs `docker compose down` to clean up + +## Usage + +### Run the application + +```bash +skaffold dev +``` + +This will: +- Build the application image +- Deploy it using Docker Compose +- Watch for changes and rebuild/redeploy automatically + +### Deploy only + +```bash +skaffold run +``` + +### Clean up + +```bash +skaffold delete +``` + +Or simply press `Ctrl+C` when running `skaffold dev`. + +## Configuration + +The key part of the `skaffold.yaml` configuration is: + +```yaml +deploy: + docker: + useCompose: true + images: + - compose-app +``` + +- `useCompose: true` - Enables Docker Compose deployment +- `images` - List of images to build and deploy + +## Environment Variables + +You can customize the Docker Compose file location: + +```bash +export SKAFFOLD_COMPOSE_FILE=custom-compose.yml +skaffold dev +``` + +Default is `docker-compose.yml` in the current directory. + +## Notes + +- The Docker Compose project name will be `skaffold-{runID}` +- Skaffold automatically replaces image names in the compose file with the built tags +- Volumes and networks are automatically cleaned up on `skaffold delete` diff --git a/integration/testdata/docker-compose-deploy/docker-compose.yml b/integration/testdata/docker-compose-deploy/docker-compose.yml new file mode 100644 index 00000000000..a8d5603620b --- /dev/null +++ b/integration/testdata/docker-compose-deploy/docker-compose.yml @@ -0,0 +1,8 @@ +version: '3.8' +services: + app: + image: compose-app + ports: + - "8080:8080" + environment: + - MESSAGE=Hello from Docker Compose! diff --git a/integration/testdata/docker-compose-deploy/go.mod b/integration/testdata/docker-compose-deploy/go.mod new file mode 100644 index 00000000000..bd2b04ed17a --- /dev/null +++ b/integration/testdata/docker-compose-deploy/go.mod @@ -0,0 +1,3 @@ +module compose-app + +go 1.22 diff --git a/integration/testdata/docker-compose-deploy/main.go b/integration/testdata/docker-compose-deploy/main.go new file mode 100644 index 00000000000..794a1cc7974 --- /dev/null +++ b/integration/testdata/docker-compose-deploy/main.go @@ -0,0 +1,27 @@ +package main + +import ( + "fmt" + "log" + "net/http" + "os" +) + +func main() { + message := os.Getenv("MESSAGE") + if message == "" { + message = "Hello, World!" + } + + http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { + fmt.Fprintf(w, "%s\n", message) + }) + + port := ":8080" + log.Printf("Server starting on port %s", port) + log.Printf("Message: %s", message) + + if err := http.ListenAndServe(port, nil); err != nil { + log.Fatal(err) + } +} diff --git a/integration/testdata/docker-compose-deploy/skaffold.yaml b/integration/testdata/docker-compose-deploy/skaffold.yaml new file mode 100644 index 00000000000..81776999392 --- /dev/null +++ b/integration/testdata/docker-compose-deploy/skaffold.yaml @@ -0,0 +1,14 @@ +apiVersion: skaffold/v4beta13 +kind: Config +metadata: + name: docker-compose-deploy-example +build: + artifacts: + - image: compose-app + docker: + dockerfile: Dockerfile +deploy: + docker: + useCompose: true + images: + - compose-app diff --git a/integration/util.go b/integration/util.go index 0bf16fa1263..73c7a15cbd0 100644 --- a/integration/util.go +++ b/integration/util.go @@ -29,6 +29,7 @@ import ( "testing" "time" + "github.com/docker/docker/api/types/container" "github.com/docker/docker/errdefs" appsv1 "k8s.io/api/apps/v1" v1 "k8s.io/api/core/v1" @@ -501,6 +502,52 @@ func waitForContainersRunning(t *testing.T, containerNames ...string) error { }) } +func waitForComposeContainersRunning(t *testing.T, projectNamePrefix string, minContainers int) error { + t.Helper() + + ctx := context.Background() + timeout := 5 * time.Minute + interval := 1 * time.Second + client := SetupDockerClient(t) + + return wait.Poll(interval, timeout, func() (bool, error) { + // List all containers + containers, err := client.ContainerList(ctx, container.ListOptions{ + All: false, // Only running containers + }) + if err != nil { + return false, err + } + + // Filter containers by compose project name prefix + var runningCount int + for _, c := range containers { + if project, ok := c.Labels["com.docker.compose.project"]; ok { + // Check if project name starts with the prefix (e.g., "skaffold-") + if strings.HasPrefix(project, projectNamePrefix) { + // The container is already known to be running from ContainerList(All: false). + // We only need to inspect to check for undesirable states like restarting. + cInfo, err := client.RawClient().ContainerInspect(ctx, c.ID) + if err != nil { + return false, err + } + + if cInfo.State.Dead || cInfo.State.Restarting { + return false, fmt.Errorf("container %v is in dead or restarting state", c.ID) + } + runningCount++ + } + } + } + + if runningCount >= minContainers { + return true, nil + } + + return false, nil + }) +} + type fakeDockerConfig struct { kubeContext string } diff --git a/pkg/skaffold/deploy/docker/deploy.go b/pkg/skaffold/deploy/docker/deploy.go index c37b463dae2..3ccb1b72347 100644 --- a/pkg/skaffold/deploy/docker/deploy.go +++ b/pkg/skaffold/deploy/docker/deploy.go @@ -20,6 +20,8 @@ import ( "context" "fmt" "io" + "os" + "os/exec" "regexp" "strings" "sync" @@ -34,6 +36,7 @@ import ( "github.com/docker/go-connections/nat" v1 "github.com/google/go-containerregistry/pkg/v1" "github.com/pkg/errors" + "gopkg.in/yaml.v3" "github.com/GoogleContainerTools/skaffold/v2/pkg/skaffold/access" "github.com/GoogleContainerTools/skaffold/v2/pkg/skaffold/config" @@ -140,6 +143,11 @@ func (d *Deployer) Deploy(ctx context.Context, out io.Writer, builds []graph.Art return fmt.Errorf("creating skaffold network %s: %w", d.network, err) } + // If using docker compose, deploy all artifacts at once + if d.cfg.UseCompose { + return d.deployAllWithCompose(ctx, out, builds) + } + // TODO(nkubala)[07/20/21]: parallelize with sync.Errgroup for _, b := range builds { if err := d.deploy(ctx, out, b); err != nil { @@ -169,10 +177,6 @@ func (d *Deployer) deploy(ctx context.Context, out io.Writer, artifact graph.Art } d.portManager.RelinquishPorts(container.Name) } - if d.cfg.UseCompose { - // TODO(nkubala): implement - return fmt.Errorf("docker compose not yet supported by skaffold") - } containerCfg, err := d.containerConfigFromImage(ctx, artifact.Tag) if err != nil { @@ -352,6 +356,11 @@ func (d *Deployer) Dependencies() ([]string, error) { } func (d *Deployer) Cleanup(ctx context.Context, out io.Writer, dryRun bool, _ manifest.ManifestListByConfig) error { + // If using compose, run docker compose down + if d.cfg.UseCompose { + return d.cleanupWithCompose(ctx, out, dryRun) + } + if dryRun { for _, container := range d.tracker.DeployedContainers() { output.Yellow.Fprintln(out, container.ID) @@ -546,3 +555,167 @@ func (d *Deployer) GetStatusMonitor() status.Monitor { func (d *Deployer) RegisterLocalImages([]graph.Artifact) { // all images are local, so this is a noop } + +// deployAllWithCompose deploys all artifacts at once using docker compose. +// This ensures that docker compose up is called only once with all image replacements. +func (d *Deployer) deployAllWithCompose(ctx context.Context, out io.Writer, artifacts []graph.Artifact) error { + // Find compose file path (default: docker-compose.yml in current directory) + composeFile := d.getComposeFilePath() + + olog.Entry(ctx).Infof("Deploying with docker compose using file: %s", composeFile) + + // Check if compose file exists + if _, err := os.Stat(composeFile); err != nil { + return fmt.Errorf("compose file not found at %s: %w", composeFile, err) + } + + // Read compose file + composeData, err := os.ReadFile(composeFile) + if err != nil { + return fmt.Errorf("failed to read compose file: %w", err) + } + + // Parse compose file + var composeConfig map[string]interface{} + if err := yaml.Unmarshal(composeData, &composeConfig); err != nil { + return fmt.Errorf("failed to parse compose file: %w", err) + } + + // Replace image names with skaffold-built images for ALL artifacts + for _, artifact := range artifacts { + if err := d.replaceComposeImages(composeConfig, artifact); err != nil { + return fmt.Errorf("failed to replace images in compose file: %w", err) + } + } + + // Write modified compose file to temp location + modifiedComposeData, err := yaml.Marshal(composeConfig) + if err != nil { + return fmt.Errorf("failed to marshal modified compose config: %w", err) + } + + tmpComposeFile, err := os.CreateTemp("", "skaffold-compose-*.yml") + if err != nil { + return fmt.Errorf("failed to create temporary compose file: %w", err) + } + defer os.Remove(tmpComposeFile.Name()) + defer tmpComposeFile.Close() + + if _, err := tmpComposeFile.Write(modifiedComposeData); err != nil { + return fmt.Errorf("failed to write temporary compose file: %w", err) + } + tmpComposeFile.Close() + + // Run docker compose up (only once for all artifacts) + args := []string{"compose", "-f", tmpComposeFile.Name(), "-p", fmt.Sprintf("skaffold-%s", d.labeller.GetRunID()), "up", "-d"} + cmd := exec.CommandContext(ctx, "docker", args...) + cmd.Stdout = out + cmd.Stderr = out + + olog.Entry(ctx).Debugf("Running: docker %v", strings.Join(args, " ")) + + if err := cmd.Run(); err != nil { + return fmt.Errorf("docker compose up failed: %w", err) + } + + olog.Entry(ctx).Infof("Successfully deployed with docker compose") + + // Track all build artifacts + d.TrackBuildArtifacts(artifacts, nil) + + return nil +} + +// getComposeFilePath returns the path to the docker compose file +func (d *Deployer) getComposeFilePath() string { + // Check environment variable first + if path := os.Getenv("SKAFFOLD_COMPOSE_FILE"); path != "" { + return path + } + // Default to docker-compose.yml in current directory + return "docker-compose.yml" +} + +// imageNameMatches checks if a compose image name matches an artifact's built image. +// It handles cases like: +// - "frontend" matching "frontend:tag" or "gcr.io/project/frontend:tag" +// - "gcr.io/project/frontend" matching "gcr.io/project/frontend:tag" +// +// But rejects false positives like: +// - "app" should NOT match "my-app" +func imageNameMatches(composeImage, artifactTag string) bool { + // Remove tags from both images + composeBase := strings.Split(composeImage, ":")[0] + artifactBase := strings.Split(artifactTag, ":")[0] + + // Exact match (most common case) + if composeBase == artifactBase { + return true + } + + // Check if artifact image ends with compose image (handles registry prefixes) + // e.g., "frontend" matches "gcr.io/project/frontend" + // But "app" does NOT match "my-app" (no "/" separator) + return strings.HasSuffix(artifactBase, "/"+composeBase) +} + +// replaceComposeImages replaces image names in compose config with skaffold-built images +func (d *Deployer) replaceComposeImages(composeConfig map[string]interface{}, artifact graph.Artifact) error { + services, ok := composeConfig["services"].(map[string]interface{}) + if !ok { + return fmt.Errorf("invalid compose file: services section not found or invalid") + } + + // Iterate through services and replace image if it matches + for serviceName, serviceConfig := range services { + service, ok := serviceConfig.(map[string]interface{}) + if !ok { + continue + } + + // Check if service has an image field + if imageName, ok := service["image"].(string); ok { + // Check if this image matches the artifact's built image + if imageNameMatches(imageName, artifact.Tag) { + olog.Entry(context.Background()).Debugf("Replacing image for service %s: %s -> %s", serviceName, imageName, artifact.Tag) + service["image"] = artifact.Tag + } + } + } + + return nil +} + +// cleanupWithCompose cleans up resources deployed with docker compose +func (d *Deployer) cleanupWithCompose(ctx context.Context, out io.Writer, dryRun bool) error { + composeFile := d.getComposeFilePath() + projectName := fmt.Sprintf("skaffold-%s", d.labeller.GetRunID()) + + if dryRun { + fmt.Fprintf(out, "Would run: docker compose -f %s -p %s down\n", composeFile, projectName) + return nil + } + + olog.Entry(ctx).Infof("Cleaning up docker compose deployment") + + // Check if compose file exists + if _, err := os.Stat(composeFile); err != nil { + olog.Entry(ctx).Warnf("Compose file not found at %s, skipping cleanup", composeFile) + return nil + } + + // Run docker compose down + args := []string{"compose", "-f", composeFile, "-p", projectName, "down", "--volumes", "--remove-orphans"} + cmd := exec.CommandContext(ctx, "docker", args...) + cmd.Stdout = out + cmd.Stderr = out + + olog.Entry(ctx).Debugf("Running: docker %v", strings.Join(args, " ")) + + if err := cmd.Run(); err != nil { + return fmt.Errorf("docker compose down failed: %w", err) + } + + olog.Entry(ctx).Infof("Successfully cleaned up docker compose deployment") + return nil +} diff --git a/pkg/skaffold/deploy/docker/deploy_test.go b/pkg/skaffold/deploy/docker/deploy_test.go index 149dcd7d48d..a56a603d4b6 100644 --- a/pkg/skaffold/deploy/docker/deploy_test.go +++ b/pkg/skaffold/deploy/docker/deploy_test.go @@ -18,10 +18,12 @@ package docker import ( "context" + "os" "testing" "github.com/docker/docker/api/types/container" "github.com/docker/go-connections/nat" + "gopkg.in/yaml.v3" "github.com/GoogleContainerTools/skaffold/v2/pkg/skaffold/config" "github.com/GoogleContainerTools/skaffold/v2/pkg/skaffold/debug/types" @@ -281,3 +283,327 @@ func (m mockConfig) GlobalConfig() string { return "" } func (m mockConfig) MinikubeProfile() string { return "" } func (m mockConfig) Mode() config.RunMode { return "" } func (m mockConfig) Prune() bool { return false } + +// Tests for Docker Compose deployer + +func TestReplaceComposeImages(t *testing.T) { + tests := []struct { + name string + composeConfig map[string]interface{} + artifact graph.Artifact + expectedConfig map[string]interface{} + shouldErr bool + }{ + { + name: "single service image replacement", + composeConfig: map[string]interface{}{ + "services": map[string]interface{}{ + "app": map[string]interface{}{ + "image": "myapp", + }, + }, + }, + artifact: graph.Artifact{ + ImageName: "myapp", + Tag: "myapp:v1.2.3", + }, + expectedConfig: map[string]interface{}{ + "services": map[string]interface{}{ + "app": map[string]interface{}{ + "image": "myapp:v1.2.3", + }, + }, + }, + shouldErr: false, + }, + { + name: "multiple services only matching one replaced", + composeConfig: map[string]interface{}{ + "services": map[string]interface{}{ + "frontend": map[string]interface{}{ + "image": "frontend-app", + }, + "backend": map[string]interface{}{ + "image": "backend-app", + }, + "database": map[string]interface{}{ + "image": "postgres:14", + }, + }, + }, + artifact: graph.Artifact{ + ImageName: "frontend-app", + Tag: "frontend-app:latest", + }, + expectedConfig: map[string]interface{}{ + "services": map[string]interface{}{ + "frontend": map[string]interface{}{ + "image": "frontend-app:latest", + }, + "backend": map[string]interface{}{ + "image": "backend-app", + }, + "database": map[string]interface{}{ + "image": "postgres:14", + }, + }, + }, + shouldErr: false, + }, + { + name: "no services section returns error", + composeConfig: map[string]interface{}{ + "version": "3.8", + }, + artifact: graph.Artifact{ + ImageName: "myapp", + Tag: "myapp:v1", + }, + expectedConfig: nil, + shouldErr: true, + }, + { + name: "service without image field is skipped", + composeConfig: map[string]interface{}{ + "services": map[string]interface{}{ + "app": map[string]interface{}{ + "build": ".", + }, + }, + }, + artifact: graph.Artifact{ + ImageName: "myapp", + Tag: "myapp:v1", + }, + expectedConfig: map[string]interface{}{ + "services": map[string]interface{}{ + "app": map[string]interface{}{ + "build": ".", + }, + }, + }, + shouldErr: false, + }, + { + name: "partial image name match with contains", + composeConfig: map[string]interface{}{ + "services": map[string]interface{}{ + "app": map[string]interface{}{ + "image": "myapp", + }, + }, + }, + artifact: graph.Artifact{ + ImageName: "gcr.io/project/myapp", + Tag: "gcr.io/project/myapp:sha256", + }, + expectedConfig: map[string]interface{}{ + "services": map[string]interface{}{ + "app": map[string]interface{}{ + "image": "gcr.io/project/myapp:sha256", + }, + }, + }, + shouldErr: false, + }, + { + name: "reject false positive: 'app' should not match 'my-app'", + composeConfig: map[string]interface{}{ + "services": map[string]interface{}{ + "myapp": map[string]interface{}{ + "image": "app", + }, + }, + }, + artifact: graph.Artifact{ + ImageName: "my-app", + Tag: "my-app:v1.0.0", + }, + expectedConfig: map[string]interface{}{ + "services": map[string]interface{}{ + "myapp": map[string]interface{}{ + "image": "app", // Should NOT be replaced + }, + }, + }, + shouldErr: false, + }, + { + name: "match with registry prefix", + composeConfig: map[string]interface{}{ + "services": map[string]interface{}{ + "frontend": map[string]interface{}{ + "image": "frontend", + }, + }, + }, + artifact: graph.Artifact{ + ImageName: "frontend", + Tag: "gcr.io/project/frontend:sha256-abc", + }, + expectedConfig: map[string]interface{}{ + "services": map[string]interface{}{ + "frontend": map[string]interface{}{ + "image": "gcr.io/project/frontend:sha256-abc", + }, + }, + }, + shouldErr: false, + }, + { + name: "match with tag in compose file", + composeConfig: map[string]interface{}{ + "services": map[string]interface{}{ + "backend": map[string]interface{}{ + "image": "backend:latest", + }, + }, + }, + artifact: graph.Artifact{ + ImageName: "backend", + Tag: "backend:v2.0.0", + }, + expectedConfig: map[string]interface{}{ + "services": map[string]interface{}{ + "backend": map[string]interface{}{ + "image": "backend:v2.0.0", + }, + }, + }, + shouldErr: false, + }, + } + + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + d := &Deployer{} + err := d.replaceComposeImages(test.composeConfig, test.artifact) + + if test.shouldErr && err == nil { + t.Error("expected error but got none") + } + if !test.shouldErr && err != nil { + t.Errorf("unexpected error: %v", err) + } + if !test.shouldErr && err == nil { + testutil.CheckDeepEqual(t, test.expectedConfig, test.composeConfig) + } + }) + } +} + +func TestGetComposeFilePath(t *testing.T) { + tests := []struct { + name string + envValue string + setEnv bool + expectedPath string + }{ + { + name: "default path when env not set", + setEnv: false, + expectedPath: "docker-compose.yml", + }, + { + name: "custom path from env variable", + envValue: "custom-compose.yml", + setEnv: true, + expectedPath: "custom-compose.yml", + }, + { + name: "custom path with directory", + envValue: "/path/to/my-compose.yml", + setEnv: true, + expectedPath: "/path/to/my-compose.yml", + }, + } + + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + // Save original env value + originalEnv := os.Getenv("SKAFFOLD_COMPOSE_FILE") + defer func() { + if originalEnv != "" { + os.Setenv("SKAFFOLD_COMPOSE_FILE", originalEnv) + } else { + os.Unsetenv("SKAFFOLD_COMPOSE_FILE") + } + }() + + // Set up test environment + if test.setEnv { + os.Setenv("SKAFFOLD_COMPOSE_FILE", test.envValue) + } else { + os.Unsetenv("SKAFFOLD_COMPOSE_FILE") + } + + d := &Deployer{} + path := d.getComposeFilePath() + + if path != test.expectedPath { + t.Errorf("expected path %q but got %q", test.expectedPath, path) + } + }) + } +} + +func TestDeployWithComposeFileOperations(t *testing.T) { + tests := []struct { + name string + composeFile string + shouldErr bool + }{ + { + name: "valid compose file", + composeFile: "testdata/docker-compose.yml", + shouldErr: false, + }, + { + name: "valid compose file with multiple services", + composeFile: "testdata/docker-compose-multi.yml", + shouldErr: false, + }, + { + name: "invalid yaml returns error", + composeFile: "testdata/docker-compose-invalid.yml", + shouldErr: true, + }, + { + name: "non-existent file returns error", + composeFile: "testdata/does-not-exist.yml", + shouldErr: true, + }, + } + + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + // Test file reading and parsing + data, err := os.ReadFile(test.composeFile) + if err != nil && !test.shouldErr { + t.Fatalf("failed to read test file: %v", err) + } + if err != nil && test.shouldErr { + // Expected error for non-existent file + return + } + + var composeConfig map[string]interface{} + err = yaml.Unmarshal(data, &composeConfig) + + if test.shouldErr && err == nil { + t.Error("expected error parsing yaml but got none") + } + if !test.shouldErr && err != nil { + t.Errorf("unexpected error parsing yaml: %v", err) + } + if !test.shouldErr && err == nil { + // Verify we can access services + if services, ok := composeConfig["services"]; !ok { + t.Error("expected services section in compose config") + } else if services == nil { + t.Error("services section is nil") + } + } + }) + } +} diff --git a/pkg/skaffold/deploy/docker/testdata/docker-compose-invalid.yml b/pkg/skaffold/deploy/docker/testdata/docker-compose-invalid.yml new file mode 100644 index 00000000000..3c9aad959ce --- /dev/null +++ b/pkg/skaffold/deploy/docker/testdata/docker-compose-invalid.yml @@ -0,0 +1,3 @@ +version: '3.8' +invalid yaml content: [ + this is not valid diff --git a/pkg/skaffold/deploy/docker/testdata/docker-compose-multi.yml b/pkg/skaffold/deploy/docker/testdata/docker-compose-multi.yml new file mode 100644 index 00000000000..ba7ef0c98cf --- /dev/null +++ b/pkg/skaffold/deploy/docker/testdata/docker-compose-multi.yml @@ -0,0 +1,14 @@ +version: '3.8' +services: + frontend: + image: frontend-app + ports: + - "3000:3000" + backend: + image: backend-app + ports: + - "8080:8080" + database: + image: postgres:14 + ports: + - "5432:5432" diff --git a/pkg/skaffold/deploy/docker/testdata/docker-compose.yml b/pkg/skaffold/deploy/docker/testdata/docker-compose.yml new file mode 100644 index 00000000000..4f70f027139 --- /dev/null +++ b/pkg/skaffold/deploy/docker/testdata/docker-compose.yml @@ -0,0 +1,6 @@ +version: '3.8' +services: + app: + image: myapp + ports: + - "8080:8080"