Problem running a docker-compose devcontainer

[arntanguy]

I am fairly new to the world of devcontainers and devpod, so please forgive me if the answer is obvious.
I'm trying to run a simple devcontainer using docker-compose for NextJS and PostgrSQL but can't make it work. Here is the output I get.
Please note that if I run docker-compose up directly everything works.

devpod up . --id nextjs-postgrsql --devcontainer-path ./.devcontainer/devcontainer.json 
12:50:27 info Creating devcontainer...
12:50:29 error Error finding project files: No such command: ls

Commands:
  build              Build or rebuild services
  config             Validate and view the Compose file
  create             Create services
  down               Stop and remove resources
  events             Receive real time events from containers
  exec               Execute a command in a running container
  help               Get help on a command
  images             List images
  kill               Kill containers
  logs               View output from containers
  pause              Pause services
  port               Print the public port for a port binding
  ps                 List containers
  pull               Pull service images
  push               Push service images
  restart            Restart services
  rm                 Remove stopped containers
  run                Run a one-off command
  scale              Set number of containers for a service
  start              Start services
  stop               Stop services
  top                Display the running processes
  unpause            Unpause services
  up                 Create and start containers
  version            Show version information and quit
: exit status 1
12:50:29 info db uses an image, skipping
12:50:29 info Building app
12:50:29 info DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
12:50:29 info Install the buildx component to build images with BuildKit:
12:50:29 info https://docs.docker.com/go/buildx/
12:50:29 info 
12:50:29 info Sending build context to Docker daemon  6.144kB
12:50:29 info Step 1/2 : ARG VARIANT=18
12:50:29 info Step 2/2 : FROM mcr.microsoft.com/vscode/devcontainers/javascript-node:${VARIANT}
12:50:29 info ---> 8e9cdf680eff
12:50:29 info Successfully built 8e9cdf680eff
12:50:29 info Successfully tagged default-ne-9ff46_app:latest
12:50:31 info devcontainer up: start container: inspect image: get image config remotely: retrieve image default-ne-9ff46-app: GET https://index.docker.io/v2/library/default-ne-9ff46-app/manifests/latest: UNAUTHORIZED: authentication required; [map[Action:pull Class: Name:library/default-ne-9ff46-app Type:repository]]
12:50:32 error Try using the --debug flag to see a more verbose output
12:50:32 fatal run agent command: Process exited with status 1

I see two issues:

• 12:50:29 error Error finding project files: No such command: ls: I don't understand what's trying to run an ls command?
• It appears that the app service (that builds from the Dockerfile) succeeds building, gets tagged as default-ne-9ff46_app:latest but on the subsequent step it tries getting an image manifest from docker.io. This docker image should be local, so why?

What did you expect to happen instead?

How can we reproduce the bug? (as minimally and precisely as possible)

My full devcontainer/docker-compose configuration: https://github.com/arntanguy/mountain-planner/tree/main/.devcontainer
My devcontainer.json:

// Update the VARIANT arg in docker-compose.yml to pick a Node.js version
{
	"name": "Node.js & PostgreSQL",
	"dockerComposeFile": "docker-compose.yml",
	"service": "app",
	"workspaceFolder": "/workspace",

	// Configure tool-specific properties.
	"customizations": {
		// Configure properties specific to VS Code.
		"vscode": {
			// Add the IDs of extensions you want installed when the container is created.
			"extensions": [
				"dbaeumer.vscode-eslint"
			]
		}
	},

	// Use 'forwardPorts' to make a list of ports inside the container available locally.
	// This can be used to network with other containers or with the host.
	// "forwardPorts": [3000, 5432],

	// Use 'postCreateCommand' to run commands after the container is created.
	// "postCreateCommand": "yarn install",

	// Comment out to connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
	"remoteUser": "node"
}

My Dockerfile:

ARG VARIANT=18
FROM mcr.microsoft.com/vscode/devcontainers/javascript-node:${VARIANT}

My docker-compose.yml:

services:
  app:
    build: 
      context: .
      dockerfile: Dockerfile
      args:
        # Update 'VARIANT' to pick an LTS version of Node.js: 18, 16, 14.
        # Append -bullseye or -buster to pin to an OS version.
        # Use -bullseye variants on local arm64/Apple Silicon.
        VARIANT: 18 

    volumes:
      - ..:/workspace:cached
      
    # Overrides default command so things don't shut down after the process ends.
    command: sleep infinity

    # Runs app on the same network as the database container, allows "forwardPorts" in devcontainer.json function.
    network_mode: service:db

    # Uncomment the next line to use a non-root user for all processes.
    # user: node

    # Use "forwardPorts" in **devcontainer.json** to forward an app port locally. 
    # (Adding the "ports" property to this file will not forward from a Codespace.)

  db:
    image: postgres:latest
    restart: unless-stopped
    volumes:
      - postgres-data:/var/lib/postgresql/data
    environment:
      POSTGRES_PASSWORD: postgres
      POSTGRES_USER: postgres
      POSTGRES_DB: postgres

    # Add "forwardPorts": ["5432"] to **devcontainer.json** to forward PostgreSQL locally.
    # (Adding the "ports" property to this file will not forward from a Codespace.)

volumes:
  postgres-data:

Local Environment:

• DevPod Version: v0.6.9
• Operating System: linux (ubuntu noble)
• ARCH of the OS: AMD64

DevPod Provider:

• Local/remote provider: docker

Anything else we need to know?
I have successfully managed using devpod with Docker in the past for cross-compiling C++ application, using Docker environments, but I have never tried it with docker-compose.

Thanks in advance for you input, and kudos for the project!

[bkneis]

Hi @arntanguy thanks for reporting your issue. I believe the issue is the version of docker compose, devpod requires docker compose version 2 which might explain the missing ls command.

In regards to the image error, I believe the default repo is docker.io so when a local image is specified without a repo this is used. Can you try upgrading your docker compose version and try again? Let me know if that helps

[arntanguy]

Thanks @bkneis for the answer, now I feel a bit silly regarding docker compose v2. My confusion came from the fact that Ubuntu has both a docker-compose and a docker-compose-v2 package. It would be nice to have a more user-friendly check within devpod ;)

Unforunately I am now running into a different issue: (full log below)

$ devpod up . --devcontainer-path ./.devcontainer/devcontainer.json --recreate --debug
16:40:18 debug Using docker command 'docker'
16:40:19 debug Loading docker compose project [/home/arnaud/devel/web/mountain-planner/.devcontainer/docker-compose.yml]
16:40:19 debug Loaded project defaultmob98ee
16:40:19 info Invalid character(s) found in build meta data "ds1-0ubuntu1~24"

I'm again not sure what's going on here. Running docker compose up directly in the .devcontainer folder works as expected.
Looking into the code, it seems that this error is reported when the project name is not alphanumerical, which is indeed the case here : "ds1-0ubuntu1~24"

	for _, build := range v.Build {
		if len(build) == 0 {
			return fmt.Errorf("Build meta data can not be empty %q", build)
		}
		if !containsOnly(build, alphanum) {
			return fmt.Errorf("Invalid character(s) found in build meta data %q", build)
		}
	}

Where does this name come from, and how do I fix it?

Full log

devpod up . --devcontainer-path ./.devcontainer/devcontainer.json --recreate --debug
16:40:15 info Workspace mountain-planner already exists
16:40:15 debug No pro instance available, skipping provider upgrade check
16:40:15 debug Acquire workspace lock...
16:40:15 debug Acquired workspace lock...
16:40:15 info Creating devcontainer...
16:40:15 debug Adding ssh keys to agent, disable via 'devpod context set-options -o SSH_ADD_PRIVATE_KEYS=false'
16:40:15 debug Inject and run command: '/home/arnaud/AppImage/devpod' helper ssh-server --stdio --debug
16:40:15 debug Execute command locally
16:40:15 debug Run ssh-add /home/arnaud/.ssh/id_rsa
16:40:17 debug Error adding key /home/arnaud/.ssh/id_rsa to agent: signal: killed
16:40:17 debug Attempting to create SSH client
16:40:17 debug SSH client created
16:40:17 debug SSH session created
16:40:17 debug Forwarding ssh-agent using /run/user/1000/gnupg/S.gpg-agent.ssh
16:40:17 info Execute SSH server command: zsh -c '/home/arnaud/AppImage/devpod' agent workspace up --workspace-info 'H4sIAAAAAAAA/7RTXW/aPBT+L+faiYGWV63veClbo3UDsTBpu6mMfSBuHTuynVCpyn+fnPDZrjeTxt2xz2Oer7zCzrpnX3GBc6e2ygADWtgSKXeG15KmEpvKSiqsCfgSPJW44bUO9IjztLS1CVyZpNLcGHSnu/TJWwPk9CfAXkFJYPAWAwTq7mL/flLaZH17gwgEKmcbJdFFsOElxi0rnjuQrYKyxseru/n0y2z5uJjk93FsuK4vVmuPbtE/JYEFV2NLIPs2mebZjyz/+ZhnX2fzVR6xf1htWwIlF4UyUURLQEk8I9R4YSVCS8Db2vVCtRVcf7K6o35pq8QGNd3h+p158QmJzdR2p+gWPBTAIO2SEIdTej4cXBYOeXQjVyX6wMsKGIwGo3EyGCajq3w4ZlfXbHjzCwho7sPKR3FvNq4HbDiOG/vAT4lEYnyLJhylAYNoTUyoJ3khcVJVWcm3SPsGAQFpd0ZbLlfLB2BQhFB5RulWhaJep8KWVNtNSHyxR1CHGrlHTw9A2gzS/9JbIKDME4rwWYWpQ4kmKK79iQ6+oOhD6sPX6P35HBXsKR/boYwPXEdNG659l6SofbBlBMZBq/mpa3+ZkMMuIzzW7319/19lD3ez5QctJIe1+/n3j5pK/vWXcLA/9szWAdhocPy1vwEAAP//AQAA//8GN8woUgQAAA==' --debug
16:40:17 info Use /home/arnaud/.devpod/agent/contexts/default/workspaces/mountain-planner as workspace dir
16:40:17 debug Created logger
16:40:17 debug Received ping from agent
16:40:17 debug Credentials server started on port 14268...
16:40:18 debug Incoming client connection at /
16:40:18 debug Credentials server started...
16:40:18 debug Workspace Folder already exists /home/arnaud/devel/web/mountain-planner
16:40:18 debug Local folder with local provider; skip downloading
16:40:18 debug Skipping configuring daemon
16:40:18 debug Using docker command 'docker'
16:40:19 debug Loading docker compose project [/home/arnaud/devel/web/mountain-planner/.devcontainer/docker-compose.yml]
16:40:19 debug Loaded project defaultmob98ee
16:40:19 info Invalid character(s) found in build meta data "ds1-0ubuntu1~24"
16:40:19 info get default image
16:40:19 info github.com/loft-sh/devpod/pkg/devcontainer.(*runner).startContainer
16:40:19 info 	/home/runner/work/devpod/devpod/pkg/devcontainer/compose.go:281
16:40:19 info github.com/loft-sh/devpod/pkg/devcontainer.(*runner).runDockerCompose
16:40:19 info 	/home/runner/work/devpod/devpod/pkg/devcontainer/compose.go:186
16:40:19 info github.com/loft-sh/devpod/pkg/devcontainer.(*runner).Up
16:40:19 info 	/home/runner/work/devpod/devpod/pkg/devcontainer/run.go:121
16:40:19 info github.com/loft-sh/devpod/cmd/agent/workspace.(*UpCmd).devPodUp
16:40:19 info 	/home/runner/work/devpod/devpod/cmd/agent/workspace/up.go:129
16:40:19 info github.com/loft-sh/devpod/cmd/agent/workspace.(*UpCmd).up
16:40:19 info 	/home/runner/work/devpod/devpod/cmd/agent/workspace/up.go:104
16:40:19 info github.com/loft-sh/devpod/cmd/agent/workspace.(*UpCmd).Run
16:40:19 info 	/home/runner/work/devpod/devpod/cmd/agent/workspace/up.go:94
16:40:19 info github.com/loft-sh/devpod/cmd/agent/workspace.NewUpCmd.func1
16:40:19 info 	/home/runner/work/devpod/devpod/cmd/agent/workspace/up.go:52
16:40:19 info github.com/spf13/cobra.(*Command).execute
16:40:19 info 	/home/runner/work/devpod/devpod/vendor/github.com/spf13/cobra/command.go:985
16:40:19 info github.com/spf13/cobra.(*Command).ExecuteC
16:40:19 info 	/home/runner/work/devpod/devpod/vendor/github.com/spf13/cobra/command.go:1117
16:40:19 info github.com/spf13/cobra.(*Command).Execute
16:40:19 info 	/home/runner/work/devpod/devpod/vendor/github.com/spf13/cobra/command.go:1041
16:40:19 info github.com/loft-sh/devpod/cmd.Execute
16:40:19 info 	/home/runner/work/devpod/devpod/cmd/root.go:81
16:40:19 info main.main
16:40:19 info 	/home/runner/work/devpod/devpod/main.go:6
16:40:19 info runtime.main
16:40:19 info 	/home/runner/go/pkg/mod/golang.org/[email protected]/src/runtime/proc.go:271
16:40:19 info runtime.goexit
16:40:19 info 	/home/runner/go/pkg/mod/golang.org/[email protected]/src/runtime/asm_amd64.s:1695
16:40:19 info start container
16:40:19 info github.com/loft-sh/devpod/pkg/devcontainer.(*runner).runDockerCompose
16:40:19 info 	/home/runner/work/devpod/devpod/pkg/devcontainer/compose.go:188
16:40:19 info github.com/loft-sh/devpod/pkg/devcontainer.(*runner).Up
16:40:19 info 	/home/runner/work/devpod/devpod/pkg/devcontainer/run.go:121
16:40:19 info github.com/loft-sh/devpod/cmd/agent/workspace.(*UpCmd).devPodUp
16:40:19 info 	/home/runner/work/devpod/devpod/cmd/agent/workspace/up.go:129
16:40:19 info github.com/loft-sh/devpod/cmd/agent/workspace.(*UpCmd).up
16:40:19 info 	/home/runner/work/devpod/devpod/cmd/agent/workspace/up.go:104
16:40:19 info github.com/loft-sh/devpod/cmd/agent/workspace.(*UpCmd).Run
16:40:19 info 	/home/runner/work/devpod/devpod/cmd/agent/workspace/up.go:94
16:40:19 info github.com/loft-sh/devpod/cmd/agent/workspace.NewUpCmd.func1
16:40:19 info 	/home/runner/work/devpod/devpod/cmd/agent/workspace/up.go:52
16:40:19 info github.com/spf13/cobra.(*Command).execute
16:40:19 info 	/home/runner/work/devpod/devpod/vendor/github.com/spf13/cobra/command.go:985
16:40:19 info github.com/spf13/cobra.(*Command).ExecuteC
16:40:19 info 	/home/runner/work/devpod/devpod/vendor/github.com/spf13/cobra/command.go:1117
16:40:19 info github.com/spf13/cobra.(*Command).Execute
16:40:19 info 	/home/runner/work/devpod/devpod/vendor/github.com/spf13/cobra/command.go:1041
16:40:19 info github.com/loft-sh/devpod/cmd.Execute
16:40:19 info 	/home/runner/work/devpod/devpod/cmd/root.go:81
16:40:19 info main.main
16:40:19 info 	/home/runner/work/devpod/devpod/main.go:6
16:40:19 info runtime.main
16:40:19 info 	/home/runner/go/pkg/mod/golang.org/[email protected]/src/runtime/proc.go:271
16:40:19 info runtime.goexit
16:40:19 info 	/home/runner/go/pkg/mod/golang.org/[email protected]/src/runtime/asm_amd64.s:1695
16:40:19 info devcontainer up
16:40:19 info github.com/loft-sh/devpod/cmd/agent/workspace.(*UpCmd).Run
16:40:19 info 	/home/runner/work/devpod/devpod/cmd/agent/workspace/up.go:96
16:40:19 info github.com/loft-sh/devpod/cmd/agent/workspace.NewUpCmd.func1
16:40:19 info 	/home/runner/work/devpod/devpod/cmd/agent/workspace/up.go:52
16:40:19 info github.com/spf13/cobra.(*Command).execute
16:40:19 info 	/home/runner/work/devpod/devpod/vendor/github.com/spf13/cobra/command.go:985
16:40:19 info github.com/spf13/cobra.(*Command).ExecuteC
16:40:19 info 	/home/runner/work/devpod/devpod/vendor/github.com/spf13/cobra/command.go:1117
16:40:19 info github.com/spf13/cobra.(*Command).Execute
16:40:19 info 	/home/runner/work/devpod/devpod/vendor/github.com/spf13/cobra/command.go:1041
16:40:19 info github.com/loft-sh/devpod/cmd.Execute
16:40:19 info 	/home/runner/work/devpod/devpod/cmd/root.go:81
16:40:19 info main.main
16:40:19 info 	/home/runner/work/devpod/devpod/main.go:6
16:40:19 info runtime.main
16:40:19 info 	/home/runner/go/pkg/mod/golang.org/[email protected]/src/runtime/proc.go:271
16:40:19 info runtime.goexit
16:40:19 info 	/home/runner/go/pkg/mod/golang.org/[email protected]/src/runtime/asm_amd64.s:1695
16:40:19 debug Connection to SSH Server closed
16:40:19 debug Done creating devcontainer
16:40:19 debug Done executing ssh server helper command
16:40:19 fatal Process exited with status 1
run agent command
github.com/loft-sh/devpod/pkg/devcontainer/sshtunnel.ExecuteCommand.func2
	/home/runner/work/devpod/devpod/pkg/devcontainer/sshtunnel/sshtunnel.go:129
runtime.goexit
	/home/runner/go/pkg/mod/golang.org/[email protected]/src/runtime/asm_amd64.s:1695

[bkneis]

Hi @arntanguy this looks similar to #1576

For some reason devpod checks the version of docker-compose to make sure it is a semver. To me it looks like the ubuntu package you got it from has a named version (ds1-0ubuntu1~24).

I'll take a look into and see why we need this check and if we can mitigate it. For now if you wanted to fix I think installing docker / compose from https://docs.docker.com/engine/install/ubuntu/ should work.

[arntanguy]

Thanks @bkneis, this indeed solves the issue for me 👍

I can report it upstream to ubuntu maintainers if you wish, however I think it'd be nicer not to have to rely explicitly on docker compose's semver. If I'm not mistaken, docker-compose v1 used to be invoked with docker-compose command, while v2 is called with docker compose, wouldn't that be enough of a check?

Best regards,
Arnaud