Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: direct api and headscale access #667

Merged
merged 1 commit into from
Jun 14, 2024
Merged

feat: direct api and headscale access #667

merged 1 commit into from
Jun 14, 2024

Conversation

Tpuljak
Copy link
Member

@Tpuljak Tpuljak commented Jun 10, 2024

Direct API and headscale access for local projects

Description

With this PR, projects that are running locally will be able to access the Server without going through FRP. This change massively improves the binary download and SSH connection for those projects.

BREAKING CHANGE

Since the provider interface changed, installed providers will need to be updated (with daytona provider update)

  • This change requires a documentation update
  • I have made corresponding changes to the documentation

Related Issue(s)

Addresses #625

Notes

This PR is one of 2 that will address #625. The second part of the solution will be allowing users to add a static IP address of the server and avoid FRP altogether.

@Tpuljak Tpuljak requested a review from a team as a code owner June 10, 2024 12:38
@Tpuljak Tpuljak force-pushed the optimize-tailscale branch from 811df52 to 5eddf23 Compare June 10, 2024 12:44
@idagelic
Copy link
Member

idagelic commented Jun 10, 2024

Having some issues testing this out:

WSL2 - obviously very fast creation now, there's an issue with the project state updating and it stays "STOPPED" - this might be due to the speed of execution, but there there's also some project state updating errors.

(provider is 0.5.1-alpha)

image

Creating workspace
Initializing network
Network initialized
Enumerating objects: 35, done.
Counting objects: 100% (35/35), done.
Compressing objects: 100% (34/34), done.
Total 35 (delta 2), reused 0 (delta 0), pack-reused 0
Creating project Tic-Tac-Toe-On-Steroids
Pulling image...
latest: Pulling from daytonaio/workspace-project
Digest: sha256:cd61990900bcaece8daaedd54938d199c979817b5c18c21a7d0dcfb29f5cfc04
Status: Image is up to date for daytonaio/workspace-project:latest
Image pulled successfully
Workspace creation complete. Pending start...
Starting workspace
Project Tic-Tac-Toe-On-Steroids created

Downloading Daytona binary from http://host.docker.internal:3986/binary/v0.0.0-dev/daytona-linux-amd64
Starting project Tic-Tac-Toe-On-Steroids
Project Tic-Tac-Toe-On-Steroids started
Workspace Tic-Tac-Toe-On-Steroids2 started
Installing server to /usr/local/bin
Starting Daytona Agent
Cloning repository...
Enumerating objects: 35, done.
Counting objects: 100% (35/35), done.
Compressing objects: 100% (34/34), done.
Total 35 (delta 2), reused 0 (delta 0), pack-reused 0
Repository cloned
No post create commands to run
Running post start commands...
Running command: sudo dockerd
Starting ssh server on port 2222...
time="2024-06-10T15:10:42.781901485Z" level=info msg="Starting up"
time="2024-06-10T15:10:42.782614512Z" level=info msg="containerd not running, starting managed containerd"
time="2024-06-10T15:10:42.783582348Z" level=info msg="started new containerd process" address=/var/run/docker/containerd/containerd.sock module=libcontainerd pid=47
time="2024-06-10T15:10:42.800234162Z" level=info msg="starting containerd" revision=8b3b7ca2e5ce38e8f31a34f35b2b68ceb8470d89 version=1.6.32-1
time="2024-06-10T15:10:42.806171999Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.aufs\"..." type=io.containerd.snapshotter.v1
time="2024-06-10T15:10:42.806307937Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.aufs\"..." error="aufs is not supported (modprobe aufs failed: exec: \"modprobe\": executable file not found in $PATH \"\"): skip plugin" type=io.containerd.snapshotter.v1
time="2024-06-10T15:10:42.806323911Z" level=info msg="loading plugin \"io.containerd.content.v1.content\"..." type=io.containerd.content.v1
time="2024-06-10T15:10:42.806528904Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.btrfs\"..." type=io.containerd.snapshotter.v1
time="2024-06-10T15:10:42.806702309Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.btrfs\"..." error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs (overlay) must be a btrfs filesystem to be used with the btrfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
time="2024-06-10T15:10:42.806715010Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.devmapper\"..." type=io.containerd.snapshotter.v1
time="2024-06-10T15:10:42.806719927Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.devmapper" error="devmapper not configured"
time="2024-06-10T15:10:42.806722950Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.native\"..." type=io.containerd.snapshotter.v1
time="2024-06-10T15:10:42.806891560Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.overlayfs\"..." type=io.containerd.snapshotter.v1
time="2024-06-10T15:10:42.807216328Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.zfs\"..." type=io.containerd.snapshotter.v1
time="2024-06-10T15:10:42.807295575Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.zfs\"..." error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
time="2024-06-10T15:10:42.807308609Z" level=info msg="loading plugin \"io.containerd.metadata.v1.bolt\"..." type=io.containerd.metadata.v1
time="2024-06-10T15:10:42.807386871Z" level=warning msg="could not use snapshotter devmapper in metadata plugin" error="devmapper not configured"
time="2024-06-10T15:10:42.807398488Z" level=info msg="metadata content store policy set" policy=shared
time="2024-06-10T15:10:42.825115536Z" level=info msg="loading plugin \"io.containerd.differ.v1.walking\"..." type=io.containerd.differ.v1
time="2024-06-10T15:10:42.825179872Z" level=info msg="loading plugin \"io.containerd.event.v1.exchange\"..." type=io.containerd.event.v1
time="2024-06-10T15:10:42.825191499Z" level=info msg="loading plugin \"io.containerd.gc.v1.scheduler\"..." type=io.containerd.gc.v1
time="2024-06-10T15:10:42.825260172Z" level=info msg="loading plugin \"io.containerd.runtime.v1.linux\"..." type=io.containerd.runtime.v1
time="2024-06-10T15:10:42.825775928Z" level=info msg="loading plugin \"io.containerd.runtime.v2.task\"..." type=io.containerd.runtime.v2
time="2024-06-10T15:10:42.825950071Z" level=info msg="loading plugin \"io.containerd.internal.v1.opt\"..." type=io.containerd.internal.v1
time="2024-06-10T15:10:42.826284389Z" level=info msg="loading plugin \"io.containerd.warning.v1.deprecations\"..." type=io.containerd.warning.v1
time="2024-06-10T15:10:42.826299471Z" level=info msg="loading plugin \"io.containerd.monitor.v1.cgroups\"..." type=io.containerd.monitor.v1
time="2024-06-10T15:10:42.826446690Z" level=info msg="loading plugin \"io.containerd.service.v1.containers-service\"..." type=io.containerd.service.v1
time="2024-06-10T15:10:42.826462264Z" level=info msg="loading plugin \"io.containerd.service.v1.content-service\"..." type=io.containerd.service.v1
time="2024-06-10T15:10:42.826467234Z" level=info msg="loading plugin \"io.containerd.service.v1.diff-service\"..." type=io.containerd.service.v1
time="2024-06-10T15:10:42.826473029Z" level=info msg="loading plugin \"io.containerd.service.v1.images-service\"..." type=io.containerd.service.v1
time="2024-06-10T15:10:42.826479200Z" level=info msg="loading plugin \"io.containerd.service.v1.introspection-service\"..." type=io.containerd.service.v1
time="2024-06-10T15:10:42.826492258Z" level=info msg="loading plugin \"io.containerd.service.v1.leases-service\"..." type=io.containerd.service.v1
time="2024-06-10T15:10:42.826497552Z" level=info msg="loading plugin \"io.containerd.service.v1.namespaces-service\"..." type=io.containerd.service.v1
time="2024-06-10T15:10:42.826502429Z" level=info msg="loading plugin \"io.containerd.service.v1.snapshots-service\"..." type=io.containerd.service.v1
time="2024-06-10T15:10:42.826506674Z" level=info msg="loading plugin \"io.containerd.service.v1.tasks-service\"..." type=io.containerd.service.v1
time="2024-06-10T15:10:42.826515827Z" level=info msg="loading plugin \"io.containerd.grpc.v1.containers\"..." type=io.containerd.grpc.v1
time="2024-06-10T15:10:42.826521786Z" level=info msg="loading plugin \"io.containerd.grpc.v1.content\"..." type=io.containerd.grpc.v1
time="2024-06-10T15:10:42.826526129Z" level=info msg="loading plugin \"io.containerd.grpc.v1.diff\"..." type=io.containerd.grpc.v1
time="2024-06-10T15:10:42.826530345Z" level=info msg="loading plugin \"io.containerd.grpc.v1.events\"..." type=io.containerd.grpc.v1
time="2024-06-10T15:10:42.826538119Z" level=info msg="loading plugin \"io.containerd.grpc.v1.images\"..." type=io.containerd.grpc.v1
time="2024-06-10T15:10:42.826543508Z" level=info msg="loading plugin \"io.containerd.grpc.v1.introspection\"..." type=io.containerd.grpc.v1
time="2024-06-10T15:10:42.826547587Z" level=info msg="loading plugin \"io.containerd.grpc.v1.leases\"..." type=io.containerd.grpc.v1
time="2024-06-10T15:10:42.826552408Z" level=info msg="loading plugin \"io.containerd.grpc.v1.namespaces\"..." type=io.containerd.grpc.v1
time="2024-06-10T15:10:42.826562484Z" level=info msg="loading plugin \"io.containerd.grpc.v1.snapshots\"..." type=io.containerd.grpc.v1
time="2024-06-10T15:10:42.826567716Z" level=info msg="loading plugin \"io.containerd.grpc.v1.tasks\"..." type=io.containerd.grpc.v1
time="2024-06-10T15:10:42.826572661Z" level=info msg="loading plugin \"io.containerd.grpc.v1.version\"..." type=io.containerd.grpc.v1
time="2024-06-10T15:10:42.826576872Z" level=info msg="loading plugin \"io.containerd.tracing.processor.v1.otlp\"..." type=io.containerd.tracing.processor.v1
time="2024-06-10T15:10:42.826584851Z" level=info msg="skip loading plugin \"io.containerd.tracing.processor.v1.otlp\"..." error="skip plugin: tracing endpoint not configured" type=io.containerd.tracing.processor.v1
time="2024-06-10T15:10:42.826588724Z" level=info msg="loading plugin \"io.containerd.internal.v1.tracing\"..." type=io.containerd.internal.v1
time="2024-06-10T15:10:42.826593679Z" level=info msg="skip loading plugin \"io.containerd.internal.v1.tracing\"..." error="skip plugin: tracing endpoint not configured" type=io.containerd.internal.v1
time="2024-06-10T15:10:42.826596588Z" level=info msg="loading plugin \"io.containerd.internal.v1.restart\"..." type=io.containerd.internal.v1
time="2024-06-10T15:10:42.826626407Z" level=info msg="loading plugin \"io.containerd.grpc.v1.healthcheck\"..." type=io.containerd.grpc.v1
time="2024-06-10T15:10:42.827359884Z" level=info msg=serving... address=/var/run/docker/containerd/containerd-debug.sock
time="2024-06-10T15:10:42.827407412Z" level=info msg=serving... address=/var/run/docker/containerd/containerd.sock.ttrpc
time="2024-06-10T15:10:42.827823368Z" level=info msg=serving... address=/var/run/docker/containerd/containerd.sock
time="2024-06-10T15:10:42.827852332Z" level=info msg="containerd successfully booted in 0.029321s"
time="2024-06-10T15:10:43.825754072Z" level=error msg="failed to mount overlay: invalid argument" storage-driver=overlay2
time="2024-06-10T15:10:43.825805578Z" level=error msg="exec: \"fuse-overlayfs\": executable file not found in $PATH" storage-driver=fuse-overlayfs
time="2024-06-10T15:10:43.831163407Z" level=info msg="Loading containers: start."
time="2024-06-10T15:10:43.906642190Z" level=info msg="Loading containers: done."
time="2024-06-10T15:10:43.910405623Z" level=warning msg="WARNING: No blkio throttle.read_bps_device support"
time="2024-06-10T15:10:43.910422948Z" level=warning msg="WARNING: No blkio throttle.write_bps_device support"
time="2024-06-10T15:10:43.910426270Z" level=warning msg="WARNING: No blkio throttle.read_iops_device support"
time="2024-06-10T15:10:43.910427948Z" level=warning msg="WARNING: No blkio throttle.write_iops_device support"
time="2024-06-10T15:10:43.910436880Z" level=info msg="Docker daemon" commit=8e96db1c328d0467b015768e42a62c0f834970bb containerd-snapshotter=false storage-driver=vfs version=26.1.3-1
time="2024-06-10T15:10:43.910665333Z" level=info msg="Daemon has completed initialization"
time="2024-06-10T15:10:43.979646377Z" level=info msg="API listen on /var/run/docker.sock"
failed to update project state: Post "http://host.docker.internal:3986/workspace/3ed2f0e6c081/Tic-Tac-Toe-On-Steroids/state": dial tcp 172.17.0.1:3986: connect: connection refused
failed to update project state: Post "http://host.docker.internal:3986/workspace/3ed2f0e6c081/Tic-Tac-Toe-On-Steroids/state": dial tcp 172.17.0.1:3986: connect: connection refused


   Workspace       Tic-Tac-Toe-On-Steroids2

   ID              3ed2f0e6c081

   Editor          VS Code

   State           STOPPED

   Branch          master

   Repository      github.com/idagelic/Tic-Tac-Toe-On-Steroids.git


 Run 'daytona code' when you're ready to start developing

SSH-ing doesn't work, getting a timeout

image

@idagelic
Copy link
Member

On ARM Mac there's an issue downloading the binary - same command (repo) works on v0.18.0

image

idagelic@Ivans-MBP daytona % go run cmd/daytona/main.go create https://github.com/idagelic/Tic-Tac-Toe-On-Steroids
Enumerating objects: 35, done.
Counting objects: 100% (35/35), done.
Creating workspaces: 50% (17/34)
Initializing network
Network initialized
Compressing objects: 100% (34/34), done.
Total 35 (delta 2), reused 0 (delta 0), pack-reused 0
Creating project Tic-Tac-Toe-On-Steroids
Pulling image...
latest: Pulling from daytonaio/workspace-project
Digest: sha256:cd61990900bcaece8daaedd54938d199c979817b5c18c21a7d0dcfb29f5cfc04
Status: Image is up to date for daytonaio/workspace-project:latest
Image pulled successfully
Project Tic-Tac-Toe-On-Steroids created
Workspace creation complete. Pending start...
Starting workspace
Starting project Tic-Tac-Toe-On-Steroids
Project Tic-Tac-Toe-On-Steroids started
Workspace Tic-Tac-Toe-On-Steroids2 started
Downloading Daytona binary from http://host.docker.internal:3986/binary/v0.0.0-dev/daytona-linux-arm64
Installing server to /usr/local/bin
/usr/local/bin/daytona: line 1: syntax error near unexpected token `newline'
/usr/local/bin/daytona: line 1: `<?xml version="1.0" encoding="UTF-8"?>

@Tpuljak
Copy link
Member Author

Tpuljak commented Jun 11, 2024

On ARM Mac there's an issue downloading the binary - same command (repo) works on v0.18.0

@idagelic this is because you need to build the binary for linux-arm64 so it can be downloaded inside the container (GOOS=linux go build cmd/daytona/main.go && mv ~/Library/Application\ Support/daytona/server/binaries/v0.0.0-dev/daytona-linux-arm64

I'll check out the WSL issue today

@Tpuljak Tpuljak force-pushed the optimize-tailscale branch from 5eddf23 to 61cfd8a Compare June 12, 2024 07:12
@Tpuljak Tpuljak added the docs: needs update This PR requires the Daytona Docs to be updated label Jun 12, 2024
Local project will now access the Server directly without going through FRP

BREAKING CHANGE: Since the provider interface changed, installed providers will need to be updated

Signed-off-by: Toma Puljak <[email protected]>
@Tpuljak Tpuljak force-pushed the optimize-tailscale branch from 61cfd8a to 6e52442 Compare June 13, 2024 07:51
@Tpuljak
Copy link
Member Author

Tpuljak commented Jun 14, 2024

I'll check out the WSL issue today

The WSL issue could not be reproduced consistently and the actual issue is not with Daytona but the Docker provider implementation. We will raise an issue on that repo if we encounter the bug again.

@Tpuljak Tpuljak merged commit 08d1912 into main Jun 14, 2024
12 checks passed
@Tpuljak Tpuljak deleted the optimize-tailscale branch June 14, 2024 09:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs: needs update This PR requires the Daytona Docs to be updated
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants