bug: SSH Connection not Working

[KimiyaMorozova]

Description

SSH Connections immediatly disconnects after authentication is complete.

Note: nsenter: reassociate to namespace 'ns/time' failed: Operation not permitted only happens when using docker as an agent.

Steps to Reproduce

Connect to any Machine

Expected Behavior

Terminal Connection should be established successful.

Edition

Community

Version

0.18.0 or 018.3, same thing happenend

Related Logs

ssh-1      | {"cols":80,"device":"d4995bc4de6651e62064aec8ef1c347169755f4708620771d72f4f015d3e81f6","error":"failed to read the message from socket\nread tcp 172.18.0.7:8080-\u003e172.18.0.8:53374: use of closed network connection","ip":"10.10.12.92","level":"error","msg":"failed to read the message from the client","rows":24,"time":"2025-03-11T14:46:07Z","user":"root"}

Related Code

No response

Additional Information

No response

[gustavosbarreto]

Can you please run the following command and share the output?

docker info

This will help us determine if Docker is running in rootless mode which could be related to the issue.

[KimiyaMorozova]

Shellhub is running underneath proxmox in an unpriviliged LXC Container, in case that matters.

 Version:    27.5.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.20.0
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.32.4
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 7
  Running: 5
  Paused: 0
  Stopped: 2
 Images: 13
 Server Version: 27.5.1
 Storage Driver: overlay2
  Backing Filesystem: zfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: true
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc version: v1.2.4-0-g6c52b3f
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.12-8-pve
 Operating System: Debian GNU/Linux 12 (bookworm)
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 512MiB
 Name: ANK-CTShellHub
 ID: 87b926f5-3618-4b3a-b49c-47368dc1cd04
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

[gustavosbarreto]

Thanks for the additional information. Since ShellHub is running inside an unprivileged LXC container, this is likely the root cause of the issue.

[KimiyaMorozova]

Running as a priviliged LXC Container does result in the same error (same logs)

[gustavosbarreto]

Running as a priviliged LXC Container does result in the same error (same logs)

Since the issue persists, it indicates that the restriction might be coming from other factors within the LXC environment on Proxmox. Even in a privileged LXC container, Proxmox may be restricting the use of time namespaces by enforcing additional security restrictions through AppArmor or seccomp.

I'm not a Proxmox expert, but it's clear that something is blocking access to the time namespace in your environment. This restriction needs to be addressed for nsenter to work properly.

If modifying the LXC configuration or host settings is not an option, you might need to use the ShellHub binary directly.

[KimiyaMorozova]

The nsenter Error only consists when connecting to another docker agent. When the Standalone option is used the error does not appear, however this log entry does still occour on the server side.

ssh-1      | {"cols":80,"device":"d4995bc4de6651e62064aec8ef1c347169755f4708620771d72f4f015d3e81f6","error":"failed to read the message from socket\nread tcp 172.18.0.7:8080-\u003e172.18.0.8:53374: use of closed network connection","ip":"10.10.12.92","level":"error","msg":"failed to read the message from the client","rows":24,"time":"2025-03-11T14:46:07Z","user":"root"}

[KimiyaMorozova]

After Installing the Server in an VM and investigating some more i found these logs in the Client.
I hope they Help a little more.

Mar 25 08:32:21 ANK-CTDash runc[24762]: time="2025-03-25T08:32:21Z" level=info msg="Starting ShellHub" mode=multi-user version=v0.18.3
Mar 25 08:32:21 ANK-CTDash runc[24762]: time="2025-03-25T08:32:21Z" level=info msg="Listening for connections" mode=multi-user preferred_>
Mar 25 08:32:21 ANK-CTDash runc[24762]: time="2025-03-25T08:32:21Z" level=info msg="Server connection established" hostname=bc-24-11-94-e>
Mar 25 08:32:21 ANK-CTDash runc[24762]: time="2025-03-25T08:32:21Z" level=info msg="Sleeping for 24 hours" mode=multi-user preferred_host>
Mar 25 08:37:25 ANK-CTDash runc[24762]: time="2025-03-25T08:37:25Z" level=info msg="Using password authentication" user=root
Mar 25 08:37:25 ANK-CTDash runc[24762]: time="2025-03-25T08:37:25Z" level=info msg="New session request"
Mar 25 08:37:25 ANK-CTDash runc[24762]: time="2025-03-25T08:37:25Z" level=info msg="Request type got" type=shell
Mar 25 08:37:25 ANK-CTDash runc[24762]: time="2025-03-25T08:37:25Z" level=warning msg="inappropriate ioctl for device"
Mar 25 08:37:25 ANK-CTDash runc[24762]: panic: runtime error: invalid memory address or nil pointer dereference
Mar 25 08:37:25 ANK-CTDash runc[24762]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x95dad3]
Mar 25 08:37:25 ANK-CTDash runc[24762]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x95dad3]
Mar 25 08:37:25 ANK-CTDash runc[24762]: goroutine 29 [running]:
Mar 25 08:37:25 ANK-CTDash runc[24762]: os.(*File).Name(...)
Mar 25 08:37:25 ANK-CTDash runc[24762]:         /usr/local/go/src/os/file.go:56
Mar 25 08:37:25 ANK-CTDash runc[24762]: github.com/shellhub-io/shellhub/pkg/agent/server/modes/host.(*Sessioner).Shell(0xc000494300, {0xc>
Mar 25 08:37:25 ANK-CTDash runc[24762]:         /go/src/github.com/shellhub-io/shellhub/pkg/agent/server/modes/host/sessioner.go:65 +0x173
Mar 25 08:37:25 ANK-CTDash runc[24762]: github.com/shellhub-io/shellhub/pkg/agent/server.(*Server).sessionHandler(0xc000424000, {0xc92708, 0xc0004a0340})
Mar 25 08:37:25 ANK-CTDash runc[24762]:         /go/src/github.com/shellhub-io/shellhub/pkg/agent/server/session.go:121 +0x6cf
Mar 25 08:37:25 ANK-CTDash runc[24762]: github.com/gliderlabs/ssh.(*session).handleRequests.func1()
Mar 25 08:37:25 ANK-CTDash runc[24762]:         /go/pkg/mod/github.com/shellhub-io/ssh@v0.0.0-20230224143412-edd48dfd6eea/session.go:263 
Mar 25 08:37:25 ANK-CTDash runc[24762]: created by github.com/gliderlabs/ssh.(*session).handleRequests in goroutine 27
Mar 25 08:37:25 ANK-CTDash runc[24762]:         /go/pkg/mod/github.com/shellhub-io/ssh@v0.0.0-20230224143412-edd48dfd6eea/session.go:262 >
Mar 25 08:37:25 ANK-CTDash systemd[1]: shellhub-agent.service: Main process exited, code=exited, status=2/INVALIDARGUMENT`