Principal "x.x.x.x" not in the set of valid principals for given certificate

[guidorugo]

I'm trying to join a ssh session with join in my command line but I'm getting the error Host validation failed: ssh: principal "x.x.x.x" not in the set of valid principals for given certificate.

guidorugo@Weed0-mac` ~ [1]> tsh ls
Node Name        Address          Labels
---------------- ---------------- -----------------------
teamspeak-server x.x.x.x:3022
teleport-master  127.0.0.1:3022   hostname=grugo-teleport

guidorugo@Weed0-mac ~> tsh ssh ubuntu@teamspeak-server
ubuntu@opentracker:~$ teleport status
User ID     : grugo, logged in as ubuntu from y.y.y.y 59574 3022
Cluster Name: nothingtoseehere
Host UUID   : aaaaaaaa-8474-4f8c-a621-05a2396a1c0e
Session ID  : aaaaaaaaaaa-943b-4a82-9ca4-493da0d7f533
Session URL : https://<proxyhost>:3080/web/cluster/nothingtoseehere/console/session/aaaaaaaaaaa-943b-4a82-9ca4-493da0d7f533
ubuntu@opentracker:~$ teleport version
Teleport v8.2.0 git:v8.2.0-0-gbfd1780 go1.17.3

guidorugo@Weed0-mac ~ [1]> teleport version
Teleport v8.2.0 git: go1.17.6
guidorugo@Weed0-mac ~> tsh login
> Profile URL:        https://nothingtoseehere:443
  Logged in as:       grugo
  Cluster:            nothingtoseehere
  Roles:              access, auditor, editor
  Logins:             root, ubuntu, ec2-user
  Kubernetes:         enabled
  Valid until:        2022-02-10 23:45:16 +0000 GMT [valid for 11h53m0s]
  Extensions:         permit-agent-forwarding, permit-port-forwarding, permit-pty

guidorugo@Weed0-mac ~ [1]> tsh -d join --login=ubuntu aaaaaaaaaaa-943b-4a82-9ca4-493da0d7f533
INFO [CLIENT]    [KEY AGENT] Connected to the system agent: "/private/tmp/com.apple.launchd.wsyC4O9Trh/Listeners" client/api.go:2973
DEBU [KEYSTORE]  Returning Teleport TLS certificate "/Users/guidorugo/.tsh/keys/nothingtoseehere/grugo-x509.pem" valid until "2022-02-10 23:45:16 +0000 UTC". client/keystore.go:283
DEBU [KEYSTORE]  Reading certificates from path "/Users/guidorugo/.tsh/keys/nothingtoseehere/grugo-ssh/nothingtoseehere-cert.pub". client/keystore.go:306
INFO [KEYAGENT]  Loading SSH key for user "grugo" and cluster "nothingtoseehere". client/keyagent.go:180
INFO [CLIENT]    Connecting proxy=nothingtoseehere:443 login="root" client/api.go:2166
DEBU [KEYSTORE]  Returning Teleport TLS certificate "/Users/guidorugo/.tsh/keys/nothingtoseehere/grugo-x509.pem" valid until "2022-02-10 23:45:16 +0000 UTC". client/keystore.go:283
DEBU [KEYAGENT]  "Checking key: [email protected] <verylongrsa>+w==\n." client/keyagent.go:338
DEBU [KEYAGENT]  Validated host nothingtoseehere:443. client/keyagent.go:344
INFO [CLIENT]    Successful auth with proxy nothingtoseehere:443. client/api.go:2173
DEBU [CLIENT]    Found clusters: [{"name":"nothingtoseehere","lastconnected":"2022-02-10T11:59:29.325182492Z","status":"online"}] client/client.go:110
DEBU [KEYSTORE]  Returning Teleport TLS certificate "/Users/guidorugo/.tsh/keys/nothingtoseehere/grugo-x509.pem" valid until "2022-02-10 23:45:16 +0000 UTC". client/keystore.go:283
INFO [CLIENT]    Client= connecting to node=x.x.x.x on cluster nothingtoseehere client/client.go:945
DEBU [KEYSTORE]  Returning Teleport TLS certificate "/Users/guidorugo/.tsh/keys/nothingtoseehere/grugo-x509.pem" valid until "2022-02-10 23:45:16 +0000 UTC". client/keystore.go:283
DEBU [KEYSTORE]  Returning Teleport TLS certificate "/Users/guidorugo/.tsh/keys/nothingtoseehere/grugo-x509.pem" valid until "2022-02-10 23:45:16 +0000 UTC". client/keystore.go:283
DEBU [KEYSTORE]  Reading certificates from path "/Users/guidorugo/.tsh/keys/nothingtoseehere/grugo-ssh/nothingtoseehere-cert.pub". client/keystore.go:306
DEBU [KEYSTORE]  Returning Teleport TLS certificate "/Users/guidorugo/.tsh/keys/nothingtoseehere/grugo-x509.pem" valid until "2022-02-10 23:45:16 +0000 UTC". client/keystore.go:283
DEBU [CLIENT]    MFA not required for access. client/client.go:352
DEBU [KEYAGENT]  "Checking key: [email protected] <verylongsrsa>==\n." client/keyagent.go:338
DEBU [KEYAGENT]  Host validation failed: ssh: principal "x.x.x.x" not in the set of valid principals for given certificate: ["aaaaaaaa-8474-4f8c-a621-05a2396a1c0e.nothingtoseehere" "aaaaaaaa-8474-4f8c-a621-05a2396a1c0e" "teamspeak-server.nothingtoseehere" "teamspeak-server" "localhost" "127.0.0.1" "::1"]. client/keyagent.go:341

ERROR REPORT:
Original Error: *trace.AccessDeniedError access denied to ubuntu connecting to x.x.x.x on cluster nothingtoseehere
Stack Trace:
        /private/tmp/teleport-20220209-67498-106wg53/teleport-8.2.0/lib/client/client.go:1046 github.com/gravitational/teleport/lib/client.(*ProxyClient).ConnectToNode
        /private/tmp/teleport-20220209-67498-106wg53/teleport-8.2.0/lib/client/api.go:1544 github.com/gravitational/teleport/lib/client.(*TeleportClient).Join
        /private/tmp/teleport-20220209-67498-106wg53/teleport-8.2.0/tool/tsh/tsh.go:1697 main.onJoin.func1
        /private/tmp/teleport-20220209-67498-106wg53/teleport-8.2.0/lib/client/api.go:514 github.com/gravitational/teleport/lib/client.RetryWithRelogin
        /private/tmp/teleport-20220209-67498-106wg53/teleport-8.2.0/tool/tsh/tsh.go:1696 main.onJoin
        /private/tmp/teleport-20220209-67498-106wg53/teleport-8.2.0/tool/tsh/tsh.go:631 main.Run
        /private/tmp/teleport-20220209-67498-106wg53/teleport-8.2.0/tool/tsh/tsh.go:295 main.main
        /usr/local/Cellar/go/1.17.6/libexec/src/runtime/proc.go:255 runtime.main
        /usr/local/Cellar/go/1.17.6/libexec/src/runtime/asm_amd64.s:1581 runtime.goexit
User Message: access denied to ubuntu connecting to x.x.x.x on cluster nothingtoseehere

[scottpgallagher]

When you just tsh join can you try using the node name that is shown when you do a tsh ls or using the host UUID (not session ID) when you do a teleport status on the node?

[guidorugo]

ubuntu@opentracker:~$ teleport status
User ID     : grugo, logged in as ubuntu from y.y.y.y 59574 3022
Cluster Name: nothingtoseehere
Host UUID   : aaaaaaaaa-8474-4f8c-a621-05a2396a1c0e
Session ID  : aaaaaaaaa-943b-4a82-9ca4-493da0d7f533
Session URL : https://<proxyhost>:3080/web/cluster/nothingtoseehereconsole/session/aaaaaaaaa-943b-4a82-9ca4-493da0d7f533

guidorugo@Weed0-mac ~ [1]> tsh join --proxy=nothingtoseehere:443 --login=ubuntu aaaaaaaaaa-943b-4a82-9ca4-493da0d7f533
ERROR: access denied to ubuntu connecting to x.x.x.x on cluster nothingtoseehere

guidorugo@Weed0-mac ~ [1]> tsh join --proxy=nothingtoseehere:443 --login=ubuntu aaaaaaaaaa-8474-4f8c-a621-05a2396a1c0e
ERROR: session 'aaaaaaaaaaaaa-8474-4f8c-a621-05a2396a1c0e' not found or it has ended

guidorugo@Weed0-mac ~ [1]> tsh ls
Node Name        Address          Labels
---------------- ---------------- -----------------------
teamspeak-server x.x.x.x:3022
teleport-master  127.0.0.1:3022   

guidorugo@Weed0-mac ~> tsh join --proxy=nothingtoseehere:443 --login=ubuntu teamspeak-server
ERROR: 'teamspeak-server' is not a valid session ID (must be GUID)

[webvictim]

Try updating the /etc/teleport.yaml config file on your teamspeak-server node to include the following, then restarting Teleport and joining the session again:

ssh_service:
  public_addr: x.x.x.x:3022 # replace x.x.x.x with the actual IP that you redacted

This is in all likelihood a bug/regression with session joining, but this workaround should regenerate the principals on the host certificate to include the host's IP address. This means that the principals will match when you try to join the session and things will work.

In all likelihood, if you joined the node to the proxy via tunnel (using port 443 or 3080) rather than directly to the auth server (port 3025) this should work without issue.

[guidorugo]

Tried both solutions and works (adding public_addr to node config file and joining the node via tunnel).
@webvictim Would you like me to create an issue about this ?

[webvictim]

@guidorugo Yes, please create an issue!