pkg/bindings: add dial-stdio fallback for SSH connections

[parmisean]

Adds a fallback mechanism for remote SSH connections when the [email protected] channel type is unsupported, such as when using Tailscale SSH or certain hardened SSH configurations.

When the primary connection method fails, Podman will now fallback to executing podman system dial-stdio over an SSH session to establish the bridge. The fallback state is stored after the first failure to prevent redundant failing connection attempts during command execution.

Fixes #27814.

Testing

The following commands were manually verified using a local build of this branch to ensure functionality, targeting remote tailscale SSH with default and custom socket paths, and remote OpenSSHD hosts to verify against regressions.

• ./bin/podman --connection [remote] ps -a
• ./bin/podman --connection [remote] pull alpine
• ./bin/podman --connection [remote] images -a
• ./bin/podman --connection [remote] volume ls

Checklist

Ensure you have completed the following checklist for your pull request to be reviewed:

• Certify you wrote the patch or otherwise have the right to pass it on as an open-source patch by signing all
  commits. (git commit -s). (If needed, use git commit -s --amend). The author email must match
  the sign-off email address. See CONTRIBUTING.md
  for more information.
• Referenced issues using Fixes: #00000 in commit message (if applicable)
• Tests have been added/updated (or no tests are needed)
• Documentation has been updated (or no documentation changes are needed)
• All commits pass make validatepr (format/lint checks)
• Release note entered in the section below (or None if no user-facing changes)

Does this PR introduce a user-facing change?

SSH remote connections now automatically fall back to `podman system dial-stdio` when the SSH server does not support direct-streamlocal channels.

[parmisean]

Thanks for the review @jankaluza, I appreciate the feedback! I'll dive into these comments this evening after work and push a new commit to address them

[parmisean]

@jankaluza the implementation significantly changed with the introduction of net.Pipe, so I opted to squash the additional commits to keep the history clean as a full re-review would likely be required.

I'm noticing a CICD error due to binary size growth, which I think is likely due to the net.Pipe import, and I may need some guidance here as to how we should resolve. I think we have the following options, any thoughts?

• Keep net.Pipe to provide a full net.Conn implementation with deadlines, and ignore binary size growth.
• Refactor to remove net.Pipe and implement deadline methods as no-op, relying on the http.Transport deadline implementation.
• Write a custom deadline implementation that wraps the ssh connection that allows us to provide deadlines without closing the underlying connection.

Let me know what you think, or if there's another approach I'm not considering that would match maintainer goals better.

[jankaluza]

@containers/podman-maintainers , What do you think?

I would probably go with option 2).

[parmisean]

@jankaluza I pushed an updated commit that uses no-op deadline methods to implement the net.Conn interface, with the new approach relying on the wrapping http.Transport deadline implementation instead.

It looks like I am still triggering the binary growth check. When running ./contrib/dependencies/dependencies.sh weight-diff main parmisean/implement-ssh-session-type, it shows several package imports growing that I wouldn't have expected from my change. I've attached a screenshot, any thoughts?

[packit-as-a-service]

[NON-BLOCKING] Packit jobs failed. @containers/packit-build please check. Everyone else, feel free to ignore.

[packit-as-a-service]

[NON-BLOCKING] Packit jobs failed. @containers/packit-build please check. Everyone else, feel free to ignore.

[packit-as-a-service]

[NON-BLOCKING] Packit jobs failed. @containers/packit-build please check. Everyone else, feel free to ignore.

[packit-as-a-service]

[NON-BLOCKING] Packit jobs failed. @containers/packit-build please check. Everyone else, feel free to ignore.

[parmisean]

@jankaluza Apologies for the force push again. I was still hitting binary growth CI failures because the check diffs against the original merge-base, which was counting upstream main changes against my PR after I merged main.

I rebased main onto my branch and force pushed to resolve, but unfortunately this breaks reviewing changes since the last review. Sorry for the extra friction, as its likely you'll have to look at the full diff again.

[jankaluza]

Is the path actually used anywhere?

It seems to be only used in the RemoteAddr, but this is called only in the tests. This looks suspicious to me.

[jankaluza]

I see it now I think. The url is passed to ssh.DialNet.

[parmisean]

The primary path passes the URL directly to ssh.DialNet, but the dial-stdio fallback was relying on the servers configuration to determine the socket path. After some digging, I realized we can easily fix this by leveraging the existing global --url flag, updating the ssh command to podman --url <path> system dial-stdio.

I've pushed a new commit to fix this and verified locally that custom socket paths now route correctly through the fallback

[jankaluza]

I was thinking if we need to handle also stderr here. But it seems the podman system dial-stdio only output stdout, so that's probably fine.

[parmisean]

Yep, from what I can tell podman system dial-stdio only communicates over stdin/stdout, so we wouldn't capture anything from stderr

[jankaluza]

I think this looks good. It would be really nice to have some functional tests for that, but I admit these would not be trivial to do and even our podman system dial-stdio test only tests the --help...

[parmisean]

I think this looks good. It would be really nice to have some functional tests for that, but I admit these would not be trivial to do and even our podman system dial-stdio test only tests the --help...

I agree, having functional tests for this would be ideal even if it would definitely be a heavy lift. Given the complexity and current state of the dial-stdio tests, it may make sense to defer on heavier automated testing here if you are comfortable with proceeding as is. I'm happy to open a separate issue to track adding comprehensive SSH/connection tests in the future if that's something the project wants to pursue.