Add host.ssh_port option and refactor gateway initialization #1090

kke · 2019-02-19T12:33:17Z

Replaces #1071

Adds ssh_port option to host options:

hosts:
  - address: 10.0.0.1
    ssh_port: 2022

And bastion options:

hosts:
  - address: 10.0.0.1
    ssh_port: 22
    bastion:
      address: 10.0.0.2
      ssh_port: 20202

Refactored the Net::SSH::Gateway from SSH::Client#gateway to Transport::Gateway for better control (should benefit the disconnecting in #1003) and because bastion.host.ssh.gateway opened extra SSH connections.

Also adds the missing ssh_proxy_command attribute to bastion.

All k8s-client traffic is now tunneled through ssh.

ConfigureClient phase is replaced with WarmUpClientCache phase.

lib/pharos/configuration/bastion.rb

lib/pharos/configuration/host.rb

lib/pharos/transport.rb

kke · 2019-02-26T08:15:16Z

PTAL

kke · 2019-02-26T09:48:22Z

This probably still needs some way to track clients that are using the gateway, otherwise it will shutdown a gateway that is still used by some other client.

#1088 will likely need something from this.

kke · 2019-02-27T14:43:46Z

Files changed: 28. Seems like this grew out of proportion.

kke · 2019-03-01T14:36:37Z

lib/pharos/transport/ssh.rb

-      def bastion
-        @bastion ||= @opts.delete(:bastion)
+      def to_s
+        "SSH #{host.user}@#{host.address}:#{host.ssh_port}#{' using proxy command' if host.ssh_proxy_command}#{" via #{@gateway}->127.0.0.1:#{@port} " if @gateway}"


kke · 2019-03-01T15:15:24Z

E2e now uses bastion and proxy command

…ros-cluster into feature/gateway_refactorings

jakolehm · 2019-03-11T13:07:45Z

Conflict..

…ros-cluster into feature/gateway_refactorings

kke · 2019-03-11T13:28:18Z

Unconflict..

lib/pharos/addon.rb

jakolehm · 2019-03-12T12:52:14Z

lib/pharos/configuration/host.rb

      def short_hostname
        return nil unless hostname

        hostname.split('.').first
      end

+      # @return [Hash]
+      def ssh_options


IMO this should be under Transport (for example Transport::SshHelpers module)

jakolehm · 2019-03-12T12:54:29Z

lib/pharos/configuration/host.rb

+      end
+
+      # @return [Pharos::Transport::Gateway]
+      def gateway


Not completely happy about the idea that Host is becoming a "god" object that knows everything. For example gateway requires host and Host requires Gateway because of kube_client 🤔 /cc @jnummelin

Probably all ssh specific stuff should be under Transport (somehow).

Yes, I agree. Now Host, Bastion, Gateway all entangle pretty badly and really hard to follow which is used where.

I moved stuff around.

Now there's Pharos::Kube::Client that understands Configuration::Host, so that host can just do @kube_client ||= Pharos::Kube::Client.new(self).

The host ssh options are generated in Pharos::Transport::SSH.options_for(host).

The Host now has:

def gateway; @gateway ||= Pharos::Transport::Gateway.new(self); end

def transport; @transport ||= Pharos::Transport.for(self); end

def kube_client; @kube_client ||= Pharos::Kube::Client.new(self); end

def disconnect

Alternative to having these memoized in Host I guess would be to go back to
Pharos::SshManager.clients[host.address] / Pharos::GatewayManager.gateways[host.address] type of connection collections or maybe have some sort of Pharos::Transport delegator/decorator and just have host.transport and all those kube_client / exec / file / gateway stuff would be contained there.

IMO the latter sounds pretty good

I still think that gateway should be masked behind transport. Transport already gets host (like Gateway) as a parameter.

@kke @jnummelin are you fine with gateway being here?

I'm now WIPping with https://github.com/kontena/net-ssh-proxy-gateway

…eature/gateway_refactorings

kke · 2019-03-19T12:50:27Z

#1191 fixes #1062
#1072 and #1088 can be fixed more simply separately now with #1185

Kimmo Lehto added 2 commits February 19, 2019 14:27

Refactor transport gateway initialization and add ssh_port

fc8fb66

Add ssh_proxy_command to bastion too

dec8a10

kke added enhancement New feature or request chore labels Feb 19, 2019

kke requested a review from jakolehm February 19, 2019 12:33

kke mentioned this pull request Feb 19, 2019

Add hosts.ssh_port option #1071

Closed

kke commented Feb 19, 2019

View reviewed changes

lib/pharos/configuration/bastion.rb Outdated Show resolved Hide resolved

kke commented Feb 19, 2019

View reviewed changes

lib/pharos/configuration/host.rb Outdated Show resolved Hide resolved

No jumps

82505de

kke commented Feb 19, 2019

View reviewed changes

lib/pharos/transport.rb Show resolved Hide resolved

kke mentioned this pull request Feb 19, 2019

Add "pharos reboot" command for rebooting hosts or the entire cluster #1003

Closed

Kimmo Lehto added 2 commits February 19, 2019 16:49

Finetunings 2

01c32d9

Rubocop

a39df3e

kke mentioned this pull request Feb 22, 2019

Move kube_client setup into config #855

Closed

Kimmo Lehto added 10 commits February 26, 2019 14:34

Taking it further

2d42c76

Merge branch 'master' into feature/gateway_refactorings

97da2e1

Final batch of refactorings

11dd947

Merge branch 'master' into feature/gateway_refactorings

fc3b953

Added 127.0.0.1 as extra sans so localhost and 127.0.0.1 both work

dad61de

Adapt non-oss specs

7c83dad

Fix #1116

04007f5

Localhost is not a real host

8a150f9

Extra empty line

d8965b1

Can't warm up the client cache if there's no master up

388977a

kke added bug Something isn't working and removed chore labels Feb 27, 2019

kke mentioned this pull request Feb 28, 2019

Run all phases in parallel mode #1125

Closed

kke commented Mar 1, 2019

View reviewed changes

Even prettier

def8c0a

Kimmo Lehto added 7 commits March 6, 2019 16:09

Merge branch 'master' into feature/gateway_refactorings

05b9101

Beautify

3665ed4

Merge branch 'feature/gateway_refactorings' of github.com:kontena/pha…

6dbfb3d

…ros-cluster into feature/gateway_refactorings

Pharos dev console v2

2648e7e

Rubocop

e517c33

Move rubocop hints to beautify console startup

192e780

Dont define Console module unless pry is defined

2ba8fb4

kke mentioned this pull request Mar 11, 2019

Bastion opens an extra ssh connection #1072

Closed

Kimmo Lehto added 2 commits March 11, 2019 15:11

Merge branch 'master' into feature/gateway_refactorings

325b4c7

Merge branch 'feature/gateway_refactorings' of github.com:kontena/pha…

1338353

…ros-cluster into feature/gateway_refactorings

jakolehm reviewed Mar 12, 2019

View reviewed changes

Kimmo Lehto added 9 commits March 12, 2019 16:05

Move stuff around

d7989cd

Missing commit

a485789

Merge branch 'master' into feature/gateway_refactorings

0275771

Up to date with helm

30a3f55

Merge remote-tracking branch 'origin/feature/pharos-console-2' into f…

b933ce6

…eature/gateway_refactorings

Detect cyclic bastion loops

ab5cc4e

Use net-ssh-proxy-gateway, refactor Transport::Gateway away

d5568ed

Reconnect and retry ssh execs on IOError

7ed962b

wip

bcdfed9

kke mentioned this pull request Mar 15, 2019

Drop Net::SSH::Gateway dependency #1185

Merged

Merge branch 'master' into feature/gateway_refactorings

4432d57

kke closed this Mar 19, 2019

kke deleted the feature/gateway_refactorings branch March 19, 2019 12:50

Add host.ssh_port option and refactor gateway initialization #1090

Add host.ssh_port option and refactor gateway initialization #1090

Uh oh!

Conversation

kke commented Feb 19, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kke commented Feb 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kke commented Feb 26, 2019

Uh oh!

kke commented Feb 27, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kke commented Mar 1, 2019

Uh oh!

jakolehm commented Mar 11, 2019

Uh oh!

kke commented Mar 11, 2019

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kke Mar 12, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kke commented Mar 19, 2019

Uh oh!

Uh oh!

kke commented Feb 19, 2019 •

edited

Loading

kke commented Feb 26, 2019 •

edited

Loading

kke Mar 12, 2019 •

edited

Loading