Provisioning new nodes fails with Flag provided but not defined

[d3473r]

Hi,

I upgraded Rancher to 2.7, and now the provisioning of new nodes fails with:
Flag provided but not defined: -hetzner-ssh-port:Timeout waiting for ssh key

The Flag that is missing is different every time, one time -hetzner-ssh-port, other times -hetzner-ssh-user or some other flags from https://github.com/JonasProgrammer/docker-machine-driver-hetzner#environment-variables-and-default-values

Was something changed, and when yes how do i provide these flags?

Rancher logs:

rancher    | 2023/07/18 12:53:40 [INFO] Creating jail for c-zdnwk
rancher    | 2023/07/18 12:53:40 [INFO] [node-controller] Provisioning node worker2
rancher    | 2023/07/18 12:53:40 [INFO] [node-controller] [cluster-id: c-zdnwk] Incorrect Usage.
rancher    | 2023/07/18 12:53:40 [INFO] [node-controller] [cluster-id: c-zdnwk] 
rancher    | 2023/07/18 12:53:40 [INFO] [node-controller] [cluster-id: c-zdnwk] Usage: docker-machine create [OPTIONS] [arg...]
rancher    | 2023/07/18 12:53:40 [INFO] [node-controller] [cluster-id: c-zdnwk] 
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk] Create a machine
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk] 
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk] Description:
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    Run 'rancher-machine create --driver name --help' to include the create flags for that driver in the help text.
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk] 
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk] Options:
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --custom-install-script                                          Use a custom provisioning script instead of installing docker
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --driver, -d "virtualbox"                                        Driver to create machine with. [$MACHINE_DRIVER]
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --engine-env [--engine-env option --engine-env option]           Specify environment variables to set in the engine
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --engine-insecure-registry [--engine-insecure-registry option --engine-insecure-registry option]  Specify insecure registries to allow with the created engine
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --engine-install-url "https://get.docker.com"                    Custom URL to use for engine installation [$MACHINE_DOCKER_INSTALL_URL]
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --engine-label [--engine-label option --engine-label option]     Specify labels for the created engine
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --engine-opt [--engine-opt option --engine-opt option]           Specify arbitrary flags to include with the created engine in the form flag=value
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --engine-registry-mirror [--engine-registry-mirror option --engine-registry-mirror option]                Specify registry mirrors to use [$ENGINE_REGISTRY_MIRROR]
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --engine-storage-driver                                          Specify a storage driver to use with the engine
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --hetzner-api-token                                              Project-specific Hetzner API token [$HETZNER_API_TOKEN]
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --hetzner-existing-key-id "0"                                    Existing key ID to use for server; requires --hetzner-existing-key-path [$HETZNER_EXISTING_KEY_ID]
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --hetzner-existing-key-path                                      Path to existing key (new public key will be created unless --hetzner-existing-key-id is specified) [$HETZNER_EXISTING_KEY_PATH]
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --hetzner-image "ubuntu-16.04"                                   Image to use for server creation [$HETZNER_IMAGE]
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --hetzner-image-id "0"                                           Image to use for server creation [$HETZNER_IMAGE_ID]
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --hetzner-server-location                                        Location to create machine at [$HETZNER_LOCATION]
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --hetzner-server-type "cx11"                                     Server type to create [$HETZNER_TYPE]
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --hetzner-user-data                                              Cloud-init based User data [$HETZNER_USER_DATA]
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --swarm                                                          Configure Machine to join a Swarm cluster
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --swarm-addr                                                     addr to advertise for Swarm (default: detect and use the machine IP)
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --swarm-discovery                                                Discovery service to use with Swarm
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --swarm-experimental                                             Enable Swarm experimental features
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --swarm-host "tcp://0.0.0.0:3376"                                ip/socket to listen on for Swarm master
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --swarm-image "swarm:latest"                                     Specify Docker image to use for Swarm [$MACHINE_SWARM_IMAGE]
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --swarm-join-opt [--swarm-join-opt option --swarm-join-opt option]Define arbitrary flags for Swarm join
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --swarm-master                                                   Configure Machine to be a Swarm master
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --swarm-opt [--swarm-opt option --swarm-opt option]              Define arbitrary flags for Swarm master
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --swarm-strategy "spread"                                        Define a default scheduling strategy for Swarm
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] [cluster-id: c-zdnwk]    --tls-san [--tls-san option --tls-san option]                    Support extra SANs for TLS certs
rancher    | 2023/07/18 12:53:41 [INFO] [node-controller] Generating and uploading node config worker2
rancher    | 2023/07/18 12:53:56 [ERROR] error syncing 'c-zdnwk/m-5n4lk': handler node-controller: flag provided but not defined: -hetzner-ssh-port, requeuing

Regards

[jomovo]

We are experiencing the same issue using rancher v2.7.2 and docker-machine driver v4.1.2

[MirosiakYakipara]

how did you solve the problem?