Windows @local/@docker support (Hello World example from tutorial doesn't work)

[karolzlot]

Describe the bug

I got error as in log below. I repeated command, but the error persist.

PowerShell 7.1.3
Copyright (c) Microsoft Corporation.

https://aka.ms/powershell
Type 'help' to get help.

PS C:\moje\programowanie\projekty2021_2\pyinfra> & c:/moje/programowanie/projekty2021_2/pyinfra/.venv_win/Scripts/Activate.ps1
(pyinfra_venv) PS C:\moje\programowanie\projekty2021_2\pyinfra> pyinfra @docker/ubuntu:18.04 exec -- echo "hello world"
--> Loading config...
--> Loading inventory...        

--> Connecting to hosts...      
    [@docker/ubuntu:18.04] Connected

--> Proposed changes:
    Groups: @docker
    [@docker/ubuntu:18.04]   Operations: 1   Commands: 1   

--> Beginning operation run...
--> Starting operation: Server/Shell (echo hello world)    
[@docker/ubuntu:18.04] 'sh' is not recognized as an internal or external command,
[@docker/ubuntu:18.04] operable program or batch file.
    [@docker/ubuntu:18.04] Error
localhost: sha256:3a3098f7fb5bb4affdd2b9a2c31579ea4c83a144d2ca9bbb7bbe8cc803b8b40a
localhost: 45aa42d84b87499a9aa88a9592c241497c4b7466c1c5e3f31cba7737fcd03f9f
    [@docker/ubuntu:18.04] docker build complete, image ID: 3a3098f7fb5b
No hosts remaining!
--> pyinfra error: 
(pyinfra_venv) PS C:\moje\programowanie\projekty2021_2\pyinfra> 

To Reproduce

pyinfra @docker/ubuntu:18.04 exec -- echo "hello world"

Expected behavior

No error

Meta

• Include output of pyinfra --support:

pyinfra --support
--> Support information:

    If you are having issues with pyinfra or wish to make feature requests, please
    check out the GitHub issues at https://github.com/Fizzadar/pyinfra/issues .
    When adding an issue, be sure to include the following:

    System: Windows
      Platform: Windows-10-10.0.19042-SP0
      Release: 10
      Machine: AMD64
    pyinfra: v1.4.11
    Executable: C:\moje\programowanie\projekty2021_2\pyinfra\.venv_win\Scripts\pyinfra.exe
    Python: 3.9.6 (CPython, MSC v.1929 64 bit (AMD64))

• How was pyinfra installed (source/pip)?

pip in venv

• Include pyinfra-debug.log (if one was created)

not created

• Consider including output with -vv and --debug.

 pyinfra -vv @docker/ubuntu:18.04 exec -- echo "hello world" 
--> Loading config...
--> Loading inventory...

--> Connecting to hosts...
localhost: >>> docker container inspect ubuntu:18.04
localhost: >>> docker run -d ubuntu:18.04 tail -f /dev/null
    [@docker/ubuntu:18.04] Connected

--> Proposed changes:
    Groups: @docker
    [@docker/ubuntu:18.04]   Operations: 1   Commands: 1

--> Beginning operation run...
--> Starting operation: Server/Shell (echo hello world)
[@docker/ubuntu:18.04] >>> sh -c 'docker exec -i 7a450bc13cd064cf993ab2988ba8775fbdeea08e1349bf91aedf9c9fc01d02f8 sh -c '"'"'sh -c '"'"'"'"'"'"'"'"'echo hello world'"'"'"'"'"'"'"'"''"'"''
[@docker/ubuntu:18.04] 'sh' is not recognized as an internal or external command,
[@docker/ubuntu:18.04] operable program or batch file.
    [@docker/ubuntu:18.04] Error
localhost: >>> docker commit 7a450bc13cd064cf993ab2988ba8775fbdeea08e1349bf91aedf9c9fc01d02f8
localhost: sha256:0b141c126885c0a356078ffad346607dfacd1dbfb82babeef90d2712686502cc
localhost: >>> docker rm -f 7a450bc13cd064cf993ab2988ba8775fbdeea08e1349bf91aedf9c9fc01d02f8
localhost: 7a450bc13cd064cf993ab2988ba8775fbdeea08e1349bf91aedf9c9fc01d02f8
    [@docker/ubuntu:18.04] docker build complete, image ID: 0b141c126885
No hosts remaining!
--> pyinfra error: 

pyinfra --debug @docker/ubuntu:18.04 exec -- echo "hello world"
    [pyinfra_cli\main] Deploy directory remains as cwd
--> Loading config...
--> Loading inventory...
    [pyinfra_cli\inventory] Creating fake inventory...

--> Connecting to hosts...
[pyinfra\api\connectors\util] --> Waiting for exit status...
[pyinfra\api\connectors\util] --> Command exit status: 1
[pyinfra\api\connectors\util] --> Waiting for exit status...
[pyinfra\api\connectors\util] --> Command exit status: 0
    [@docker/ubuntu:18.04] Connected
    [pyinfra\api\state] Activating host: @docker/ubuntu:18.04
    [pyinfra\api\operation] Adding operation, {'Server/Shell'}, opOrder=(0,), opHash=784a97bf1955d5f7a2b9dd6c1e371e17b73c42bc

--> Proposed changes:
    Groups: @docker
    [@docker/ubuntu:18.04]   Operations: 1   Commands: 1

--> Beginning operation run...
--> Starting operation: Server/Shell (echo hello world)
    [pyinfra\api\operations] Starting operation Server/Shell on @docker/ubuntu:18.04
[pyinfra\api\connectors\local] --> Running command on localhost: sh -c 'docker exec -i e1f7dbe16ca169f6f33d6f33ca42aa220a249466d26fa35e8cc7556b4da7aaab sh -c '"'"'sh -c '"'"'"'"'"'"'"'"'echo hello world'"'"'"'"'"'"'"'"''"'"''
[@docker/ubuntu:18.04] 'sh' is not recognized as an internal or external command,
[@docker/ubuntu:18.04] operable program or batch file.
[pyinfra\api\connectors\util] --> Waiting for exit status...
[pyinfra\api\connectors\util] --> Command exit status: 1
    [@docker/ubuntu:18.04] Error
    [pyinfra\api\state] Failing hosts: @docker/ubuntu:18.04
localhost: sha256:3c26a9596d24efca7f17e60683f33a4193fc1004fa8b022a01ce87c6d92f9141
[pyinfra\api\connectors\util] --> Waiting for exit status...
[pyinfra\api\connectors\util] --> Command exit status: 0
localhost: e1f7dbe16ca169f6f33d6f33ca42aa220a249466d26fa35e8cc7556b4da7aaab
[pyinfra\api\connectors\util] --> Waiting for exit status...
[pyinfra\api\connectors\util] --> Command exit status: 0
    [@docker/ubuntu:18.04] docker build complete, image ID: 3c26a9596d24
No hosts remaining!
--> pyinfra error: 

[Fizzadar]

Hi @karolzlot! I believe this is due to the @docker trying to execute the local docker command insdie a unix sh shell. I've not got Windows access currently but need to double check this - I think that both @docker and @local connectors will be broken due to this.

However - it's possible to override this value via the --shell-executable CLI flag. I've not tested but one of these may work:

pyinfra --shell-executable ps @docker/ubuntu:18.04 exec -- echo "hello world"
pyinfra --shell-executable pws @docker/ubuntu:18.04 exec -- echo "hello world"

I'll be able to test this properly on my Windows laptop over the next few days.

[hornyjailfish]

Because this issue related let me put some research on issue and in general how to make @local works on windows:

1. some facts (Home, Where, Date) force shell-executable=cmd so cli option has no effect and dont work at all because @local use cmd -c and its wrong syntax

2. escaping problems...pyinfra send command in single quotes so powershell -c interprets it as string and not command to execute i.e. I've try to rewrite facts that use cmd and cannot execute command because of this

--> Gathering facts...
    [pyinfra\api\facts] Getting fact: windows.Home () (ensure_hosts: None)
[pyinfra\connectors\local] --> Running command on localhost: pwsh -NoLogo -NoProfile -c 'Write-Host $env:HOMEPATH'
[@local] >>> pwsh -NoLogo -NoProfile -c 'Write-Host $env:HOMEPATH'
[@local] Write-Host $env:HOMEPATH
[pyinfra\connectors\util] --> Waiting for exit status...
[pyinfra\connectors\util] --> Command exit status: 0
    [@local] Loaded fact windows.Home

--> Fact data for: windows.Home
{
    "@local": "Write-Host $env:HOMEPATH\r"
}

3. problems with parsing cli commands in powershell (could be my personal issue or related to 2) i.e.

   $>pyinfra.exe @local --shell-executable=pwsh exec -- ls
   Invalid operations: ('ls',)
   

   same command in cmd works as expected

[Fizzadar]

Moving this to pyinfra-windows for potential support via a new connector specific for Windows @windowslocal or something..