Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Writable AzInstallFolder requirement breaks self-hosted runner #93

Closed
belcher-rok opened this issue Nov 20, 2023 · 4 comments
Closed

Writable AzInstallFolder requirement breaks self-hosted runner #93

belcher-rok opened this issue Nov 20, 2023 · 4 comments
Assignees

Comments

@belcher-rok
Copy link

The following code was recently added to Utils.getDefaultAzInstallFolder() via PR #91:

        ...

        if(Utils.isFolderExistAndWritable(defaultAzInstallFolder)){
            return defaultAzInstallFolder;
        }
        if(Utils.isFolderExistAndWritable(process.env.RUNNER_TOOL_CACHE)){
            return process.env.RUNNER_TOOL_CACHE;
        }
        return process.cwd();

It is difficult to know for certain, but I believe this new "writable" requirement is causing my self-hosted, custom runners to not find the "latest" installed Az module. What I can say for certain, is that azure/[email protected] works as expected, but 1.4.0 fails to find any Az modules (Get-Module -Name Az -ListAvailable returns nothing).

The command we use to load the Az module on the runner image is Save-Module -Path /usr/share/az_7.1.0 -Name az -RequiredVersion 7.1.0 -Force, which is exactly where the azure/powershell action expects it to be. So, the only explanation I can come up with is /usr/share/ is not writable on my runners.

Any chance we can remove the "writeable" requirement? It should not be necessary.

Details

Workflow invocation:

  my_job:
    runs-on:
      - self-hosted
    steps:
      - uses: actions/checkout@v3
      - uses: azure/powershell@v1
        with:
          azPSVersion: latest
          inlineScript: ./scripts/foo.ps1

Self hosted runner details (very briefly):

FROM summerwind/actions-runner:ubuntu-20.04
.
.
.
# Cache the Azure Powershell modules. Based on https://github.com/actions/virtual-environments/blob/d3de894568d2662083e3515b884c86e9326ea255/images/linux/scripts/installers/Install-AzureModules.ps1#L18
# Use the [Azure/powershell@v1](https://github.com/Azure/powershell/tree/v1) action with `azPSVersion: latest` to activate it.
RUN sudo pwsh -c 'Save-Module -Path /usr/share/az_7.1.0 -Name az -RequiredVersion 7.1.0 -Force'
.
.
.
@belcher-rok belcher-rok changed the title Writable Az Install Folder requirement breaks selh-hosted runner Writable Az Install Folder requirement breaks self-hosted runner Nov 20, 2023
@belcher-rok belcher-rok changed the title Writable Az Install Folder requirement breaks self-hosted runner Writable AzInstallFolder requirement breaks self-hosted runner Nov 20, 2023
@YanaXu
Copy link
Collaborator

YanaXu commented Nov 21, 2023

Hi @belcher-rok, sorry for your inconvenience.
First, please use v1.3.0 for a workaround.

As you know, before the latest release, Azure PowerShell action does not officially support self-hosted runners.
As I understand, you're using a docker container for a self-hosted runner. And you want to use Az 7.1.0. Is that right?
If so, could you replace the step Save-Module -Path /usr/share/az_7.1.0 -Name az -RequiredVersion 7.1.0 -Force with Install-Module -Name Az? Just FYI, you're using a very old version of Az and it's insecure. Could you please upgrade it to the latest one?

We're trying to support all self-hosted runners and that's why I can't apply the required change from you.
If you can't update the docker image used for self-hosted runners, or you can't apply the Az version in Azure PowerShell Action, could you provide more information about your workflow file, the failed debug log and the details about your docker image? Like its structure and how you configure it as a self-hosted runner. Thanks. We'll see how we can do it better.

@YanaXu YanaXu self-assigned this Nov 21, 2023
@belcher-rok
Copy link
Author

belcher-rok commented Nov 21, 2023

Hi @belcher-rok, sorry for your inconvenience. First, please use v1.3.0 for a workaround.

As you know, before the latest release, Azure PowerShell action does not officially support self-hosted runners. As I understand, you're using a docker container for a self-hosted runner. And you want to use Az 7.1.0. Is that right? If so, could you replace the step Save-Module -Path /usr/share/az_7.1.0 -Name az -RequiredVersion 7.1.0 -Force with Install-Module -Name Az? Just FYI, you're using a very old version of Az and it's insecure. Could you please upgrade it to the latest one?

Hi @YanaXu, Thanks for the quick reply.

Your understanding is correct. I've modified our runner image to Install-Module -Name Az -Scope AllUsers -Force. This avoids the error by having at least one version of Az always available (also the latest version).

Can you explain why getDefaultAzInstallFolder will not return a non-writable folder? I suppose it makes sense in the cases where the Action must install a version of Az, but in my case the version of Az I wanted was already available (saved in a non-writable folder). Perhaps getDefaultAzInstallFolder is doing too much? Maybe it should just return the default location of saved modules. If the Action decides it needs to install Az, then you call getWritableAzInstallFolder which contains the code to fallback to a writable location. Would that work?

@YanaXu
Copy link
Collaborator

YanaXu commented Nov 22, 2023

Hi @belcher-rok , the method getDefaultAzInstallFolder is trying to find a folder to download Az and import Az from it.
In the former version, Azure PowerShell Action do not support self-hosted runners. In your self-hosted runner, you tried to mock the behavior of GitHub-hosted Linux runners and that's why it worked before. But I think it only worked when you used latest Azure PowerShell version. Right? Now you can use other Azure PowerShell versions for Azure PowerShell Action.
It's glad to know your use scenario. Your suggestions and feedback are appreciated.

@belcher-rok
Copy link
Author

... I think it only worked when you used latest Azure PowerShell version. Right?

I never tried with anything other than latest, but I believe you are correct.

Now you can use other Azure PowerShell versions for Azure PowerShell Action.

...except any saved versions. The new Action requires my runners to have Az pre-installed, and it won't use any saved versions. This is okay for my organization since we only need a compatible version. If we ever do need a specific version, the Action will/should download it for us, however it comes at a cost -- adding ~45 seconds and ~750MB of ingress charges to every workflow job using a specific Az version. Again, this doesn't affect me right now, so I'll consider this issue resolved.

Thanks for your help @YanaXu.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants