Windows `PATH` no longer acts reliably / rearranges itself

[mrmundt]

Description

With a recent update to the Windows runner images, the PATH variable is no longer acting as expected. With no other changes introduced to our workflow, our Windows jobs are failing because the PATH variable is rearranging itself between steps.

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 20.04
• Ubuntu 22.04
• Ubuntu 24.04
• macOS 12
• macOS 13
• macOS 13 Arm64
• macOS 14
• macOS 14 Arm64
• macOS 15
• macOS 15 Arm64
• Windows Server 2019
• Windows Server 2022
• Windows Server 2025

Image version and build link

Failed build: https://github.com/Pyomo/pyomo/actions/runs/12397077388

The failure here is innocuous. There are two versions of the executable ipopt on the PATH, one that works as we expect and one that doesn't. We intentionally update the PATH to catch the correct one first, but somehow in the newer image version, the PATH is rearranging between steps.

Here is a good example of just looking at the PATH printouts (they are at the end of each step): https://github.com/mrmundt/pyomo/actions/runs/12673191510/job/35318997883

Is it regression?

Yes. Last working version: 20241201.2.0; https://github.com/Pyomo/pyomo/actions/runs/12201613092

Expected behavior

We expect PATH updates to act the same between Linux, OSX, and Windows, if using the same steps for all OS'. Specifically, we expect the PATH to remain consistent between steps when using the correct mechanism for updating it.

Actual behavior

The PATH updates are happening correctly in Linux and OSX. The PATH updates in Windows when requested but will inexplicably change order in between steps.

Repro steps

It's unclear exactly how to reproduce this, but I did minimize our workflow to a file that shows the effects happening: https://github.com/mrmundt/pyomo/actions/runs/12673191510/job/35318997883

For example, inspect the "Install Pyomo" and "Install HiGHS" steps. The PATH is incorrect (wrong order) in the first, but it is correct (right order) in the second. Then it switches back to the wrong order in the next step "Download and install extensions."

[vidyasagarnimmagaddi]

Hi @mrmundt ,Thank you for bringing this issue to our attention. We are looking into this issue and will update you on this issue after investigating.

[Prabhatkumar59]

Hi @mrmundt - Could you please try checking with latest image version 20250105.1 .Please update your sync process and try using this new version. It might work. Let us know if you encounter any further issues. Thanks!

[mrmundt]

Hi @Prabhatkumar59 - seems like nope. I ran our job suite again, and we are still seeing the failures: https://github.com/Pyomo/pyomo/actions/runs/12749612711/job/35532536024

[Prabhatkumar59]

Hi @mrmundt - Thanks for your valuable inputs. We will check more into this and will update you.

[Prabhatkumar59]

Hi @mrmundt - We would like to inform you that the latest windows image version has been rolled out. Please update your sync process and try using the new version(20250113.1). Let us know if you encounter any further issues. Thanks!

[Prabhatkumar59]

Hi @mrmundt - Since we haven't heard back, we'll assume your issue is resolved incase please update your sync process and try using the new version and will close this issue for now. Feel free to reach out to us for any other queries. Thanks.

[mrmundt]

Hi there, apologies for the delay. I missed your earlier tag. Unfortunately, no, it did not fix the issue, but I suspect that there is something going on with the interaction with miniconda rather than a problem with the base image itself. Thank you.