Skip to content

(maint) Correct some typos and fix some tests #3589

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jun 9, 2025

Conversation

corbob
Copy link
Member

@corbob corbob commented Dec 6, 2024

Description Of Changes

  1. Remove the mention of choco list -lo from the NuSpec file.
  2. Update an upgrade all test to ensure that it does not report failure if a newer Chocolatey version is available on internal repositories.
  3. Use PowerShell instead of powershell.
  4. Fix Get-EnvironmentVariableNames tests to run.
  5. Update Vagrant box used for building and running tests to use the chocolatey/test-environment box.

Motivation and Context

  1. Local Only on the list command was removed in 2.0, and is an error now.
  2. When tests are running as we come up towards a release, we end up with reports of failures of the upgrade all command because for instance it will be testing version 2.4.1-beta-20241204, but because of the way the builds work, CI will have build and published to our repository version 2.5.0-alpha-20241204 which is deemed an upgrade for the pre-release version being tested.
  3. The correct casing is PowerShell.
  4. Due to a misconfiguration, these tests were not running, and would start failing in Pester 6.x.
  5. The previous box hasn't been updated for several years. We also encourage testing on the chocolatey/test-environment box, so running out tests on it makes sense.

Testing

  • Run through Test Kitchen
  • Run tests through vagrant up in the tests directory. (There are some known failures here, the important thing is that the majority of them run successfully)

Operating Systems Testing

  • Windows Server 2016
  • Windows Server 2019
  • Windows Server 2022

Change Types Made

  • Bug fix (non-breaking change).
  • Feature / Enhancement (non-breaking change).
  • Breaking change (fix or feature that could cause existing functionality to change).
  • Documentation changes.
  • PowerShell code changes.
  • Pester test changes.

Change Checklist

  • Requires a change to the documentation.
  • Documentation has been updated.
  • Tests to cover my changes, have been added.
  • All new and existing tests passed?
  • PowerShell code changes: PowerShell v3 compatibility checked?

Related Issue

N/A - These are things that I found doing the recent 2.4.1 release.

@corbob corbob added the NO RELEASE NOTES Should not be included in the release notes - not enhancing or fixing end product. label Dec 6, 2024
@corbob corbob self-assigned this Dec 6, 2024
@corbob
Copy link
Member Author

corbob commented Dec 6, 2024

This PR is draft until I can run the tests through Test Kitchen. I also need to update the testing steps to validate that the test failure is gone, but will need to think on the best way to do that.

@corbob
Copy link
Member Author

corbob commented Jan 8, 2025

A few things I need to update about this before it's out of WIP. Adding here so I don't forget:

  • There's a few places we say powershell instead of PowerShell
  • There are some tests that fail in the local vagrant environment, but not Test Kitchen. Would like to get those updated to pass everywhere.
  • There is one test file that fails with Pester 6.x alpha build due to a ForEach variable being null. This will need to be addressed.
    It 'Gets the complete list of environment variables in the <Scope> scope"' -ForEach $variables {
    $variables is not defined here.
  • Vagrant box used for building and running tests hasn't been updated in a while.

@corbob corbob requested a review from gep13 June 4, 2025 01:00
@corbob corbob marked this pull request as ready for review June 4, 2025 01:01
@corbob
Copy link
Member Author

corbob commented Jun 4, 2025

@gep13 I was looking at some of these tests, and although I wanted to get a few more things in here, I've paired this down to things that I think will help you with the next release (in particular the upgrade all test fix).

(I've added this PR to the document for our weekly meeting if we want to discuss it)

@corbob corbob changed the title (maint) A few minor updates (maint) Correct some typos and fix some tests Jun 4, 2025
corbob added 6 commits June 9, 2025 09:38
Sometimes tests in Test Kitchen are run against versions of Chocolatey
that are available on the setup repository, and so they are found as
part of upgrade testing. The tests don't need these newer Chocolatey
versions, nor do we expect to find them. This excludes all of the
packages that are known to be on testing environments but not part of
the test in question.

This is to reduce the number of false positive tests we have, especially
as we come up to a release.
The $variables passed to the ForEach for these tests was found to be
null, and so these tests weren't actually running. Upon looking at the
tests, determined the ForEach variable was unintentional and was
removed.

The tests get the variable names in one order, and the .Net method to
get the names presents them in a different order. Changed this to sort
the list for both of them so that the Pester tests were consistent.
We use the Chocolatey Test Environment for testing packages on the
Chocolatey Community Repository, so we should probably use it here for
testing Chocolatey itself.

This also adds a copy of the `testResults.xml` generated by Pester from
the test system to the host system so it can be analyzed.
Ignore the file if it's testresult.xml or testresults.xml
Copy link
Member

@gep13 gep13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@gep13
Copy link
Member

gep13 commented Jun 9, 2025

@corbob thank you for getting these updates done! I will get this merged in once the CI builds are finished.

@gep13 gep13 merged commit bb460ed into chocolatey:develop Jun 9, 2025
5 checks passed
@gep13
Copy link
Member

gep13 commented Jun 9, 2025

@corbob thank you for getting these updates done!

@corbob corbob deleted the minor-tweaks branch June 9, 2025 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NO RELEASE NOTES Should not be included in the release notes - not enhancing or fixing end product.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants