Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump minimum python version to 3.8 and Upgrade pantsbuild to 2.17 #6118

Merged
merged 22 commits into from
Jan 29, 2024

Conversation

cognifloyd
Copy link
Member

  • Updates pantsbuild to 2.17.
  • Switches from ./pants script to pants binary downloaded with ./get-pants.sh.
  • Updates minimum python version to 3.8 (only for the lockfiles. Old build infra not adjusted).
  • Resolves a pylint-identified issue with jsonschema 3.2 by updating the code to work with jsonschema 3.

update lockfiles except for st2 which is having an issue that I hope will
be resolved after upgrading pants to even newer versions.

Lockfile diff: lockfiles/pytest.lock [pytest]

==                    Upgraded dependencies                     ==

  icdiff                         2.0.6        -->   2.0.7
  pygal                          3.0.0        -->   3.0.1

Lockfile diff: lockfiles/twine.lock [twine]

==                    Upgraded dependencies                     ==

  certifi                        2023.7.22    -->   2023.11.17
  idna                           3.4          -->   3.6
  urllib3                        1.26.16      -->   1.26.18

Lockfile diff: lockfiles/bandit.lock [bandit]

==                    Upgraded dependencies                     ==

  pbr                            5.11.1       -->   6.0.0

Lockfile diff: lockfiles/pants-plugins.lock [pants-plugins]

==                    Upgraded dependencies                     ==

  attrs                          23.1.0       -->   23.2.0
  certifi                        2023.7.22    -->   2023.11.17
  charset-normalizer             3.2.0        -->   3.3.2
  idna                           3.4          -->   3.6
  pantsbuild-pants               2.16.1rc0    -->   2.16.1
  pantsbuild-pants-testutil      2.16.1rc0    -->   2.16.1
  urllib3                        1.26.16      -->   1.26.18
The old ./pants script is deprecated upstream. The new pants binary is
much faster and handles bootstrapping the version of python used by
pants itself. For many projects, that means the interpreter_constraints
of pants are irrelevant for in-repo python code. But, we have in-repo
pants-plugins, so we still have to keep our tool lockfiles compatible
with pants code.

documentaiton: https://www.pantsbuild.org/2.18/docs/getting-started/installing-pants

local copy of get-pants.sh downloaded from: https://static.pantsbuild.org/setup/get-pants.sh
(Pants is Apache 2.0 licensed)
Update lockfiles except for st2

Lockfile diff: lockfiles/pants-plugins.lock [pants-plugins]

==                    Upgraded dependencies                     ==

  pantsbuild-pants               2.16.1       -->   2.17.1
  pantsbuild-pants-testutil      2.16.1       -->   2.17.1
  pex                            2.1.134      -->   2.1.137

==                      Added dependencies                      ==

  node-semver                    0.9.0

==                     Removed dependencies                     ==

  certifi                        2023.11.17
  charset-normalizer             3.3.2
  humbug                         0.2.7
  idna                           3.6
  requests                       2.31.0
  urllib3                        1.26.18
Lockfile diff: lockfiles/flake8.lock [flake8]

==                    Upgraded dependencies                     ==

  setuptools                     59.6.0       -->   69.0.3

==                     Removed dependencies                     ==

  importlib-metadata             4.2.0
  typing-extensions              4.1.1
  zipp                           3.6.0

Lockfile diff: lockfiles/bandit.lock [bandit]

==                    Upgraded dependencies                     ==

  gitdb                          4.0.9        -->   4.0.11
  setuptools                     59.6.0       -->   69.0.3
  smmap                          5.0.0        -->   5.0.1
  stevedore                      3.5.2        -->   5.1.0

==                     Removed dependencies                     ==

  importlib-metadata             4.8.3
  typing-extensions              4.1.1
  zipp                           3.6.0

Lockfile diff: lockfiles/twine.lock [twine]

==                    Upgraded dependencies                     ==

  cffi                           1.15.1       -->   1.16.0
  charset-normalizer             2.0.12       -->   3.3.2
  colorama                       0.4.5        -->   0.4.6
  cryptography                   40.0.2       -->   42.0.1
  docutils                       0.18.1       -->   0.20.1
  importlib-metadata             4.8.3        -->   7.0.1
  importlib-resources            5.4.0        -->   6.1.1
  jeepney                        0.7.1        -->   0.8.0
  keyring                        23.4.1       -->   24.3.0
  pygments                       2.14.0       -->   2.17.2
  readme-renderer                34.0         -->   42.0
  requests                       2.27.1       -->   2.31.0
  rfc3986                        1.5.0        -->   2.0.0
  tqdm                           4.64.1       -->   4.66.1
  urllib3                        1.26.18      -->   2.1.0
  zipp                           3.6.0        -->   3.17.0

==                      Added dependencies                      ==

  jaraco-classes                 3.3.0
  more-itertools                 10.2.0
  nh3                            0.2.15

==                     Removed dependencies                     ==

  bleach                         4.1.0
  packaging                      21.3
  pyparsing                      3.0.7
  six                            1.16.0
  typing-extensions              4.1.1
  webencodings                   0.5.1

Lockfile diff: lockfiles/black.lock [black]

==                    Upgraded dependencies                     ==

  click                          8.0.4        -->   8.1.7
  pathspec                       0.9.0        -->   0.12.1
  platformdirs                   2.4.0        -->   4.1.0
  tomli                          1.2.3        -->   2.0.1
  typing-extensions              4.1.1        -->   4.9.0

==                     Removed dependencies                     ==

  dataclasses                    0.6
  importlib-metadata             4.8.3
  typed-ast                      1.5.5
  zipp                           3.6.0

Lockfile diff: lockfiles/pylint.lock [pylint]

==                    Upgraded dependencies                     ==

  isort                          5.8.0        -->   5.13.2
  lazy-object-proxy              1.7.1        -->   1.10.0
  setuptools                     59.6.0       -->   69.0.3

==                     Removed dependencies                     ==

  typed-ast                      1.4.3

Lockfile diff: lockfiles/pytest.lock [pytest]

==                    Upgraded dependencies                     ==

  attrs                          22.2.0       -->   23.2.0
  coverage                       6.2          -->   7.4.1
  execnet                        1.9.0        -->   2.0.2
  importlib-metadata             4.8.3        -->   7.0.1
  iniconfig                      1.1.1        -->   2.0.0
  packaging                      21.3         -->   23.2
  pluggy                         1.0.0        -->   1.4.0
  pygal                          3.0.1        -->   3.0.4
  pygments                       2.14.0       -->   2.17.2
  pytest-forked                  1.4.0        -->   1.6.0
  pytest-icdiff                  0.6          -->   0.9
  tomli                          1.2.3        -->   2.0.1
  zipp                           3.6.0        -->   3.17.0

==                     Removed dependencies                     ==

  pyparsing                      3.0.7
  typing-extensions              4.1.1
This allows us to get rid of python 3.7 in all of the tool lockfiles.

Lockfile diff: lockfiles/pants-plugins.lock [pants-plugins]

==                    Upgraded dependencies                     ==

  pluggy                         1.2.0        -->   1.4.0
  ujson                          5.7.0        -->   5.9.0

==                     Removed dependencies                     ==

  importlib-metadata             6.7.0
  zipp                           3.15.0
@cognifloyd cognifloyd added this to the pants milestone Jan 28, 2024
@cognifloyd cognifloyd requested review from nzlosh, amanda11 and a team January 28, 2024 18:29
@cognifloyd cognifloyd self-assigned this Jan 28, 2024
@pull-request-size pull-request-size bot added the size/XXL PR that changes 1000+ lines. You should absolutely split your PR into several. label Jan 28, 2024
amanda11

This comment was marked as resolved.

@cognifloyd cognifloyd changed the title Upgrade pantsbuild to 2.17; bump minimum python version to 3.8 Bump minimum python version to 3.8 and Upgrade pantsbuild to 2.17 Jan 29, 2024
v4 adds several new schemas changing the default behavior of
some of the validators. So, care needs to be taken during upgrade.
@nzlosh nzlosh mentioned this pull request Jan 29, 2024
The action_params schema defines the "type" property as:

    anyOf: [{enum: [string, object, ...]}]

jsonschema v2.6.0 reported the anyOf error "matched none of the schemas"
jsonschema v3.2.0 is a bit more helpful by reporting the underlying enum
error.

I also looked at removing the anyOf (since there is only one entry), but
the anyOf makes it easier to compare our action_params schema with the
draft4 schema to make the differences very clear.
@cognifloyd
Copy link
Member Author

I'm merging this. Hopefuly subsequent PRs won't be so big...

@cognifloyd cognifloyd merged commit f744a26 into master Jan 29, 2024
29 checks passed
@cognifloyd cognifloyd deleted the pants-upgrade branch January 29, 2024 22:28
@nzlosh
Copy link
Contributor

nzlosh commented Jan 30, 2024

Nice job! Thanks @cognifloyd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
external dependency infrastructure: ci/cd pantsbuild size/XXL PR that changes 1000+ lines. You should absolutely split your PR into several.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants