Skip to content

feat: Python 3.12 #8811

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 28 commits into
base: main
Choose a base branch
from
Draft

feat: Python 3.12 #8811

wants to merge 28 commits into from

Conversation

jennifer-richards
Copy link
Member

@jennifer-richards jennifer-richards commented Apr 17, 2025

Updates Python to 3.12. Fixes #7929.

Draft while WIP.

Outstanding issues:

  • Coverage is outrageously slow, currently disabled (resolved by refactor: update to coverage 7.9.2 + cleanup #9256, which upgrades to coverage 7.9.2)
    • Context: the old version of coverage that we use (4.5.4, ca 2019) is falling back from its fast CTracer module to its very slow PyTracer when used on Python 3.12. It's not clear exactly why, but it's almost 3x slower.
    • The situation may be better if we can update to a current version of coverage, but see Poor performance in Python 3.12 nedbat/coveragepy#1665 for more info.
  • Should at least skim major requirements to confirm actual Python 3.12 compatibility (tests do pass, aside from coverage issues) (reviewed - not all requirements are officially supported, so we'll need to do some testing - see chore: update deps for py3.12 #9270)
  • Need to decide if we want to move from :latest to a versioned tag (e.g., :bookworm-py312) to avoid pulling the rug out from under users' dev environments Tagged as :py312

jennifer-richards and others added 16 commits April 16, 2025 17:46
The test has been working, but in a broken way, for as long as it has existed. The smtpd-based test_smtpserver was masking an exception that did not interfere with the test's effectiveness.
Removed a few suppressions that were OBE based on running the tests and checking versions of the dependencies that were causing them. Reordered kwargs to make it more readable (to me anyway).
See the comment in settings.py for details.

tl;dr coverage is unusably slow under python 3.12 as we're using it
# Conflicts:
#	dev/build/Dockerfile
#	dev/build/TARGET_BASE
@jennifer-richards jennifer-richards changed the base branch from main to feat/blobstage April 22, 2025 15:00
@jennifer-richards jennifer-richards changed the base branch from feat/blobstage to main April 22, 2025 15:00
jennifer-richards and others added 10 commits April 22, 2025 15:11
# Conflicts:
#	dev/build/Dockerfile
#	dev/build/TARGET_BASE
#	ietf/doc/tests_review.py
#	ietf/review/mailarch.py
# Conflicts:
#	dev/build/Dockerfile
#	dev/build/TARGET_BASE
* ci: tag py312 base app

* ci: datatrackerbase-app:latest -> py312
# Conflicts:
#	dev/celery/Dockerfile
#	docker/celery.Dockerfile
* refactor: drop unused code_coverage_collection var

* refactor: @skip_coverage -> pragma: no cover

* chore(deps): bump coverage to current ver

* refactor: split up set_coverage_checking()

* refactor: inline IetfLiveServerTestCase

(there's only one subclass)

* feat: disable_coverage context mgr

* chore: remove unused import

* refactor: set_coverage_checking -> disable_coverage

* refactor: elim more set_coverage_checking

* refactor: start using coverage 7.9.2

* feat: working coverage 7.9 implementation

 * Extract coverage tools to ietf.utils.coverage
 * Revert to starting checker in settings_test

Does not exactly match previous coverage reports. Need to investigate.

* refactor: CustomJsonReporter->CustomDictReporter

* chore: remove "migration" coverage entry

Has not been populated in quite some time

* test: test CoverageManager class

* chore: exclude CustomDictReporter from coverage

Setting up to test this will be complex and we'll notice other test failures/coverage weirdness if this does not behave.

* chore: exclude coverage.py from coverage

Way too meta
* chore(deps): argon2-cffi (supports py3.14)

* chore(deps): setuptools to latest (py3.9+)

* chore(deps): bump beautifulsoup4 (py3.7+)

* chore(deps): bump bibtexparser (py3)

* chore(deps): bump bleach (py3.13)

* chore(deps): bump bleach (py3.13)

* chore(deps): lift pin on boto3 + adjust settings

* chore(deps): bump celery (py3.13)

* chore(deps): bump django-admin-rangefilter (py3.12)

* chore(deps): bump django-analytical (py3.13)

* chore(deps): bump django-bootstrap5 (py3.13)

* chore(deps): bump django-celery-beat (py3.12)

Still holding back until their #894 is conclusively resolved. The 2.8.x release adds official py3.13 support.

* chore(deps): bump django-celery-results (py3.13)

* chore(deps): remove django-csp (not used)

* chore(deps): bump django-cors-headers (py3.13)

* chore(deps): bump django-debug-toolbar (py3.13)

* refactor: drop stale django-referrer-policy pkg

Supported via django's SecurityMiddleware since longtime

* chore(deps): bump django-simple-history (py3.13)

* chore(deps): bump django-storages (py3.12)

* chore(deps): bump django-tastypie+update patch

* chore(deps): bump django_vite+update config

* chore(deps): bump djangorestframework+remove cap

* chore(deps): remove djlint

* chore(deps): bump docutils (py3.14)

* chore(deps): bump drf-standardized-errors (py3.13)

* chore(deps): bump factory-boy (py3.13)

* chore(deps): bump github3.py (py3.11??)

* chore(deps): bump gunicorn (py3.12)

* chore(deps): bump html2text (py3.13)

* chore(deps): bump inflect

* chore(deps): bump jsonfield (py3.10-3.13)

* chore(deps): bump jsonschema (py3.13)

* chore(deps): bump logging_tree (py3.12)

* chore(deps): bump lxml (py3.13)

* chore(deps): bump markdown (py3.13)

* chore(deps): bump mock

* chore(deps): bump oic (py3.11)

* chore(deps): bump pillow (py3.13)

* chore(deps): bump psycopg2 (py3.13)

* chore(deps): bump pyang (py3.11)

* chore(deps): bump pydyf (py3.12)

* chore(deps): bump pyflakes (py3.9+)

* chore(deps): bump pyopenssl (py3.13)

* chore(deps): bump pyquery (py3.12)

* chore(deps): bump python-dateutil (py3.12)

* chore(deps): bump python-json-logger (py3.13)

* chore(deps): bump python-mimeparse (py3.13)

* chore(deps): bump pytz (py3.13)

Brings a meeting migration to adjust tz/country choices.

* chore(deps): bump requests (py3.13)

* chore(deps): bump requests-mock (py3.12)

* chore(deps): bump scout-apm (py3.12)

* chore(deps): bump selenium (py3.13)

* chore(deps): bump tblib (py3.13)

* chore(deps): bump tqdm (py3.12)

* chore(deps): bump unidecode (py3.11)

* chore(deps): adjust requirements.txt to install correctly

* chore(deps): bump urllib3, remove pin (py3.13)

Situation requiring the pin to < 2.0 appears to have resolved.

* chore(deps): bump weasyprint (py3.13)

* chore(deps): bump xml2rfc (py3.13)

* fix: lint
# Conflicts:
#	dev/build/Dockerfile
#	dev/build/TARGET_BASE
#	ietf/api/__init__.py
#	ietf/utils/tests.py
#	requirements.txt
chore: merge main into feat/py312
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Move datatracker off of python 3.9
1 participant