Pod v4 MAIN #1229

Badatos · 2024-11-21T11:05:51Z

This PR is here to compare changes mades from Pod V3 DEV to Pod V4.

Summary of changes:

Replace Django 3.2 by 4.2
Drop Python 3.8 from compatible py versions, and add 3.12
Drop Compatibility with ES 6 & 7. Default to ES 8
Remove LTI compatibility (until lti_provider has been upgraded > 1.0, with upgraded oauth2)
Remove Shibboleth authentification (Brown-University-Library/django-shibboleth-remoteuser hasn't been updated since Jan 20, 2021 and isn't compatible with Django 4.2)

Detailed changes:

Improve security in pod playlists, as suggested by CodeQL
Remove USE_L10N deprecated param
Replace some re_path() by path()
Replace remaining ugettext by gettext
Replace admin.site.register() call by @admin.register decorator
Replace HTTP_X_REQUESTED_WITH by headers={"x-requested-with"
Replace request.META by headers
Replace assertQuerysetEqual by assertQuerySetEqual
Replace force_text by force_str
Remove default_app_config in init.py
Replace js_asset by django.templatetags.static, as js_asset is deprecated
Only run cut tests when USE_CUT==True
Only run dressing tests when USE_DRESSING==True
Replace deprecated django-tagging by django-tagulous
Use Tagulous to display a tag cloud with tags assigned to videos
Store tag list in CACHE
Replace "order" by "date_added" in channel video sorting field
Upgrade all py packages
add a reindex_videos script to recreate ES video index

* Upgrade securtity dependances * Remove deprecated django-tagging * Add new django-tagulous * Elasticsearch default version is Now 8 (remove compatibility with ES 6) * Replace deprecated ugettext_lazy by gettext_lazy * Add some temporary deprecations Hacks for cas-client & Chunked-upload * Replace deprecated url() by re_path() * Disable all Pods apps by default * Correct error handling for ElasticSearch

# Conflicts: # pod/settings.py

* Replace some re_path() by path() * Replace remaining ugettext by gettext * Replace admin.site.register() call by @admin.register decorator * Replace HTTP_X_REQUESTED_WITH by `headers={"x-requested-with"` * Replace request.META by headers * Replace `assertQuerysetEqual` by `assertQuerySetEqual` * Replace force_text by force_str * Remove default_app_config in __init__.py

…select2 CSS calls) * Correct Enrichment form validation process

* Replace Django 3.2 by 4.2 in all links

# Conflicts: # pod/settings.py

…ated

pod/playlist/views.py

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

pod/playlist/views.py

… pod_V4

…ionalError: database table is locked"

…lity

…en/dev/releases/4.2.17 )

Correct CVE-2024-56374: Potential denial-of-service vulnerability in IPv6 validation

# Conflicts: # .env.dev-exemple # dockerfile-dev-with-volumes/README.adoc # pod/main/test_settings.py # pod/video/models.py # pod/video/utils.py # requirements.txt

LoicBonavent

Bravo ! Vraiment, super boulot !
Pour ma part, j'ai bien compris les changements suivants :

remplacement ugettext_lazy par gettext_lazy,
remplacement urls par re_path
remplacement force_text par force_str
remplacement tagging par tagulous.models
remplacement pour les asserts (ex: assertNotEquals par assertNotEqual)
remplacement pour les requêtes AJAX, is_ajax(request)
remplacement CKEditor
les valeurs par défaut à False maintenant
la gestion des header (referer, x-requested...)
la gestion des @admin.register
la modification vis-à-vis de la qualité du code (paramètres, espaces, refactor...)
le script de réindexation ES des videos
les modifications vis-à-vis d'Elasticsearch
le fait de supprimer le deprecated USE_L10N

Par contre, je n'ai pas compris les modifications vis-à-vis de :

Typing,
la suppresion des short_description
les modifications vis-à-vis de CustomFileModel
Par quoi ces dernières modifications se traduisent ?

Badatos · 2025-01-31T16:26:13Z

Bravo ! Vraiment, super boulot ! Pour ma part, j'ai bien compris les changements suivants :

Merci ;)

remplacement ugettext_lazy par gettext_lazy,

remplacement urls par re_path

remplacement force_text par force_str

remplacement tagging par tagulous.models

remplacement pour les asserts (ex: assertNotEquals par assertNotEqual)

remplacement pour les requêtes AJAX, is_ajax(request)

remplacement CKEditor

les valeurs par défaut à False maintenant

la gestion des header (referer, x-requested...)

la gestion des @admin.register

la modification vis-à-vis de la qualité du code (paramètres, espaces, refactor...)

le script de réindexation ES des videos

les modifications vis-à-vis d'Elasticsearch

le fait de supprimer le deprecated USE_L10N

Par contre, je n'ai pas compris les modifications vis-à-vis de :

Typing,

As-tu un exemple ? J'ai fait cette PR l'an dernier et j'ai une toute petite mémoire ^^.
Il me semble si on parle de la même chose que c'est une histoire de version de Python.
Par exemple, en Python 3.9, on pouvait définir le retour d'une fonction ainsi : Optional[str], mais en 3.10 cette notation est deprecated et est remplacée par str | None (que je trouve plus claire d'ailleurs)

la suppresion des short_description

les modifications vis-à-vis de CustomFileModel
Par quoi ces dernières modifications se traduisent ?

As-tu des exemples pour me rafraichir la mémoire ? (n'hésites-pas à commenter directement dans les lignes de code sur Github)

Badatos · 2025-01-31T16:33:34Z

pod/authentication/admin.py

        return format_html('<a href="mailto:{}">{}</a>', email, email)

-    clickable_email.allow_tags = True
-    clickable_email.short_description = _("Email")


J'ai trouvé ! En fait les short_description sont remplacés par un tag @admin.display(description=_("xxx")) (cf authentication/admin.py)

Merci pour l'explication

Badatos · 2025-01-31T16:40:50Z

Ah oui, je crois me rappeler que j'ai simplement mis "en pause" les CustomFileModel, car ils ne fonctionnent pas encore avec Pod V4.
Ils demanderont un travail de développement supplémentaire pour être opérationnels, mais j'ai trouvé que cette PR était assez grosse sans rajouter encore ce dev dedans ^^

LoicBonavent · 2025-02-03T08:54:09Z

pod/live/pilotingInterface.py

Ok, pour Le coup du typing, il a juste été supprimé ici car inutile dans ce fichier; mais il est toujours appelé ailleurs.
EN fait, j'avais peur qu'il y ait eu des modifications en lien avec Django 4, ce qui n'est pas le cas; donc nickel. Merci

Badatos and others added 13 commits October 24, 2024 12:10

Merge branch 'develop' into pod_V4

e50187e

# Conflicts: # pod/settings.py

Remove USE_L10N from test_settings

af15471

Merge branch 'develop' into pod_V4

7c01d4b

* Correct video_edit slect2 display with tagulous (remove duplicated …

a721a03

…select2 CSS calls) * Correct Enrichment form validation process

* Remove Python 3.8 from compatible py versions, and add 3.12

4deb722

* Replace Django 3.2 by 4.2 in all links

Merge branch 'develop' into pod_V4

b2975a6

# Conflicts: # pod/settings.py

Upgrade dependencies

645b930

several corrections for 4.0. Prepare to say goodbye to CKEditor

7c5d362

correct video/models.py

917a006

Remove Compatibility with ES 6

9a4ab7f

Replace js_asset by django.templatetags.static, as js_asset is deprec…

097edab

…ated

Badatos self-assigned this Nov 21, 2024

github-advanced-security bot found potential problems Nov 21, 2024

View reviewed changes

pod/playlist/views.py Fixed Show fixed Hide fixed

pod/playlist/views.py Fixed Show fixed Hide fixed

pod/playlist/views.py Fixed Show fixed Hide fixed

pod/playlist/views.py Fixed Show fixed Hide fixed

pod/playlist/views.py Fixed Show fixed Hide fixed

Fix code scanning alert no. 81: URL redirection from remote source

151af7c

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

github-advanced-security bot found potential problems Nov 21, 2024

View reviewed changes

pod/playlist/views.py Fixed Show fixed Hide fixed

Badatos added 13 commits November 21, 2024 12:26

Improve security in pod playlists, as suggested by CodeQL

ccfebc3

Disable some video_tags functions

5941015

Flake8 compliance

bf7dc21

Disable Debug in test_settings

cb32056

only add debug toolbar when not in testing mode

c9de8e1

Add title_en to sample videos.json

d28b3dc

Reset ES-INDEX for tests as "Pod"

6bd0164

Use Elasticsearch 7 in github workflows

3c17c09

set Video tags as "not required" + correct bug on video tags indexing

b54952b

Adapt Video views and video-info to Tagulous

fd9fbaa

adapt video get_json_to_index() function to Tagulous

bf5cabd

Adapt Video Recorder to Tagulous

92ac5b2

Only do dressing tests when `USE_DRESSING==True

cecec76

github-actions added 2 commits December 3, 2024 16:03

Fixup. Format code with Black

a18ea13

Auto-update configuration files

76e6191

Badatos changed the base branch from develop to main December 4, 2024 09:05

Merge branch 'main' into pod_V4

7b58f99

Badatos changed the title ~~[NOT TO MERGE] Pod v4 new~~ Pod v4 MAIN Dec 4, 2024

Badatos marked this pull request as ready for review December 4, 2024 09:14

Badatos and others added 17 commits December 4, 2024 10:35

Bump ELASTICSEARCH_TAG to 8.16.1

c2d1011

Change elastic-github-actions to master to avoid ES 8 issue

b5fc92a

Set ES_VERSION=8 in settings_local_docker

92fa056

Fixup. Format code with Black

4853cdd

test ES with CURL after create_pod_index

12d0c3f

Merge branch 'pod_V4' of https://github.com/EsupPortail/Esup-Pod into…

e62575b

… pod_V4

test with es1:9200 url

68d7f5f

Return to elasticsearch.localhost url

3216702

add --insecure to curl command

6eeedd4

Disable elastic-github-actions security

8cfc848

replace https by http in curl

50c5437

increase sqlite timeout in test settings to have less "sqlite3.Operat…

7343fa0

…ionalError: database table is locked"

Replace assertNotEqualsby assertNotEqualfor Python 3.12 compatibi…

5115cb4

…lity

Replace remaining assertEquals by assertEqual

2f928f9

Bump Django to 4.2.17 (fix 2 CVE, see https://docs.djangoproject.com/…

ec9addf

…en/dev/releases/4.2.17 )

Bump Django to 4.2.18

e91fea4

Correct CVE-2024-56374: Potential denial-of-service vulnerability in IPv6 validation

Merge branch 'main' into pod_V4

1b02e4f

# Conflicts: # .env.dev-exemple # dockerfile-dev-with-volumes/README.adoc # pod/main/test_settings.py # pod/video/models.py # pod/video/utils.py # requirements.txt

LoicBonavent approved these changes Jan 31, 2025

View reviewed changes

Badatos commented Jan 31, 2025

View reviewed changes

LoicBonavent reviewed Feb 3, 2025

View reviewed changes

Badatos merged commit 8d1c341 into main Feb 7, 2025
4 checks passed

Badatos deleted the pod_V4 branch June 4, 2025 12:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Pod v4 MAIN #1229

Pod v4 MAIN #1229

Uh oh!

Badatos commented Nov 21, 2024 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LoicBonavent left a comment

Uh oh!

Badatos commented Jan 31, 2025 •

edited

Loading

Uh oh!

Badatos Jan 31, 2025

Uh oh!

LoicBonavent Feb 3, 2025

Uh oh!

Badatos commented Jan 31, 2025

Uh oh!

LoicBonavent Feb 3, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Pod v4 MAIN #1229

Pod v4 MAIN #1229

Uh oh!

Conversation

Badatos commented Nov 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes:

Detailed changes:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LoicBonavent left a comment

Choose a reason for hiding this comment

Uh oh!

Badatos commented Jan 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Badatos Jan 31, 2025

Choose a reason for hiding this comment

Uh oh!

LoicBonavent Feb 3, 2025

Choose a reason for hiding this comment

Uh oh!

Badatos commented Jan 31, 2025

Uh oh!

LoicBonavent Feb 3, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Badatos commented Nov 21, 2024 •

edited

Loading

Badatos commented Jan 31, 2025 •

edited

Loading