Skip to content

[RELEASE] [WIP] Pod 3.9.0 #1255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Badatos wants to merge 6 commits into
base: master
Choose a base branch
from
Open

[RELEASE] [WIP] Pod 3.9.0 #1255

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
ee288d7
Change Pod version to 3.9.0
Badatos Jan 24, 2025
6956d9d
Merge branch 'master' into develop
Badatos Jan 24, 2025
19ada5c
Store draft state before archiving video (#1260)
Badatos Feb 10, 2025
1fd6f52
Fixup. Format code with Prettier
invalid-email-address Feb 10, 2025
4aa9144
Fixup. Format code with Black
invalid-email-address Feb 10, 2025
dd08c8e
Rétro-portage des améliorations Docker #1328 (#1337)
azmeuk Jul 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .env.dev-exemple
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ DJANGO_SUPERUSER_EMAIL=<MAIL>
### You can use internal registry
ELASTICSEARCH_TAG=elasticsearch:8.13.0
NODE_TAG=node:23
PYTHON_TAG=python:3.9-bullseye
PYTHON_TAG=python:3.9-bookworm
REDIS_TAG=redis:alpine3.16
### DOCKER_ENV: You can specify light or full.
### In case of value changing, you have to rebuild and restart your container.
Expand Down
8 changes: 6 additions & 2 deletions docker-compose-dev-with-volumes.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,10 @@ services:
context: .
dockerfile: dockerfile-dev-with-volumes/pod/Dockerfile
depends_on:
- elasticsearch
- redis
elasticsearch:
condition: service_healthy
redis:
condition: service_healthy
env_file:
- ./.env.dev
ports:
Expand Down Expand Up @@ -42,6 +44,8 @@ services:
- ./.env.dev
ports:
- 6379:6379
healthcheck:
test: [ "CMD", "redis-cli", "--raw", "incr", "ping" ]

# redis-commander:
# container_name: redis-commander
Expand Down
17 changes: 12 additions & 5 deletions docker-compose-full-dev-with-volumes-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,10 @@ services:
context: .
dockerfile: dockerfile-dev-with-volumes/pod-back/Dockerfile
depends_on:
- elasticsearch
- redis
elasticsearch:
condition: service_healthy
redis:
condition: service_healthy
env_file:
- ./.env.dev
ports:
Expand All @@ -27,7 +29,8 @@ services:
context: .
dockerfile: dockerfile-dev-with-volumes/pod-encode/Dockerfile
depends_on:
- pod-back
pod-back:
condition: service_healthy
env_file:
- ./.env.dev
volumes: *pod-volumes
Expand All @@ -39,7 +42,8 @@ services:
context: .
dockerfile: dockerfile-dev-with-volumes/pod-transcript/Dockerfile
depends_on:
- pod-back
pod-back:
condition: service_healthy
env_file:
- ./.env.dev
volumes: *pod-volumes
Expand All @@ -51,7 +55,8 @@ services:
context: .
dockerfile: dockerfile-dev-with-volumes/pod-xapi/Dockerfile
depends_on:
- pod-back
pod-back:
condition: service_healthy
env_file:
- ./.env.dev
volumes: *pod-volumes
Expand All @@ -78,6 +83,8 @@ services:
- ./.env.dev
ports:
- 6379:6379
healthcheck:
test: [ "CMD", "redis-cli", "--raw", "incr", "ping" ]

pa11y-ci:
container_name: pa11y-ci
Expand Down
17 changes: 12 additions & 5 deletions docker-compose-full-dev-with-volumes.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,10 @@ services:
context: .
dockerfile: dockerfile-dev-with-volumes/pod-back/Dockerfile
depends_on:
- elasticsearch
- redis
elasticsearch:
condition: service_healthy
redis:
condition: service_healthy
env_file:
- ./.env.dev
ports:
Expand All @@ -27,7 +29,8 @@ services:
context: .
dockerfile: dockerfile-dev-with-volumes/pod-encode/Dockerfile
depends_on:
- pod-back
pod-back:
condition: service_healthy
env_file:
- ./.env.dev
volumes: *pod-volumes
Expand All @@ -39,7 +42,8 @@ services:
context: .
dockerfile: dockerfile-dev-with-volumes/pod-transcript/Dockerfile
depends_on:
- pod-back
pod-back:
condition: service_healthy
env_file:
- ./.env.dev
volumes: *pod-volumes
Expand All @@ -51,7 +55,8 @@ services:
context: .
dockerfile: dockerfile-dev-with-volumes/pod-xapi/Dockerfile
depends_on:
- pod-back
pod-back:
condition: service_healthy
env_file:
- ./.env.dev
volumes: *pod-volumes
Expand All @@ -78,6 +83,8 @@ services:
- ./.env.dev
ports:
- 6379:6379
healthcheck:
test: [ "CMD", "redis-cli", "--raw", "incr", "ping" ]

# redis-commander:
# container_name: redis-commander
Expand Down
2 changes: 1 addition & 1 deletion dockerfile-dev-with-volumes/README.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ DJANGO_SUPERUSER_PASSWORD=
DJANGO_SUPERUSER_EMAIL=
ELASTICSEARCH_TAG=elasticsearch:8.13.0
NODE_TAG=node:23
PYTHON_TAG=python:3.9-bullseye
PYTHON_TAG=python:3.9-bookworm
REDIS_TAG=redis:alpine3.16
DOCKER_ENV=light
----
Expand Down
5 changes: 3 additions & 2 deletions dockerfile-dev-with-volumes/elasticsearch/dockerfile-elasticsearch-dev
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@
# Conteneur elasticsearch
ARG ELASTICSEARCH_VERSION
FROM $ELASTICSEARCH_VERSION
# TODO
#FROM harbor.urba.univ-lille.fr/store/elasticsearch:7.17.7

HEALTHCHECK --timeout=3s --start-period=60s --start-interval=1s CMD \
curl --fail --silent http://elasticsearch.localhost:9200/_cluster/health || exit 1

RUN /usr/share/elasticsearch/bin/elasticsearch-plugin install analysis-icu

18 changes: 11 additions & 7 deletions dockerfile-dev-with-volumes/pod-back/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,25 +20,29 @@ FROM $PYTHON_VERSION
#FROM harbor.urba.univ-lille.fr/store/python:3.7-buster

RUN apt-get update \
&& apt-get install -y --no-install-recommends netcat gettext \
&& apt-get install --yes --no-install-recommends \
gettext \
curl \
&& apt-get clean\
&& rm -rf /var/lib/apt/lists/*

WORKDIR /usr/src/app

COPY ./requirements.txt .
COPY ./requirements-encode.txt .
COPY ./requirements-conteneur.txt .
COPY ./requirements-dev.txt .
COPY ./requirements.txt ./requirements-encode.txt ./requirements-conteneur.txt ./requirements-dev.txt .
RUN mkdir /tmp/node_modules/

COPY --from=source-build-js /tmp/pod/node_modules/ /tmp/node_modules/
# TODO remove ES version - move it into env var
RUN pip3 install --no-cache-dir -r requirements-conteneur.txt \
RUN pip3 install --no-cache-dir \
--requirement requirements-dev.txt \
--requirement requirements-conteneur.txt \
&& pip3 install elasticsearch==7.17.7

# ENTRYPOINT:
COPY ./dockerfile-dev-with-volumes/pod-back/my-entrypoint-back.sh /tmp/my-entrypoint-back.sh
RUN chmod 755 /tmp/my-entrypoint-back.sh

ENTRYPOINT ["bash", "/tmp/my-entrypoint-back.sh"]
HEALTHCHECK --timeout=3s --start-period=60s --start-interval=1s CMD \
curl --fail http://pod.localhost:8000 || exit 1

ENTRYPOINT ["bash", "/tmp/my-entrypoint-back.sh"]
3 changes: 1 addition & 2 deletions dockerfile-dev-with-volumes/pod-back/my-entrypoint-back.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,13 @@ echo "Launching commands into pod-dev"
mkdir -p pod/node_modules
mkdir -p pod/db_migrations && touch pod/db_migrations/__init__.py
ln -fs /tmp/node_modules/* pod/node_modules
until nc -z elasticsearch.localhost 9200; do echo waiting for elasticsearch; sleep 10; done;
# Mise en route
# Base de données SQLite intégrée
BDD_FILE=/usr/src/app/pod/db.sqlite3
if test ! -f "$BDD_FILE"; then
echo "$BDD_FILE does not exist."
python3 manage.py create_pod_index
curl -XGET "elasticsearch.localhost:9200/pod/_search"
curl -XGET "elasticsearch:9200/pod/_search"
# Deployez les fichiers statiques
python3 manage.py collectstatic --no-input --clear
# Lancez le script présent à la racine afin de créer les fichiers de migration, puis de les lancer pour créer la base de données SQLite intégrée.
Expand Down
7 changes: 4 additions & 3 deletions dockerfile-dev-with-volumes/pod-encode/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,16 +18,17 @@ COPY ./pod/ .

RUN apt-get update \
&& apt-get install -y --no-install-recommends \
netcat \
ffmpeg \
&& apt-get clean\
&& rm -rf /var/lib/apt/lists/*

WORKDIR /usr/src/app

COPY ./requirements-encode.txt .
COPY ./requirements-encode.txt ./requirements-conteneur.txt .

RUN pip3 install --no-cache-dir -r requirements-encode.txt
RUN pip3 install --no-cache-dir \
--requirement requirements-encode.txt \
--requirement requirements-conteneur.txt

# ENTRYPOINT:
COPY ./dockerfile-dev-with-volumes/pod-encode/my-entrypoint-encode.sh /tmp/my-entrypoint-encode.sh
Expand Down
2 changes: 1 addition & 1 deletion dockerfile-dev-with-volumes/pod-encode/my-entrypoint-encode.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/bin/sh
echo "Launching commands into pod-dev"
until nc -z pod.localhost 8000; do echo waiting for pod-back; sleep 10; done;
# Serveur d'encodage
watchmedo auto-restart --directory=/usr/src/app --pattern=*.py --recursive -- \
celery -A pod.video_encode_transcript.encoding_tasks worker -l INFO -Q encoding --concurrency 1 -n encode
sleep infinity
10 changes: 5 additions & 5 deletions dockerfile-dev-with-volumes/pod-transcript/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,19 +18,19 @@ COPY ./pod/ .

RUN apt-get update \
&& apt-get install -y --no-install-recommends \
netcat \
sox \
libsox-fmt-mp3 \
&& apt-get clean\
&& rm -rf /var/lib/apt/lists/*

WORKDIR /usr/src/app

COPY ./requirements-encode.txt .
COPY ./requirements-transcripts.txt .
COPY ./requirements-encode.txt ./requirements-transcripts.txt ./requirements-conteneur.txt .

RUN pip3 install --no-cache-dir -r requirements-transcripts.txt \
&& pip3 install --no-cache-dir -r requirements-encode.txt
RUN pip3 install --no-cache-dir \
--requirement requirements-encode.txt \
--requirement requirements-transcripts.txt \
--requirement requirements-conteneur.txt

# ENTRYPOINT:
COPY ./dockerfile-dev-with-volumes/pod-transcript/my-entrypoint-transcript.sh /tmp/my-entrypoint-transcript.sh
Expand Down
2 changes: 1 addition & 1 deletion dockerfile-dev-with-volumes/pod-transcript/my-entrypoint-transcript.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/bin/sh
echo "Launching commands into pod-dev"
until nc -z pod.localhost 8000; do echo waiting for pod-back; sleep 10; done;
# Serveur d'encodage
watchmedo auto-restart --directory=/usr/src/app --pattern=*.py --recursive -- \
celery -A pod.video_encode_transcript.transcripting_tasks worker -l INFO -Q transcripting --concurrency 1 -n transcript
sleep infinity
11 changes: 4 additions & 7 deletions dockerfile-dev-with-volumes/pod-xapi/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,13 @@ COPY ./pod/ .
# TODO
#FROM harbor.urba.univ-lille.fr/store/python:3.7-buster

RUN apt-get update \
&& apt-get install -y --no-install-recommends netcat\
&& apt-get clean\
&& rm -rf /var/lib/apt/lists/*

WORKDIR /usr/src/app

COPY ./requirements-encode.txt .
COPY ./requirements-encode.txt ./requirements-conteneur.txt .

RUN pip3 install --no-cache-dir -r requirements-encode.txt
RUN pip3 install --no-cache-dir \
--requirement requirements-encode.txt \
--requirement requirements-conteneur.txt

# ENTRYPOINT:
COPY ./dockerfile-dev-with-volumes/pod-xapi/my-entrypoint-xapi.sh /tmp/my-entrypoint-xapi.sh
Expand Down
2 changes: 1 addition & 1 deletion dockerfile-dev-with-volumes/pod-xapi/my-entrypoint-xapi.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/bin/sh
echo "Launching commands into pod-dev"
until nc -z pod.localhost 8000; do echo waiting for pod-back; sleep 10; done;
# Serveur xAPI
watchmedo auto-restart --directory=/usr/src/app --pattern=*.py --recursive -- \
celery -A pod.xapi.xapi_tasks worker -l INFO -Q xapi --concurrency 1 -n xapi
sleep infinity
14 changes: 8 additions & 6 deletions dockerfile-dev-with-volumes/pod/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,27 +21,29 @@ FROM $PYTHON_VERSION

RUN apt-get update \
&& apt-get install -y --no-install-recommends \
netcat \
ffmpeg \
gettext \
curl \
&& apt-get clean\
&& rm -rf /var/lib/apt/lists/*

WORKDIR /usr/src/app

COPY ./requirements.txt .
COPY ./requirements-conteneur.txt .
COPY ./requirements-encode.txt .
COPY ./requirements-dev.txt .
COPY ./requirements.txt ./requirements-conteneur.txt ./requirements-encode.txt ./requirements-dev.txt .
RUN mkdir /tmp/node_modules/

COPY --from=source-build-js /tmp/pod/node_modules/ /tmp/node_modules/

RUN pip3 install --no-cache-dir -r requirements-conteneur.txt \
RUN pip3 install --no-cache-dir \
--requirement requirements-dev.txt \
--requirement requirements-conteneur.txt \
&& pip3 install elasticsearch==8.9.0

# ENTRYPOINT:
COPY ./dockerfile-dev-with-volumes/pod/my-entrypoint.sh /tmp/my-entrypoint.sh
RUN chmod 755 /tmp/my-entrypoint.sh

HEALTHCHECK --timeout=3s --start-period=60s --start-interval=1s CMD \
curl --fail http://pod.localhost:8000 || exit 1

ENTRYPOINT ["bash", "/tmp/my-entrypoint.sh"]
1 change: 0 additions & 1 deletion dockerfile-dev-with-volumes/pod/my-entrypoint.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ echo "Launching commands into pod-dev"
mkdir -p pod/node_modules
mkdir -p pod/db_migrations && touch pod/db_migrations/__init__.py
ln -fs /tmp/node_modules/* pod/node_modules
until nc -z elasticsearch 9200; do echo waiting for elasticsearch; sleep 10; done;
# Mise en route
# Base de données SQLite intégrée
BDD_FILE=/usr/src/app/pod/db.sqlite3
Expand Down
5 changes: 3 additions & 2 deletions pod/main/static/css/pod.css
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,8 +91,9 @@
--bs-orange: #c85208;

/* Use system font (faster load) */
--font-family-sans-serif: -apple-system, system-ui, "BlinkMacSystemFont",
"Segoe UI", "Roboto", "Verdana", sans-serif;
--font-family-sans-serif:
-apple-system, system-ui, "BlinkMacSystemFont", "Segoe UI", "Roboto",
"Verdana", sans-serif;

/** reinit before cleaning all old css vars ***/
--primary-background-color: var(--pod-background-neutre1-bloc) !important;
Expand Down
2 changes: 1 addition & 1 deletion pod/settings.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
# Version of the project
#

VERSION = "3.8.4"
VERSION = "3.9.0"


##
Expand Down
2 changes: 2 additions & 0 deletions pod/video/management/commands/check_obsolete_videos.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -509,6 +509,7 @@ def write_in_csv(self, vid: Video, arch_type: str) -> None:
"Source file",
"Description",
"Views",
"Draft",
]
if exists:
self.check_csv_header(file, fieldnames)
Expand Down Expand Up @@ -536,6 +537,7 @@ def write_in_csv(self, vid: Video, arch_type: str) -> None:
.replace("\n\n", "\n")
.replace("\n", "$newl$"),
"Views": vid.viewcount,
"Draft": vid.is_draft,
}
)

Expand Down
2 changes: 1 addition & 1 deletion requirements-conteneur.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
-r ./requirements-dev.txt
importlib-metadata==4.13.0
watchdog
Loading