Skip to content

Commit

Permalink
setup: upgrade celery and other dependencies
Browse files Browse the repository at this point in the history
  • Loading branch information
ntarocco authored and zzacharo committed Nov 18, 2024
1 parent 299dda7 commit d7d7e47
Show file tree
Hide file tree
Showing 6 changed files with 235 additions and 42 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ jobs:

- name: Install dependencies
run: |
pip install -r requirements.pinned.txt
pip install -r requirements.txt
pip install .[$EXTRAS]
pip freeze
docker --version
Expand Down
34 changes: 22 additions & 12 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,25 @@ Prerequisites

Ensure that the following dependencies are installed with the specified versions:

1. **Python 3.9**
1. **Python 3.9**

2. **Node.js v18**
2. **Node.js v18**

3. **FFmpeg v5.0**
3. **FFmpeg v5.0**

4. **Docker v2 or later:**
4. **Docker v2 or later:**
If Docker is not already installed, download and install Docker Desktop from the `official Docker website <https://www.docker.com/products/docker-desktop/>`_.

Update dependencies
======================

To update Python dependencies you need to run `npm install` in the target deployment environment:

.. code-block:: shell
$ docker run -it --platform="linux/amd64" --rm -v $(pwd):/app -w /app \
registry.cern.ch/inveniosoftware/almalinux:1 \
sh -c "dnf install -y openldap-devel && pip install -e . && pip freeze > requirements.new.txt"
Installation and Setup
======================
Expand Down Expand Up @@ -50,14 +60,14 @@ Use Docker Compose to start the required containers in detached mode:

The ``scripts`` folder contains the necessary setup scripts to initialize and configure your instance.

**1. Bootstrap Script**
**1. Bootstrap Script**
Initialize the environment by running the bootstrap script:

.. code-block:: bash
./scripts/bootstrap
**Troubleshooting**:
**Troubleshooting**:

These are the macOS solutions using ``brew`` for installation.

Expand All @@ -82,14 +92,14 @@ The ``scripts`` folder contains the necessary setup scripts to initialize and co
brew install openssl@3
**2. Setup Script**
**2. Setup Script**
Run the setup script to finalize the installation and configuration:

.. code-block:: bash
./scripts/setup
**Troubleshooting**:
**Troubleshooting**:
If you encounter the error ``connection to server at "localhost", port 5432 failed: FATAL: role ".." does not exist``, it may indicate an issue with the database role or a port conflict. To diagnose:

1. First, connect to the Docker database container and verify that the expected role exists and the database is working correctly.
Expand All @@ -112,21 +122,21 @@ The ``scripts`` folder contains the necessary setup scripts to initialize and co

To facilitate local development, open multiple terminal sessions and run the following commands separately:

- **Start Web Server**
- **Start Web Server**
This command launches the web server:

.. code-block:: bash
./scripts/server
- **Start Celery Workers**
- **Start Celery Workers**
Celery workers are required for background task processing:

.. code-block:: bash
./scripts/celery
- **Watch Frontend Code**
- **Watch Frontend Code**
This command watches frontend code for changes and rebuilds assets as needed:

.. code-block:: bash
Expand All @@ -147,7 +157,7 @@ or (to also show test coverage) ::
License
=======

Copyright (C) 2013-2016 CERN.
Copyright (C) 2013-2024 CERN.

CDS is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

Expand Down
26 changes: 0 additions & 26 deletions requirements.pinned.txt

This file was deleted.

211 changes: 211 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,211 @@
alembic==1.10.4
amqp==5.3.1
aniso8601==9.0.1
appdirs==1.4.4
arrow==0.17.0
asttokens==2.4.1
async-timeout==5.0.1
attrs==24.2.0
babel==2.16.0
base32-lib==1.0.2
beautifulsoup4==4.12.3
billiard==4.2.1
bleach==6.2.0
blinker==1.9.0
cachelib==0.9.0
cairocffi==1.7.1
CairoSVG==1.0.22
celery==5.3.6
certifi==2024.8.30
cffi==1.17.1
charset-normalizer==3.4.0
click==8.1.7
click-default-group==1.2.4
click-didyoumean==0.3.1
click-plugins==1.1.1
click-repl==0.3.0
commonmark==0.9.1
counter-robots==2018.6
cryptography==43.0.3
datacite==1.2.0
dcxml==0.1.1
decorator==5.1.1
defusedxml==0.7.1
Deprecated==1.2.15
dictdiffer==0.9.0
distlib==0.3.8
dnspython==2.7.0
dojson==1.5.0
email_validator==2.2.0
Events==0.5
exceptiongroup==1.2.2
executing==2.1.0
fastjsonschema==2.20.0
Flask==2.2.5
Flask-Admin==1.6.1
Flask-Alembic==2.0.1
flask-babel==4.0.0
Flask-Breadcrumbs==0.5.1
Flask-Caching==2.3.0
Flask-CeleryExt==0.5.0
Flask-Collect-Invenio==1.4.0
Flask-Cors==5.0.0
flask-iiif==1.1.1
Flask-KVSession-Invenio==0.6.3
Flask-Limiter==2.9.2
Flask-Login==0.6.3
Flask-Mail==0.9.1
flask-menu==1.0.1
Flask-OAuthlib==0.9.6
Flask-Principal==0.4.0
Flask-RESTful==0.3.10
Flask-Security-Invenio==3.3.3
flask-shell-ipython==0.5.3
Flask-SQLAlchemy==2.5.1
flask-talisman==0.8.1
flask-webpackext==1.0.2
Flask-WTF==1.2.2
fs==2.4.16
ftfy==6.3.1
future==1.0.0
github3.py==4.0.1
greenlet==3.1.1
idna==3.10
idutils==1.4.2
importlib_metadata==8.5.0
importlib_resources==6.4.5
infinity==1.5
intervals==0.9.2
invenio-access==2.1.0
invenio-accounts==5.1.5
invenio-admin==1.5.1
invenio-app==1.5.0
invenio-assets==3.0.3
invenio-base==1.4.0
invenio-cache==1.3.0
invenio-celery==1.3.1
invenio-config==1.0.4
invenio-db==1.3.1
invenio-files-rest==2.2.2
invenio-formatter==2.0.3
invenio-i18n==2.1.2
invenio-iiif==1.2.0
invenio-indexer==2.4.0
invenio-jsonschemas==1.1.4
invenio-logging==2.1.1
invenio-mail==2.1.1
invenio-oaiserver==2.2.3
invenio-oauth2server==2.4.0
invenio-oauthclient==3.5.1
invenio-opendefinition==2.0.0a2
invenio-pages==1.0.0a7
invenio-pidstore==1.3.2
invenio-previewer==2.2.1
invenio-queues==1.0.1
invenio-records==2.4.1
invenio-records-files==1.2.1
invenio-records-rest==2.4.1
invenio-records-ui==1.2.1
invenio-rest==1.3.1
invenio-search==2.4.1
invenio-search-ui==2.9.1
invenio-sequencegenerator==1.0.0a3
invenio-stats==4.2.0
invenio-theme==3.5.0
invenio-userprofiles==1.2.5
ipython==8.18.1
isbnlib==3.10.14
itsdangerous==2.0.1
jedi==0.19.2
Jinja2==3.1.4
jsmin==3.0.1
jsonpatch==1.33
jsonpointer==3.0.0
jsonref==1.1.0
jsonresolver==0.3.2
jsonschema==4.9.1
jupyter_client==8.6.3
jupyter_core==5.7.2
jupyterlab_pygments==0.3.0
kombu==5.4.2
limits==3.13.0
lxml==5.3.0
lxml_html_clean==0.4.1
Mako==1.3.6
MarkupSafe==3.0.2
marshmallow==3.23.1
matplotlib-inline==0.1.7
maxminddb==2.6.2
maxminddb-geolite2==2018.703
mistune==3.0.2
msgpack==1.1.0
nbclient==0.10.0
nbconvert==7.16.4
nbformat==5.10.4
oauthlib==2.1.0
opensearch-dsl==2.1.0
opensearch-py==2.7.1
ordered-set==4.1.0
packaging==24.2
pandocfilters==1.5.1
parso==0.8.4
passlib==1.7.4
pexpect==4.9.0
pillow==11.0.0
platformdirs==4.2.0
pluggy==0.13.1
prompt_toolkit==3.0.48
psycopg2-binary==2.9.10
ptyprocess==0.7.0
pure_eval==0.2.3
pyasn1==0.6.1
pyasn1_modules==0.4.1
pycountry==22.3.5
pycparser==2.22
Pygments==2.18.0
PyJWT==2.10.0
pynpm==0.2.0
pyrsistent==0.20.0
python-dateutil==2.9.0.post0
python-geoip==1.2
python-ldap==3.4.4
pytz==2024.2
pywebpack==2.0.1
pyzmq==26.2.0
redis==5.2.0
requests==2.32.3
requests-oauthlib==1.1.0
requests-toolbelt==1.0.0
rich==12.6.0
sentry-sdk==1.45.1
simplejson==3.19.3
simplekv==0.14.1
six==1.15.0
soupsieve==2.6
speaklater==1.3
SQLAlchemy==1.4.54
SQLAlchemy-Continuum==1.4.1
SQLAlchemy-Utils==0.38.3
stack-data==0.6.3
tinycss2==1.4.0
tornado==6.4.1
traitlets==5.14.3
typing_extensions==4.12.2
ua-parser==0.18.0
uritemplate==4.1.1
uritemplate.py==1.0.1
uritools==4.0.3
urllib3==1.26.20
validators==0.34.0
vine==5.1.0
Wand==0.6.13
watchdog==2.2.1
wcwidth==0.2.13
webargs==5.5.3
webencodings==0.5.1
Werkzeug==2.2.3
wrapt==1.16.0
WTForms==3.1.2
WTForms-Alchemy==0.18.0
WTForms-Components==0.11.0
zipp==3.21.0
2 changes: 1 addition & 1 deletion scripts/bootstrap
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ set -e

script_path=$(dirname "$0")

requirements_file=${1:-'requirements.pinned.txt'}
requirements_file=${1:-'requirements.txt'}

pip install -r $requirements_file
pip install -e . --upgrade
Expand Down
2 changes: 0 additions & 2 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -102,8 +102,6 @@ install_requires =
# Pinned due to before_first_request deprecation https://flask.palletsprojects.com/en/2.2.x/api/#flask.Flask.before_first_request
Flask>=2.2.0,<2.3.0
Werkzeug>=2.2.0,<2.3.0
# check pin
celery==5.2.7
# extras
arrow>=0.7.0,<1.0.0
CairoSVG>=1.0.20,<2.0.0
Expand Down

0 comments on commit d7d7e47

Please sign in to comment.