Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adjust CI scripts for package specific tasks #139

Merged
merged 18 commits into from
Nov 1, 2023
Merged

Adjust CI scripts for package specific tasks #139

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
e934d90
Add path ignore
maxschulz-COL Oct 31, 2023
12a7769
Change to certain path only
maxschulz-COL Oct 31, 2023
6703662
Same change to integration tests
maxschulz-COL Oct 31, 2023
14e7cfb
Linting
maxschulz-COL Oct 31, 2023
eaff55b
Merge branch 'main' into test/check_CI
maxschulz-COL Nov 1, 2023
ba39803
Divide tests to correct packages
maxschulz-COL Nov 1, 2023
8a18ad6
TBR: Change in vizro-ai
maxschulz-COL Nov 1, 2023
9137bba
Fix unit test script
maxschulz-COL Nov 1, 2023
d6d6a03
Revert "TBR: Change in vizro-ai"
maxschulz-COL Nov 1, 2023
644241a
TBR change in docs of vizro-ai
maxschulz-COL Nov 1, 2023
3118b8a
Revert "TBR change in docs of vizro-ai"
maxschulz-COL Nov 1, 2023
d270d86
TBR chagne in core
maxschulz-COL Nov 1, 2023
a795213
Revert "TBR chagne in core"
maxschulz-COL Nov 1, 2023
a80399e
PR comments
maxschulz-COL Nov 1, 2023
720b86c
Changelog
maxschulz-COL Nov 1, 2023
e153a5d
Changelog
maxschulz-COL Nov 1, 2023
2c9e591
Run all tests alwyas
maxschulz-COL Nov 1, 2023
ca8c1c1
Fix concurrency
maxschulz-COL Nov 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 9 additions & 11 deletions .github/workflows/lint-vizro-ai.yml → .github/workflows/checks-vizro-ai.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,17 +1,21 @@
name: lint-vizro-ai
name: checks-vizro-ai
defaults:
run:
working-directory: vizro-ai

on:
push:
branches: [main]
paths:
- "vizro-ai/**"
pull_request:
branches:
- "main"
paths:
- "vizro-ai/**"

concurrency:
group: lint-${{ github.head_ref }}
group: checks-ai-${{ github.head_ref }}
cancel-in-progress: true

env:
Expand Down Expand Up @@ -42,18 +46,12 @@ jobs:
- name: List dependencies
run: hatch run all.py${{ matrix.python-version }}:pip freeze

- name: Lint
run: hatch run all.py${{ matrix.python-version }}:lint
maxschulz-COL marked this conversation as resolved.
Show resolved Hide resolved

- name: Check requirements for Snyk are up to date
run: |
pwd
hatch run all.py${{ matrix.python-version }}:update-snyk-requirements --check
run: hatch run all.py${{ matrix.python-version }}:update-snyk-requirements --check

- name: Find added changelog fragments
id: added-files
run: |
pwd
if ${{ github.event_name == 'pull_request' }}; then
echo "added_files=$(git diff --name-only --diff-filter=A -r HEAD^1 HEAD -- changelog.d/*.md | xargs)" >> $GITHUB_OUTPUT
else
Expand All @@ -64,9 +62,9 @@ jobs:
run: |
if [ -z "${{ steps.added-files.outputs.added_files }}" ];
then
echo "No changelog fragment .md file within changelog.d was detected. Run 'hatch run docs:changelog' to create such a fragment.";
echo "No changelog fragment .md file within changelog.d was detected. Run 'hatch run changelog:add' to create such a fragment.";
echo "If your PR contains changes that should be mentioned in the CHANGELOG in the next release, please uncomment the relevant section in your created fragment and describe the changes to the user."
echo "If your changes are not relevant for the CHANGELOG, please save and commit the file as."
echo "If your changes are not relevant for the CHANGELOG, please save and commit the file as is."
exit 1
else
echo "${{ steps.added-files.outputs.added_files }} was added - ready to go!";
Expand Down
11 changes: 6 additions & 5 deletions .github/workflows/lint-vizro-core.yml → .github/workflows/checks-vizro-core.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: lint-vizro-core
name: checks-vizro-core

defaults:
run:
Expand All @@ -7,12 +7,16 @@ defaults:
on:
push:
branches: [main]
paths:
- "vizro-core/**"
pull_request:
branches:
- "main"
paths:
- "vizro-core/**"

concurrency:
group: lint-${{ github.head_ref }}
group: checks-core-${{ github.head_ref }}
cancel-in-progress: true

env:
Expand Down Expand Up @@ -43,9 +47,6 @@ jobs:
- name: List dependencies
run: hatch run all.py${{ matrix.python-version }}:pip freeze

- name: Lint
run: hatch run all.py${{ matrix.python-version }}:lint

- name: Check schema is up to date
run: hatch run all.py${{ matrix.python-version }}:schema --check

Expand Down
47 changes: 47 additions & 0 deletions .github/workflows/lint-vizro-all.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: lint-vizro-all

defaults:
run:
working-directory: vizro-core # but could be any folder

on:
push:
branches: [main]
pull_request:
branches:
- "main"

concurrency:
group: lint-${{ github.head_ref }}
cancel-in-progress: true

env:
PYTHONUNBUFFERED: "1"
FORCE_COLOR: "1"

jobs:
run:
name: Python ${{ matrix.python-version }} on Linux
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.8"]

steps:
- uses: actions/checkout@v4
with:
fetch-depth: ${{ github.event_name == 'pull_request' && 2 || 0 }}

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}

- name: Install Hatch
run: pip install --upgrade hatch

- name: List dependencies
run: hatch run all.py${{ matrix.python-version }}:pip freeze

- name: Lint
run: hatch run all.py${{ matrix.python-version }}:lint
Comment on lines +42 to +47
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since linting just requires pre-commit actually we should probably make a new deatched=True Hatch environment lint for it that doesn't include all of the dev dependencies. This way we could just do hatch run lint:lint on CI and it would be significantly faster. At the moment we install all the dependencies for all.py3.8 and then run pre-commit that uses only one of those dependencies which is a bit silly.

Not a big deal though, so feel free to do in a separate PR another time if you prefer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm good idea, let me see if I can incorporate that quickly still

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the other hand, hatch run lint is really used a lot, so would prefer not to have hatch run lint:lint. Or do you mean to keep pre-commit in the default environment as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's revisit in another PR if we still want to do changes

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maxschulz-COL yes, you're right - I originally put lint in the default environment precisely so that we don't have to write hatch run lint:lint. But probably the right way to do it is to make the hatch run lint:lint the "correct" command with a detached lint environment and then put a script "lint" = ["hatch run lint:lint"] in the default environment to just act as a shortcut so you can still do hatch run lint.

1 change: 1 addition & 0 deletions .github/workflows/test-integration-vizro-ai.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
# run:
# working-directory: vizro-core
#
##### TODO: adjust below according to other scripts
#on:
# # push:
# # branches: [main]
Expand Down
48 changes: 48 additions & 0 deletions vizro-ai/changelog.d/20231101_120705_maximilian_schulz_check_CI.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
<!--
A new scriv changelog fragment.

Uncomment the section that is right (remove the HTML comment wrapper).
-->

<!--
### Highlights ✨

- A bullet item for the Highlights ✨ category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
<!--
### Removed

- A bullet item for the Removed category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
<!--
### Added

- A bullet item for the Added category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
<!--
### Changed

- A bullet item for the Changed category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
<!--
### Deprecated

- A bullet item for the Deprecated category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
<!--
### Fixed

- A bullet item for the Fixed category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
<!--
### Security

- A bullet item for the Security category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
4 changes: 1 addition & 3 deletions vizro-ai/hatch.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,9 +59,7 @@ dependencies = [
"mkdocstrings[python]"
]
detached = true

[envs.docs.scripts]
serve = "mkdocs serve"
scripts = {serve = "mkdocs serve"}

[version]
path = "src/vizro_ai/__init__.py"
48 changes: 48 additions & 0 deletions vizro-core/changelog.d/20231101_120104_maximilian_schulz_check_CI.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
<!--
A new scriv changelog fragment.

Uncomment the section that is right (remove the HTML comment wrapper).
-->

<!--
### Highlights ✨

- A bullet item for the Highlights ✨ category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
<!--
### Removed

- A bullet item for the Removed category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
<!--
### Added

- A bullet item for the Added category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
<!--
### Changed

- A bullet item for the Changed category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
<!--
### Deprecated

- A bullet item for the Deprecated category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
<!--
### Fixed

- A bullet item for the Fixed category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
<!--
### Security

- A bullet item for the Security category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
3 changes: 1 addition & 2 deletions vizro-core/hatch.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,8 +75,7 @@ dependencies = [
"mkdocs",
"mkdocs-material",
"mkdocs-git-revision-date-localized-plugin",
"mkdocstrings[python]",
"scriv"
"mkdocstrings[python]"
]
detached = true
scripts = {serve = "mkdocs serve"}
Expand Down
Loading