Keep improving CI (#292)

BryanQuigley · web-flow · commit 81017695076d · 2025-02-04T00:06:27.000-08:00
Improve Linting, spelling, links and better control script

this includes spellcheck for both other files and markdown
updated lint script that has more options - including formatting
enabled markdown link checker as the first pass on making sure links work
update CI to only activate on changed files
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -10,6 +10,6 @@ jobs:
     steps:
       - uses: actions/checkout@v4
       - name: MegaLinter
-        run: ./scripts/lint
+        run: ./scripts/lint -c
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
diff --git a/.mega-linter.yml b/.mega-linter.yml
@@ -1,21 +1,22 @@
 # Configuration file for MegaLinter
 # See all available variables at https://megalinter.io/configuration/ and in linters documentation
 
-APPLY_FIXES: none
 DISABLE:
-  - SPELL
   - REPOSITORY
   - JSON
-  - MARKDOWN
   - JAVASCRIPT
   - CSS
   - HTML
   - COPYPASTE
 DISABLE_LINTERS:
   - YAML_V8R
+  - SPELL_LYCHEE
+  - MARKDOWN_MARKDOWNLINT
+  - MARKDOWN_MARKDOWN_TABLE_FORMATTER
 PRINT_ALL_FILES: false
 DOCKERFILE_HADOLINT_ARGUMENTS: "--ignore DL3008 --ignore DL4006 --ignore DL3009 --ignore DL3059"
+BASH_SHFMT_ARGUMENTS: "-i 4"
 SHOW_ELAPSED_TIME: true
-FLAVOR_SUGGESTIONS: false
+FLAVOR_SUGGESTIONS: true
 EMAIL_REPORTER: false
 FILEIO_REPORTER: false
diff --git a/_dataset_categories/budget-finance.md b/_dataset_categories/budget-finance.md
@@ -1,6 +1,7 @@
 ---
 name: Budget / Finance
 logo: "/img/categories/budget-finance.svg"
+# spell-checker: disable
 logo_credit: Renan Ferreira Santos from the Noun Project
 featured: true
 ---
diff --git a/_dataset_categories/education.md b/_dataset_categories/education.md
@@ -1,6 +1,7 @@
 ---
 name: Education
 logo: "/img/categories/education.svg"
+# spell-checker: disable
 logo_credit: Sergey Patutin from the Noun Project
 featured: true
 ---
diff --git a/_dataset_categories/parks-recreation.md b/_dataset_categories/parks-recreation.md
@@ -1,6 +1,7 @@
 ---
 name: Parks / Recreation
 logo: "/img/categories/parks-recreation.svg"
+# spell-checker: disable
 logo_credit: Pipe Rosas Licht from the Noun Project
 featured: true
 ---
diff --git a/_dataset_categories/public-safety.md b/_dataset_categories/public-safety.md
@@ -1,6 +1,7 @@
 ---
 name: Public Safety
 logo: "/img/categories/public-safety.svg"
+# spell-checker: disable
 logo_credit: Rohith M S from the Noun Project
 featured: true
 ---
diff --git a/_dataset_categories/transportation.md b/_dataset_categories/transportation.md
@@ -1,6 +1,7 @@
 ---
 name: Transportation
 logo: "/img/categories/transportation.svg"
+# spell-checker: disable
 logo_credit: Dmitry Baranovskiy from the Noun Project
 featured: true
 ---
diff --git a/_dataset_categories/uncategorized.md b/_dataset_categories/uncategorized.md
@@ -1,6 +1,7 @@
 ---
 name: Uncategorized
 logo: "/img/categories/uncategorized.svg"
+# spell-checker: disable
 logo_credit: Edward Boatman from the Noun Project
 featured: true
 ---
diff --git a/cspell.json b/cspell.json
@@ -0,0 +1,24 @@
+// cSpell Settings
+{
+    // Version of the setting file.  Always 0.2
+    "version": "0.2",
+    // language - current active spelling language
+    "language": "en",
+    "allowCompoundWords": true,
+    // words - list of words to be always considered correct
+    "words": [
+        "jkan",
+        "esac",
+        "jsonify",
+        "markdownify",
+        "Wisniewski",
+        "Licence",
+        "Uncategorized",
+        "decap",
+        "Phila",
+        "ublabs",
+        "customised",
+        "customise",
+	"shfmt"
+    ]
+}
diff --git a/docs/architecture.md b/docs/architecture.md
@@ -33,7 +33,7 @@ JKAN is a [Jekyll](http://jekyllrb.com/) site that uses JavaScript for enhanced
 | `_data/licenses.yml` | A list of licenses that datasets can be associated with |
 | `_datasets` | A directory of `.md` files containing dataset information in YAML front-matter |
 | `_dataset_categories` | A directory of `.md` files containing dataset categories in YAML front-matter |
-| `_includes` | HTML partials that are included in layouts. `display` is used by [schema files](/docs/configuration.md) |
+| `_includes` | HTML partials that are included in layouts. |
 | `_layouts` | Page templates |
 | `_organizations` | A directory of `.md` files containing organization information in YAML front-matter |
 | `editor` | Configuration for the Editor UI, described below |
diff --git a/docs/upgrading.md b/docs/upgrading.md
@@ -1,5 +1,5 @@
 # Upgrading
-See [release notes](/releases) for details of what's changed between versions.
+See [release notes](https://github.com/timwis/jkan/releases) for details of what's changed between versions.
 
 ## For every upgrade
 If your JKAN repository is a fork, you should see a banner at the top of the repository saying, e.g. "This branch is 20 commits ahead, 32 commits behind timwis:gh-pages". Clicking on the "X commits behind" link should take you to a page where you can initiate a pull request, which should show you the changes that will be applied to your repository. It should take you to:
@@ -12,11 +12,11 @@ If there are no conflicting changes in your fork, you'll be able to merge the pu
 
 ## v1.x to v2.0
 - If you use the editing UI
-  - You'll need to switch to the new editing UI, powered by Decap CMS. You don't need to host your JKAN site on Netlify to do so. Follow the [installation](installation) instructions to setup the OAuth integration.
+  - You'll need to switch to the new editing UI, powered by Decap CMS. You don't need to host your JKAN site on Netlify to do so. Follow the [installation](installation.md) instructions to setup the OAuth integration.
 - If you have custom categories
-  - You'll need to convert `_data/categories.yml` into a collection (a directory of files) in the `_dataset_categories` directory. See the [sample categories](/_dataset_categories) for the format. You should delete `_data/categories.yml` afterwards.
+  - You'll need to convert `_data/categories.yml` into a collection (a directory of files) in the `_dataset_categories` directory. See the [sample categories](../_dataset_categories/) for the format. You should delete `_data/categories.yml` afterwards.
 - If you have custom licenses
-  - We've also changed the structure of `_data/licenses.yml`. If you've customised that file at all, you'll need to update the structure to match the [new format](/_data/licenses.yml) as well.
+  - We've also changed the structure of `_data/licenses.yml`. If you've customised that file at all, you'll need to update the structure to match the [new format](../_data/licenses.yml) as well.
 - If you have customised templates
   - Replace any references to `site.data.categories` with `site.dataset_categories`
   - Replace any references to `site.data.licenses` with `site.data.licenses.items`
diff --git a/scripts/lint b/scripts/lint
@@ -1,2 +1,28 @@
-docker run --rm  -v $PWD:/tmp/lint oxsecurity/megalinter-cupcake
+#!/bin/bash
 
+USAGE="./lint [-f | --fix] [-c | --changes-only]"
+
+APPLY_FIXES=none
+VALIDATE_ALL_CODEBASE=true
+
+while [[ $# -gt 0 ]]; do
+    case "$1" in
+    -f | --fix)
+        APPLY_FIXES=all
+        ;;
+    -c | --changes-only)
+        VALIDATE_ALL_CODEBASE=false
+        ;;
+    *)
+        echo "Unknown option: $1"
+        echo "$USAGE"
+        exit 1
+        ;;
+    esac
+    shift
+done
+
+docker run --rm \
+    -e APPLY_FIXES=$APPLY_FIXES \
+    -e VALIDATE_ALL_CODEBASE=$VALIDATE_ALL_CODEBASE \
+    -v "$PWD":/tmp/lint oxsecurity/megalinter-python
