Final renaming of models to include sources

Author: jochemvandooren

CHANGELOG.md

@@ -8,6 +8,11 @@ and this project adheres to
 
 ## [Unreleased]
 
+- **Breaking**: Support linting of sources.
+- **Breaking**: `--fail_any_model_under` becomes `--fail-any-item-under` and
+  `--fail_project_under` becomes `--fail-project-under`.
+- **Breaking**: `model_filter_names` becomes `rule_filter_names`.
+
 ## [0.6.0] - 2024-08-23
 
 - **Breaking**: Improve error handling in CLI. Log messages are written in

README.md

@@ -11,7 +11,7 @@
 
 ## What is `dbt-score`?
 
-`dbt-score` is a linter for dbt model metadata.
+`dbt-score` is a linter for dbt metadata.
 
 [dbt][dbt] (Data Build Tool) is a great framework for creating, building,
 organizing, testing and documenting _data models_, i.e. data sets living in a

docs/configuration.md

@@ -51,8 +51,8 @@ The following options can be set in the `pyproject.toml` file:
 - `disabled_rules`: A list of rules to disable.
 - `fail_project_under` (default: `5.0`): If the project score is below this
   value the command will fail with return code 1.
-- `fail_any_item_under` (default: `5.0`): If any model or source scores below this value
-  the command will fail with return code 1.
+- `fail_any_item_under` (default: `5.0`): If any model or source scores below
+  this value the command will fail with return code 1.
 
 #### Badges configuration
 
@@ -70,7 +70,7 @@ All badges except `wip` can be configured with the following option:
 
 - `threshold`: The threshold for the badge. A decimal number between `0.0` and
   `10.0` that will be used to compare to the score. The threshold is the minimum
-  score required for a model to be rewarded with a certain badge.
+  score required for a model or source to be rewarded with a certain badge.
 
 The default values can be found in the
 [BadgeConfig](reference/config.md#dbt_score.config.BadgeConfig).

docs/create_rules.md

@@ -1,9 +1,9 @@
 # Create rules
 
-In order to lint and score models or sources, `dbt-score` uses a set of 
-rules that are applied to each item. A rule can pass or fail when it is run. 
-Based on the severity of the rule, items are scored with the weighted 
-average of the rules results. Note that `dbt-score` comes bundled with a
+In order to lint and score models or sources, `dbt-score` uses a set of rules
+that are applied to each item. A rule can pass or fail when it is run. Based on
+the severity of the rule, items are scored with the weighted average of the
+rules results. Note that `dbt-score` comes bundled with a
 [set of default rules](rules/generic.md).
 
 On top of the generic rules, it's possible to add your own rules. Two ways exist
@@ -31,10 +31,11 @@ The name of the function is the name of the rule and the docstring of the
 function is its description. Therefore, it is important to use a
 self-explanatory name for the function and document it well.
 
-The type annotation for the rule's argument dictates whether the rule should
-be applied to dbt models or sources.
+The type annotation for the rule's argument dictates whether the rule should be
+applied to dbt models or sources.
 
 Here is the same example rule, applied to sources:
+
 ```python
 from dbt_score import rule, RuleViolation, Source
 
@@ -68,7 +69,7 @@ class ModelHasDescription(Rule):
         """Evaluate the rule."""
         if not model.description:
             return RuleViolation(message="Model lacks a description.")
-        
+
 class SourceHasDescription(Rule):
    description = "A source should have a description."
 
@@ -116,8 +117,8 @@ def sql_has_reasonable_number_of_lines(model: Model, max_lines: int = 200) -> Ru
 ### Filtering rules
 
 Custom and standard rules can be configured to have filters. Filters allow
-models or sources to be ignored by one or multiple rules if the item doesn't satisfy
-the filter criteria.
+models or sources to be ignored by one or multiple rules if the item doesn't
+satisfy the filter criteria.
 
 Filters are created using the same discovery mechanism and interface as custom
 rules, except they do not accept parameters. Similar to Python's built-in
@@ -138,7 +139,8 @@ class SkipSchemaY(RuleFilter):
       return model.schema.lower() != 'y'
 ```
 
-Filters also rely on type-annotations to dictate whether they apply to models or sources:
+Filters also rely on type-annotations to dictate whether they apply to models or
+sources:
 
 ```python
 from dbt_score import RuleFilter, rule_filter, Source
@@ -154,7 +156,6 @@ class SkipSourceDatabaseB(RuleFilter):
       return source.database.lower() != 'b'
 ```
 
-
 Similar to setting a rule severity, standard rules can have filters set in the
 [configuration file](configuration.md/#tooldbt-scorerulesrule_namespacerule_name),
 while custom rules accept the configuration file or a decorator parameter.

docs/get_started.md

@@ -40,8 +40,8 @@ It's also possible to automatically run `dbt parse`, to generate the
 dbt-score lint --run-dbt-parse
 ```
 
-To lint only a selection of models, the argument `--select` can be used. It
-accepts any
+To lint only a selection of models or sources, the argument `--select` can be
+used. It accepts any
 [dbt node selection syntax](https://docs.getdbt.com/reference/node-selection/syntax):
 
 ```shell

docs/index.md

@@ -2,8 +2,9 @@
 
 `dbt-score` is a linter for [dbt](https://www.getdbt.com/) metadata.
 
-dbt allows data practitioners to organize their data in to _models_. Those
-models have metadata associated with them: documentation, tests, types, etc.
+dbt allows data practitioners to organize their data in to _models_ and
+_sources_. Those models and sources have metadata associated with them:
+documentation, tests, types, etc.
 
 `dbt-score` allows to lint and score this metadata, in order to enforce (or
 encourage) good practices.
@@ -25,15 +26,15 @@ score.
 
 ## Philosophy
 
-dbt models are often used as metadata containers: either in YAML files or
-through the use of `{{ config() }}` blocks, they are associated with a lot of
+dbt models/sources are often used as metadata containers: either in YAML files
+or through the use of `{{ config() }}` blocks, they are associated with a lot of
 information. At scale, it becomes tedious to enforce good practices in large
-data teams dealing with many models.
+data teams dealing with many models/sources.
 
 To that end, `dbt-score` has 2 main features:
 
-- It runs rules on dbt models and sources, and displays any rule violations. These can be used in
-  interactive environments or in CI.
+- It runs rules on dbt models and sources, and displays any rule violations.
+  These can be used in interactive environments or in CI.
 - Using those run results, it scores items, to ascribe them a measure of their
   maturity. This score can help gamify metadata improvements/coverage, and be
   reflected in data catalogs.

docs/programmatic_invocations.md

@@ -61,9 +61,9 @@ When `dbt-score` terminates, it exists with one of the following exit codes:
   project being linted either doesn't raise any warning, or the warnings are
   small enough to be above the thresholds. This generally means "successful
   linting".
-- `1` in case of linting errors. This is the unhappy case: some models in the
-  project raise enough warnings to have a score below the defined thresholds.
-  This generally means "linting doesn't pass".
+- `1` in case of linting errors. This is the unhappy case: some models or
+  sources in the project raise enough warnings to have a score below the defined
+  thresholds. This generally means "linting doesn't pass".
 - `2` in case of an unexpected error. This happens for example if something is
   misconfigured (for example a faulty dbt project), or the wrong parameters are
   given to the CLI. This generally means "setup needs to be fixed".

src/dbt_score/cli.py

@@ -80,7 +80,7 @@ def cli() -> None:
     default=False,
 )
 @click.option(
-    "--fail_project_under",
+    "--fail-project-under",
     help="Fail if the project score is under this value.",
     type=float,
     is_flag=False,

src/dbt_score/formatters/json_formatter.py

@@ -4,7 +4,7 @@
 
 ```json
 {
-    "models": {
+    "evaluables": {
         "model_foo": {
             "score": 5.0,
             "badge": "🥈",
@@ -60,14 +60,14 @@ class JSONFormatter(Formatter):
     def __init__(self, *args: Any, **kwargs: Any):
         """Instantiate formatter."""
         super().__init__(*args, **kwargs)
-        self._model_results: dict[str, dict[str, Any]] = {}
+        self.evaluable_results: dict[str, dict[str, Any]] = {}
         self._project_results: dict[str, Any]
 
     def evaluable_evaluated(
         self, evaluable: Evaluable, results: EvaluableResultsType, score: Score
     ) -> None:
         """Callback when an evaluable item has been evaluated."""
-        self._model_results[evaluable.name] = {
+        self.evaluable_results[evaluable.name] = {
             "score": score.value,
             "badge": score.badge,
             "pass": score.value >= self._config.fail_any_item_under,
@@ -76,19 +76,19 @@ def evaluable_evaluated(
         for rule, result in results.items():
             severity = rule.severity.name.lower()
             if result is None:
-                self._model_results[evaluable.name]["results"][rule.source()] = {
+                self.evaluable_results[evaluable.name]["results"][rule.source()] = {
                     "result": "OK",
                     "severity": severity,
                     "message": None,
                 }
             elif isinstance(result, RuleViolation):
-                self._model_results[evaluable.name]["results"][rule.source()] = {
+                self.evaluable_results[evaluable.name]["results"][rule.source()] = {
                     "result": "WARN",
                     "severity": severity,
                     "message": result.message,
                 }
             else:
-                self._model_results[evaluable.name]["results"][rule.source()] = {
+                self.evaluable_results[evaluable.name]["results"][rule.source()] = {
                     "result": "ERR",
                     "severity": severity,
                     "message": str(result),
@@ -102,7 +102,7 @@ def project_evaluated(self, score: Score) -> None:
             "pass": score.value >= self._config.fail_project_under,
         }
         document = {
-            "models": self._model_results,
+            "evaluables": self.evaluable_results,
             "project": self._project_results,
         }
         print(json.dumps(document, indent=2, ensure_ascii=False))

src/dbt_score/lint.py

@@ -1,4 +1,4 @@
-"""Lint dbt models metadata."""
+"""Lint dbt metadata."""
 
 from pathlib import Path
 from typing import Iterable, Literal

src/dbt_score/models.py

@@ -44,7 +44,7 @@ def from_raw_values(cls, raw_values: dict[str, Any]) -> "Constraint":
 
 @dataclass
 class Test:
-    """Test for a column or model.
+    """Test for a column, model or source.
 
     Attributes:
         name: The name of the test.
@@ -372,7 +372,7 @@ def __hash__(self) -> int:
 
 
 class ManifestLoader:
-    """Load the models and tests from the manifest."""
+    """Load the models, sources and tests from the manifest."""
 
     def __init__(self, file_path: Path, select: Iterable[str] | None = None):
         """Initialize the ManifestLoader.

tests/formatters/test_json_formatter.py

@@ -29,7 +29,7 @@ def test_json_formatter(
     assert (
         stdout
         == """{
-  "models": {
+  "evaluables": {
     "model1": {
       "score": 10.0,
       "badge": "🥇",

tests/test_cli.py

@@ -77,7 +77,7 @@ def test_fail_project_under(manifest_path):
     with patch("dbt_score.cli.Config._load_toml_file"):
         runner = CliRunner()
         result = runner.invoke(
-            lint, ["--manifest", manifest_path, "--fail_project_under", "10.0"]
+            lint, ["--manifest", manifest_path, "--fail-project-under", "10.0"]
         )
 
         assert "model1" in result.output