Add support for use as pre-commit hook

[nialov]

Hey,

Thanks for the useful tool! Using cog as a pre-commit hook would increase its usefulness as you could verify that all cog code is run before committing. The config is as simple as I have implemented here: https://github.com/nialov/cog/blob/feat-add-as-pre-commit-hook/.pre-commit-hooks.yaml.
I have configured the implemented hook locally in a project as such:

  - repo: https://github.com/nialov/cog
    rev: 6d73562c5cb38753bd94414a6e7612b263bc278d
    hooks:
      - id: cog
        files: "(docs_src/index.rst)"

However, what flags to use and what to put as the default is definitely an open question. In my example I have used the checksum (-c) and inplace conversion (-r) flags. These probably cannot be defaults as cog can be used in numerous different ways.

Secondly, the command line cog should not fail when ran with no files inputted. This does not have to be the default but should be enabled by e.g., a flag (--no-fail-on-no-filenames or something more succinct).

I would not be surprised if multiple have not already implemented this functionality in their own projects/forks. A caveat in the implementation as a pre-commit hook is that the Python interpreter will not have access to any other packages other than cog and the standard library of Python so more advanced use cases are probably out of scope of the hook implementation.

[nedbat]

I like the idea of using cog as a pre-commit hook. I'm not sure what your first requested change is. It sounds like you have cog running just fine, so you don't need a change?

The second change (being ok with no files) makes sense. I'm not sure why I had it checking for that case explicitly.

[nialov]

The second change (being ok with no files) makes sense. I'm not sure why I had it checking for that case explicitly.

Sorry for the confusion, as I remember and from short testing right now the ability to not error when no files are passed is the only change I would require in the actual code.

Other than that, if you support the idea I think the .pre-commit-hooks.yaml file could be added so that this repo could be directly used in other people's pre-commit-config.yaml configuration. However, there is the caveat that the user will always have to explicitly setup the files in pre-commit-config.yaml that they want to run the cog hook on as there is no way for pre-commit to know which files to pass to cog otherwise.

I can make a pull request with these minor changes if you give a thumbs up!

[vergenzt]

FYI here's a pre-commit hook I've used locally that installs cogapp as a dependency and uses my preferred set of options (feel free to change for yourself):

click to expand

- repo: local
  hooks:

  - # https://github.com/nedbat/cog/issues/21#issuecomment-1919626992
    id: cog
    name: cog (regenerate files)
    language: python
    additional_dependencies:
    - cogapp
    entry: cog
    args:
    - -r # Replace the input file with the output.
    - -c # Checksum the output to protect it against accidental change.
    - -P # Use print() instead of cog.outl() for code output.
    - -p # Prepend the generator source with PROLOGUE:
    - 'import subprocess as sp, re, os, sys'
    always_run: true # so that new files get picked up in generated file list below
    files: |
      (?x)^(
      # (auto-determine which files have cog snippets)
      #[[[cog
      #  cog_files = sp.check_output(["git", "grep", "-lF", "[[[" + "cog"], text=True).splitlines()
      #  print(" |\n".join(map(re.escape, cog_files)))
      #]]]
      \.pre\-commit\-config\.yaml
      #[[[end]]] (checksum: c9a00cdff387b441693470ba709fa23d)
      )$

edit 2024-04-05: After using this for a while I've found that pre-commit's "only run on changed files" filter misses times that it needs to re-run cog. Since I always keep my cog snippets fast I'm now using this config:

  - # https://github.com/nedbat/cog/issues/21#issuecomment-1919626992
    id: cog
    name: cog (regenerate files)
    language: python
    additional_dependencies:
    - cogapp
    entry: bash -c 'git grep -lzF "[[[""cog" | xargs -0 cog "$@"'
    args:
    - -r # Replace the input file with the output.
    - -c # Checksum the output to protect it against accidental change.
    - -P # Use print() instead of cog.outl() for code output.
    - -p # Prepend the generator source with PROLOGUE:
    - 'import subprocess as sp, re, os, sys, pathlib as pl'
    pass_filenames: false
    always_run: true

[henryiii]

FWIW, pre-commit's caching is making the upgrade to 3.5 (which changes the checksum output) annoying when using the local hook; it only switches to the new version when the cache is invalidated. If there was a built-in .pre-commit-hooks.yaml, then pre-commit would be aware of the update and it wouldn't break randomly; it would be handled by the pre-commit autoupdate tooling. For now, cogapp>=3.5 fixes it.

Also, https://nedbatchelder.com/code/cog/ is behind and doesn't mention 3.5 (or 3.4).

I think the file would look something like this:

- id: cog
  name: cog
  description: Regenerate files with cog
  entry: cogapp -r
  language: python
  types_or: [text]
  minimum_pre_commit_version: 2.9.0

And a user would use it something like this:

  - id: cog
    args:
    - -c # Checksum the output to protect it against accidental change.
    - -P # Use print() instead of cog.outl() for code output.
    - -p # Prepend the generator source with PROLOGUE:
    - 'import subprocess as sp, re, os, sys, pathlib as pl'
    files: README.txt
    always_run: true # if it depends on other files changing

[nedbat]

Also, https://nedbatchelder.com/code/cog/ is behind and doesn't mention 3.5 (or 3.4).

Oops, thanks. I've added a redirect from that old URL to the new actual docs.

I'll work on the pre-commit support.

[nedbat]

I've added .pre-commit-hooks.yaml in https://github.com/nedbat/cog/tree/nedbat/pre-commit-hooks. Maybe I've misunderstood, but I don't think types_or is needed with just one type, so I've also removed the minimum pre-commit version. What is a good practice for documenting this?

[nedbat]

And, any pointers for how to write tests for this?

[neutrinoceros]

@nedbat I just tried your branch. A couple remarks:
you'll need to match entry to the command that's to be invoked (not the package)

-   entry: cogapp
+   entry: cog

I would also advise to keep it simple, at least at first

-   stages: [pre-commit, pre-merge-commit, pre-push, manual]
+   stages: [pre-commit]

regarding testing strategy, here's how I manage mine: https://github.com/neutrinoceros/inifix/blob/main/.github/workflows/check-pre-commit-hooks.yml

basically I rely on the fact that there are existing users and check that I don't break them. Possibly not exactly what you're looking for but I guess you could always spin your own "user" repo for this purpose.

[webknjaz]

@nedbat you might want to make the command serial like ruff does: https://github.com/astral-sh/ruff-pre-commit/blob/46b1c6f3270fdf8a53210daad27b30bd53dec8e2/.pre-commit-hooks.yaml#L1-L21. Plus, it might be interesting to have two entries as well — one for formatting and the other just for checking.

[webknjaz]

-   entry: cogapp
+   entry: cog

or go for

-   entry: cogapp
+   entry: python -Im cogapp