Throw error if umbrella-dir is contained in source-dir

[nh13]

When the source-dir is the repository root (though using a custom path), we'll end up having recursive copies of the umbrella-dir for each new preview in a PR. If we have many previews, then the history can get quick large, and the repo quite big (gigabytes!). Ideally, the umbrella-dir would not be committed inside of source-dir.

[rossjrw]

I typically see this issue come up in instances where the action is being misused.

The setup implied by encountering this problem is that your previews are both originating from and being deployed to the same place. I could see this happening if e.g. the entire main branch is being used as a Pages source - that would make sense for a website so simple that having a build process at all would be overkill.

Introduce PR previews to that setup and it is no longer just a simple static website - the main branch would contain a mix of human-authored and machine-generated files, the commits are a mix of human-authored and automated preview deployments, and worse, anyone making a PR that has the preview workflow running on it immediately has their unverified work committed directly to the main branch. At that point separating the website's source and what actually gets deployed becomes worthwhile.

I've no idea if anything like that applies to your use case. I'm tentatively leaving this issue open in case there's something I've missed - I'd be delighted to be incorrect.

I agree that the data build-up is a nasty side effect and is not explicitly warned against. As it is, rather than quietly ignore the umbrella-dir if I detect this situation, I'd prefer to throw an error and prevent it entirely.

[nh13]

The process I used was:

1. Generate docs on the main branch.
2. Clone a new version of the repo (say into the build sub-dir), then checkout the gh-pages branch.
3. Add the docs generated from the main branch to the gh-pages branch (in the sub-dir) and commit.
4. Run pr-preview-action with source-dir set to the build sub-dir that has the gh-pages branch checked out.

Since the gh-pages branch in the build sub-dir ALSO has the pr-preview sub-dir (build/pr-preview by default), the whole sub-dir is copied.

So it would go like this. With no pr-preview, the git repo on the gh-pages branch would have no pr-preview sub-directory. After a single run of this action, we'd get a pr-preview directory with the pr-preview/pr-123 sub-directory. So when we make new changes, the pr-preview/pr-123 sub-directory would get copied into new pr-preview/pr-123 sub-directory, so we'd have pr-preview/pr-123/pr-preview/pr-123` and so on.

So my question is if we build docs off main, and then commit to the gh-pages branch for docs (which we clone), we should always delete any existing pr-preview directory when using this action. I think it's a bit of a footgun in the way I am using it.

I agree an error would be nice (e.g. found an existing pr-preview directory at the root of your source-dir) to catch this. And I'd also ask is there a different intended way to use this action when using a main and gh-pages branch as per above?

[rossjrw]

Thanks for sharing your process. If I'm understanding you correctly, you are deploying your docs via gh-pages in a fairly standard way, just that you're doing it manually instead of via CI.

Typically the gh-pages branch has zero human-authored commits on it and is generated entirely by CI. I use https://github.com/JamesIves/github-pages-deploy-action to automate commits to gh-pages (and rossjrw/pr-preview-action uses it internally) - a typical workflow file using it might look like this:

# .github/workflows/deploy.yml
name: Deploy
on:
  push:
    branches:
      - main
jobs:
  build:
    steps:
      - name: Checkout
        # Checks out the main branch
        uses: actions/checkout@v4

      - name: Build
        # Builds documentation on the master branch - script outputs to build/
        run: ./build-docs.sh

      - name: Deploy
        # Copies built docs to the gh-pages branch (everything else there is removed, except pr-preview/), and commits gh-pages
        uses: JamesIves/github-pages-deploy-action@v4
        with:
          branch: gh-pages
          folder: build/
          clean-exclude: pr-preview/

That workflow doesn't commit anything to the main branch, and the gh-pages branch contains only the build output. (If you need all the other files too - no problem, change folder: build/ to folder: . and now gh-pages contains everything that's on the main branch, but also build/, which isn't on the main branch.)

(If you still want to only have documentation builds triggered manually, you don't have to use on: push: branches: - main like I suggested, you could use the worflow_dispatch event for example)

A PR preview workflow that works in tandem with that deploy workflow might look like this:

# .github/workflows/preview.yml
name: Preview
on:
  pull_request:
    types:
      - opened
      - reopened
      - synchronize
      - closed
concurrency: preview-${{ github.ref }}
jobs:
  deploy-preview:
    steps:
      - name: Checkout
        # Checkout the current branch (the one used for the PR)
        uses: actions/checkout@v4

      - name: Build
        # Builds documentation on the this branch - script outputs to build/
        run: ./build-docs.sh

      - name: Deploy preview
        # Copies build/ on this branch to pr-preview/pr-X/build/ on gh-pages branch, and commits gh-pages
        uses: rossjrw/pr-preview-action@v1
        with:
          source-dir: build/
          preview-branch: gh-pages
          umbrella-dir: pr-preview/

The key insight is that the PR's branch was (probably!) branched from main, so it doesn't include any of the generated files that are on gh-pages. The build/ directory and the pr-preview/ directory never get committed to the main branch, they only ever exist on the gh-pages branch. Every time you add a new preview, it comes from a PR that branched from main, so it never contains any stale previews. You never need to do anything to gh-pages yourself after setting it up once, and all files generated by the build process never leak back into the main branch.

That's what I'd consider a typical process for gh-pages. Does that make sense? Would be more than happy to clarify anything.

[nh13]

To make it more concrete, here's the action in action (sic): https://github.com/fulcrumgenomics/fgbio/blob/1f3a6a67499f8d8ce951a9a97ca64f062b8cede7/.github/workflows/unittests.yaml#L97-L105

The update to the gh-pages branch occurs here: https://github.com/fulcrumgenomics/fgbio/blob/1f3a6a67499f8d8ce951a9a97ca64f062b8cede7/.github/actions/docs/action.yaml#L1-L134

In particular, I had to enable sparse checkout here: https://github.com/fulcrumgenomics/fgbio/blob/1f3a6a67499f8d8ce951a9a97ca64f062b8cede7/.github/actions/docs/action.yaml#L32-L39

So we clone the repo into the build directory, make our changes there, then pass this as the source-dir to this action.

[rossjrw]

That's useful, thanks for the context.

Everything I've said so far has assumed that your main and gh-pages branches are derived from the same content, i.e. the source code for the stuff that's in the gh-pages branch can be found somewhere in the main branch.

I don't think that's the case for fgbio, though I found the workflows difficult to follow. I think the main branch and gh-pages branch contain totally different projects, like a monorepo setup, except projects are separated by branch rather than by directory. Is that right? (the rest of this comment will assume so)

To get around that and communicate between projects you have to temporarily clone one of them into a subdirectory of the other, make changes, and then re-commit it? If that's right this is some wild git usage. I've not seen this before.

That being said, even if I consider your main branch and your gh-pages branch to be entirely separate projects, then I think most of what I've already said still applies. The preview action still expects that the website source and the website deployment will be in separate branches; you are currently using gh-pages for both, but you don't have to be.

I would split your setup into 3 branches:

• main to contain whatever the repo actually is (I didn't look into it)
• docs (e.g.) to contain your documentation source, index.html etc - i.e. what is currently in your gh-pages, excluding PR previews
• gh-pages (by convention) is what actually gets deployed as the website, including generated files like PR previews

Then to build your docs you'd checkout main or whatever feature branch, clone docs into build/, run the build script, and commit the result to gh-pages. gh-pages is the only branch that contains PR previews, so when you checkout main or docs, there's no risk of duplicating previews.