Canonize all the things

[icweaver]

Just opening this tracking issue for moving links and such under the JuliaAstro MultiDocumenter umbrella

Inspo: SciML/SciMLDocs#53

In progress

• PhotometricFilters.jl
  • Needs stable release

Complete

• AstroAngles.jl (Canonize AstroAngles.jl#15)
• AstroImages.jl (Canonize AstroImages.jl#68) @sefffal
• AstroTime.jl (Canonize AstroTime.jl#77) @helgee
• BoxLeastSquares.jl (Canonize BoxLeastSquares.jl#12)
• CALCEPH.jl (Canonize CALCEPH.jl#21) @bgodard
  • Should we register this package first? Register? BackgroundMeshes.jl#14
  • Combine with EphemerisSources.jl? CALCEPH.jl#20
• CCDReduction.jl (Canonize CCDReduction.jl#51)
• CFITSIO.jl (Canonize CFITSIO.jl#95)
• Cosmology.jl (Canonize Cosmology.jl#56) @giordano
• DustExtinction.jl (Canonize DustExtinction.jl#72) @cgarling
• EarthOrientation.jl (Canonize EarthOrientation.jl#26) @helgee
• EphemerisSources.jl
  • Uses own docs build
• ERFA.jl (Canonize ERFA.jl#71)
• GeneralAstrodynamics.jl
  • Uses own docs build
• FITSIO.jl (Canonize FITSIO.jl#226)
• LACosmic.jl (Canonize LACosmic.jl#17)
• LombScargle.jl (Canonize LombScargle.jl#41) @giordano
• Orbits.jl (Canonize Orbits.jl#12) @sefffal
  • Joining forces with PlanetOrbits.jl Orbits.jl#5?
• Photometry.jl (Canonize Photometry.jl#73)
• PSFModels.jl (Canonize PSFModels.jl#26)
• Reproject.jl (Canonize Reproject.jl#23) @giordano
• SAOImageDS9.jl (Canonize SAOImageDS9.jl#11) @emmt
• Spectra.jl
• SPICE.jl (Canonize SPICE.jl#33) @helgee
  • Combine with EphemerisSources.jl? SPICE.jl#32
• SkyCoords.jl (Canonize SkyCoords.jl#64)
• Transits.jl (Canonize Transits.jl#56)
• UnitfulAstro.jl (Canonize UnitfulAstro.jl#53) @giordano
• WCS.jl (Canonize WCS.jl#52) @giordano
• XPA.jl (Canonize XPA.jl#11) @emmt

To discuss

• AstroBase.jl @helgee
  • Discuss plans to archive?
• AstroImageView.jl @aquatiko
  • Is this package being maintained? Status of this package? AstroImageView.jl#8
• AstroLib.jl @giordano
  • Discuss plans to archive?
• BackgroundMeshes.jl*
  • Register at some point? Register? BackgroundMeshes.jl#14
    (Canonize ERFA.jl#71)
• Planck.jl*
  • Register, or maybe combine with another package?
• PulsarSearch.jl* @matteobachetti
  • Currently a draft
  • Maybe combine with https://github.com/astrogewgaw/EpochFolding.jl?
• VOTables.jl
  • Archived: Consolidate VOTables.jl#3, Deprecate JuliaAstro/VOTables.jl VOTables.jl#4

[icweaver]

Canonization experiment updates

Following up from: JuliaAstro/AstroAngles.jl#15 (comment)

Ok, wasn't too bad. Here are the changes:

• Updated badges and Documenter canonical link: https://github.com/JuliaAstro/Spectra.jl/pull/18/files
• Added redirect in gh-pages repo: JuliaAstro/Spectra.jl@05f77bf
• Added juliaastro.org/Spectra/stable link to repo sidebar

https://juliaastro.org/Spectra.jl now redirects to our MultiDocumenter site, while https://juliaastro.org/Spectra.jl/stable and https://juliaastro.org/Spectra.jl/dev still point to their respective original github pages. Does that seem reasonable?

I then made a test change in the original docs and it seemed to reflect on the the MultiDocumenter site fine after re-running the doc build. Is my understanding correct that our cron job will help keep changes like this in JuliaAstro proper in sync automatically?

JuliaAstro.github.io/.github/workflows/Documentation.yml

Line 7 in cc3d1b0

- cron: '15 2 * * *' # 2:15 AM UTC every day

Are there any remaining bits to consider?

[icweaver]

Nvm about the redirects, it was causing MultiDocumenter to complain, so I undid it. I'm actually thinking of taking a page out of ArviZ.jl's book now and just turning off gh-pages so that the MultiDocumenter version is the only source for docs. I imagine that might help with SEO too, but wanted to check in about how folks feel about this first

Following up on this, I've tagged people that seem to still be active in JuliaAstro and are maintaining their respective packages. If you've been tagged, could you respond with whether you are fine or not with us making this change, i.e., transitioning from https://juliaastro.org/<yourpackage.jl>/ that is hosted in your repo, to https://juliaastro.org/<yourpackage>/ that is hosted on the main JuliaAstro MulitDocumenter site?

Apologies if I missed someone, or if you are no longer the point person for maintenance!

[icweaver]

Howdy again all, just putting a gentle ping here about the above. I'll also be sure to add an item in our community call tomorrow if folks would like to discuss anything there

[cgarling]

I feel like I've seen you mention this in a few repos but I'm not clear on what the benefit of this change is. It would be nice to have a short (~paragraph) pitch that explains the pros and cons of this approach, both for current maintainers to review but also for the future to recall why the choice was made.

[icweaver]

Hey Chris, good idea! Copying an updated tl;dr here from slack:

Would it make sense to disable our repo-specific gh-page doc deploys so that our main site would essentially serve as our "centralized source of truth"? I'm looking at SciML and ArviZ.jl for prior art. As a consequence:

• Links like https://juliaastro.org/AstroAngles.jl/stable/ would be deprecated in favor of https://juliaastro.org/AstroAngles/stable/
• Since our main site pulls doc updates once a day, it seems that there would be about that amount of lag time for package-specific doc changes to reflect on the JuliaAstro site
• Sample package with these changes here: https://github.com/JuliaAstro/Spectra.jl

I think these changes seem reasonable for the benefit of having a unified source for finding documentation under the JuliaAstro umbrella, but wanted to get other folks' feedback too before making any changes

[cgarling]

Thanks for the overview! It looks like arviz is also using a cron job to rebuild the documentation once per day (see here).

Are JuliaAstro/AstroAngles.jl#15 and JuliaAstro/Spectra.jl#18 your example implementations for "client" packages?

I personally find having ~instant docs updates on pushes to the main branch to be a very nice feature, particularly when diagnosing CI-specific docs issues (deploying push previews also helps with this for open PRs). To clarify, is your idea to stop building documentation on the "client" packages altogether and only build and deploy the docs on the MultiDocumenter site with the cron job?

I wonder if there's a middle ground, where "stable" docs (for tagged and released versions) link to the JuliaAstro site, but we still build repo-specific docs on pushes to the main branch for the "dev" docs. In this case, we would still be building docs on each repo, but we would change the "stable" links in the readmes to direct to the main JuliaAstro docs. Arviz seems to still be building documentation and hosting gh-pages branches on their individual repos (for example, see here) so that the "dev" docs are still built and accessible (see here).

I am hesitant to give up the automatic building of the "dev" docs entirely, but I have no substantive issue with changing the "stable" docs to link to the JuliaAstro hosted docs.

[icweaver]

Sweet! Yep, nail on the head. Thanks for taking the time to give your feedback, Chris. I hadn't considered the middle ground option and think it's a really good point. I'd be all for having the stable docs point to the JuliaAstro site while keeping the dev docs built on a per-repo basis for testing and such

[icweaver]

Mostly recording this here for future self: Following up from our community meeting, it seemed like the middle ground approach suggested by Chris was received positively. Below is a running list of my attempts and failures to try and make this happen, using Spectra.jl as our playground:

• Attempt: Only deploy from #gh-pages/dev folder
• Result: Does not seem to be supported on GitHub's side. Can only deploy from / and /docs/

---

• Attempt: Move dev build over to #gh-pages/docs and deploy from there (JuliaAstro/Spectra.jl@97f2913)
• Result: Only dev docs deploy now as desired, but stable and versioned docs no longer work from dropdown since they are not in the #gh-pages/docs folder. Maybe that's fine since the dev docs are only meant to serve as an "internal tool". Rebuilding the MultiDocumenter site to see how this shakes out
• Update: Yep, breaks things as expected. Since this caused dev docs for Spectra.jl to now live under docs, trying to select it from the sidebar dropdown does not work (i.e., https://juliaastro.org/Spectra/dev/ does not exist under this scheme, while https://juliaastro.org/Spectra/docs/dev/ does). Not a fan, will try something else

---

• Attempt: Just drop dev docs from the sidebar
• Result: Seems to work pretty well!
  • Stable docs in JuliaAstro site: https://juliaastro.org/Spectra
  • Dev docs as usual: https://juliaastro.org/Spectra.jl/dev

[icweaver]

Okie doke, think I'm landing on a pretty decent approach here. @abhro @cgarling, following up on our discussion from yesterday, does this seem reasonable/minimally disruptive to you? Essentially, just the tagged releases for a given JuliaAstro package would show up in the sidebar on the JuliaAstro site, and a dedicated dev docs link would still exist as usual.

Example:

• Stable docs in JuliaAstro site: https://juliaastro.org/Spectra
• Dev docs: https://juliaastro.org/Spectra.jl/dev, and also via PR previews as usual

This would be accomplished by just passing an explicit versions keyword to deploydocs on a per-repo basis. What do y'all think?

There is still the issue of having duplicate dev docs built on the JuliaAstro site (e.g., https://juliaastro.org/Spectra/dev/), but maybe that's not so bad if we don't explicitly point to it

[abhro]

Maybe we can try it as some sort of command-line argument to docs/make.jl? We can pass in something like --dev-only when the docs are being built from Spectra.jl's own workflows; but when MultiDocumenter builds it, the default docs/make.jl will try and build all versions except dev?

For having a duplicate of dev on the main juliaastro.org website, we can also probably add to the actions that just deletes the dev folders in all of the built docs.

[icweaver]

Thanks Abhro! Here's a go at the dev-only option. Since the dev and stable docs both exist in the #gh-pages branch of Spectra.jl, it's not super clear to me what restricting Spectra.jl's make.jl to just the dev docs exactly buys us over the current scheme (just removing dev from the version selector). AFAICT, MultiDocumenter has no control over Spectra.jl's make.jl and just pulls whaterver is in Spectra.jl's #gh-pages branch, but I could be misunderstanding what you wanted to try here

Deleting the dev folder on the JuliaAstro/MultiDocumenter.jl side seems staightforward enough though:

• https://juliaastro.org/Spectra/stable/ (works)
• https://juliaastro.org/Spectra/dev/ (no longer available, as desired)

I just did it manually here to try things out, and will start looking into automating this with an action next

Update: Ok, should be up now. Can generalize this to all JuliaAstro packages next after updating their version selectors

[abhro]

AFAICT, MultiDocumenter has no control over Spectra.jl's make.jl and just pulls whaterver is in Spectra.jl's #gh-pages branch, but I could be misunderstanding what you wanted to try here

Hmm. I think I may have misunderstood how MultiDocumenter does its build process. I thought it ran an include("<package_clone_path>/docs/make.jl") and then copied all the built docs from there (like how this repository used to do builds before MultiDocumenter). And if we made the default behavior for every docs/make.jl to not produce a dev version, that would be what the MultiDocumenter build would end up with.

And then, for the Spectra.jl website itself, it would only have a single version in its own gh-pages branch for dev.

But if MultiDocumenter only does a git pull from the Spectra.jl's gh-pages branch, then I agree that this doesn't really give us anything, since we'd need to have all versions of the docs (including dev) built within Spectra.jl anyway.

[icweaver]

Ah, gotcha. Yea, my understanding is that <package_clone_path> is just the corresponding #gh-pages branch by default, e.g., here for the MultiDocRefs that we are using, but I'll defer to @asinghvi17 if that is not accurate?

[asinghvi17]

MultiDocumenter only sees built HTML from gh-pages, it doesn't even have any (native) control over versioning.

[asinghvi17]

You can pre clone to a known path and edit the contents of the repo though, and just point MultiDocRef there for the clonedir. Then it shouldn't clone...

[icweaver]

Thanks for confirming, Anshul! It seems like just

i) removing the dev docs, and
ii) removing the dev version selector

is sufficient enough for our purposes then. I've moved the first step to the make file instead of a GitHub action now, and can start applying the second step to our other JuliaAstro packages if that sounds good to folks?

[icweaver]

Dope, will start making moves