This package fails PkgEval

[lgoettgens]

See https://s3.amazonaws.com/julialang-reports/nanosoldier/pkgeval/by_date/2023-10/15/DocumenterCitations.primary.log

I think the problem is that Pkg is used for the documentation/doctests somewhere, but is not declared a dependency/test-dependency/docs-dependency.

[goerz]

I can reproduce it locally, but this doesn't seem actionable.

Slack suggested it might a bug in Nightly, but julia --project bin/test_package.jl --name=DocumenterCitations --path=/home/goerz/Documents/Programming/DocumenterCitations.jl --julia=1.9.3 also fails.

Adding Pkg to both Project.toml and test/Project.toml doesn't do anything.

My suspicion is that this might be the issue described in JuliaDocs/DocumentationGenerator.jl#200

I don't really know what weird things PkgEval does internally, so this kinda seems like a problem for the JuliaHub people to solve on their end.

I'm going to tag this as invalid, but keep it open (at least for a while).

If @pfitzseb or @mortenpi tell me what I have to do to make PkgEval work, I'll be happy to do that ;-)

[pfitzseb]

cc @maleadt

[goerz]

It's gotta be some interaction between how Documenter evaluates @eval blocks and PkgEvals sandboxing

[goerz]

One thing I could do is to wrap that @eval block into a try/catch. It's not essential for testing (it just displays the version badge on the documentation).

While I was trying that, I found that the --path option to bin/test_package.jl apparently doesn't work, or at least PkgEval isn't picking up on my modifications to the package source. So when I said that adding Pkg to Project.toml doesn't help, that may not have been true 🤷

[maleadt]

I don't really know what weird things PkgEval does internally, so this kinda seems like a problem for the JuliaHub people to solve on their end.

PkgEval is a Julia community project, and is not related to JuliaHub. The code is public: https://github.com/JuliaCI/PkgEval.jl
You can see that it just uses Pkg.test, so it doesn't do anything weird internally: https://github.com/JuliaCI/PkgEval.jl/blob/2831046f1407eda2b403824b93cd1e08b25d9d70/src/evaluate.jl#L481

[goerz]

Fair enough... Indeed, just doing Pkg.add("DocumenterCitations"); Pkg.test("DocumenterCitations") in a temporary environment also fails with the same error. I still don't understand why, but at least that's something that's easier to test with.

[goerz]

Well, it looks like just adding Pkg to the [deps] in test/Project.toml does solve the problem after all, as far as Pkg.test is concerned. I'll make sure that'll be there in the next release, and then hopefully PkgEval will be able to run through.

[goerz]

I'll leave this issue open until after the next release, when I've confirmed that the PkgEval badge has turned green on https://juliahub.com/ui/Packages/General/DocumenterCitations

[goerz]

Thanks to the discussion at https://discourse.julialang.org/t/how-does-the-pkgeval-badge-work-on-juliahub/105662, I've added a workflow in #66 that checks the PkgEval status as part of the release process.