Skip to content

Commit 41c6f03

Browse files
sethaxensethaxen
authored
Add documenter interlinks and citations (#73)
* Remove outdated docs of Distributions integration * Add DocumenterInterLinks as docs dependency * Copy example to `psis` docstring * Link out to MCMCDiagnosticTools docs * Filter out warnings line in doctests * Add MCMCDiagnosticTools as docs dep * Increment patch number * Add references file * Add bibliography to docs * Reference bibliography entry everywhere * Add custom CSS for citations * Match Distributions test dependency bound
1 parent 9acd5a5 commit 41c6f03

File tree

9 files changed

+145
-26
lines changed

9 files changed

+145
-26
lines changed

‎Project.toml

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
name = "PSIS"
22
uuid = "ce719bf2-d5d0-4fb9-925d-10a81b42ad04"
33
authors = ["Seth Axen <[email protected]> and contributors"]
4-
version = "0.9.6"
4+
version = "0.9.7"
55

66
[deps]
77
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"

‎docs/Project.toml

+7-1
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,17 @@
11
[deps]
22
Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
33
Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
4+
DocumenterCitations = "daee34ce-89f3-4625-b898-19384cb65244"
5+
DocumenterInterLinks = "d12716ef-a0f6-4df4-a9f1-a5a34e75c656"
6+
MCMCDiagnosticTools = "be115224-59cd-429b-ad48-344e309966f0"
47
PSIS = "ce719bf2-d5d0-4fb9-925d-10a81b42ad04"
58
Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
69
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
710

811
[compat]
9-
Distributions = "0.23.2, 0.24, 0.25"
12+
Distributions = "0.25.81"
1013
Documenter = "1"
14+
DocumenterCitations = "1.2"
15+
DocumenterInterLinks = "1"
16+
MCMCDiagnosticTools = "0.3.2"
1117
Plots = "1.10.1"

‎docs/make.jl

+15-1
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,35 @@
11
using Distributions, Plots, PSIS
22
using Documenter
3+
using DocumenterCitations
4+
using DocumenterInterLinks
35

46
DocMeta.setdocmeta!(PSIS, :DocTestSetup, :(using PSIS); recursive=true)
57

8+
bib = CitationBibliography(joinpath(@__DIR__, "src", "references.bib"); style=:numeric)
9+
10+
links = InterLinks(
11+
"MCMCDiagnosticTools" => "https://julia.arviz.org/MCMCDiagnosticTools/stable/"
12+
)
13+
614
makedocs(;
715
modules=[PSIS],
816
authors="Seth Axen <[email protected]> and contributors",
917
repo=Remotes.GitHub("arviz-devs", "PSIS.jl"),
1018
sitename="PSIS.jl",
11-
format=Documenter.HTML(; prettyurls=get(ENV, "CI", "false") == "true", assets=String[]),
19+
format=Documenter.HTML(;
20+
prettyurls=get(ENV, "CI", "false") == "true",
21+
assets=[joinpath("assets", "citations.css")],
22+
),
1223
pages=[
1324
"Home" => "index.md",
1425
"Plotting" => "plotting.md",
1526
"API" => "api.md",
1627
"Internal" => "internal.md",
28+
"References" => "references.md",
1729
],
30+
doctestfilters=[r"└.*"], # ignore locations in warning messages
1831
warnonly=:missing_docs,
32+
plugins=[bib, links],
1933
)
2034

2135
deploydocs(; repo="github.com/arviz-devs/PSIS.jl.git", devbranch="main")

‎docs/src/assets/citations.css

+29
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
/* Adapted from DocumenterCitations.jl's docs */
2+
.citation dl {
3+
display: grid;
4+
grid-template-columns: max-content auto;
5+
}
6+
7+
.citation dt {
8+
grid-column-start: 1;
9+
}
10+
11+
.citation dd {
12+
grid-column-start: 2;
13+
margin-bottom: 0.75em;
14+
}
15+
16+
.citation ul {
17+
padding: 0 0 2.25em 0;
18+
margin: 0;
19+
list-style: none !important;
20+
}
21+
22+
.citation ul li {
23+
text-indent: -2.25em;
24+
margin: 0.33em 0.5em 0.5em 2.25em;
25+
}
26+
27+
.citation ol li {
28+
padding-left: 0.75em;
29+
}

‎docs/src/index.md

+1-5
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ CurrentModule = PSIS
44

55
# PSIS
66

7-
PSIS.jl implements the Pareto smoothed importance sampling (PSIS) algorithm from [^VehtariSimpson2021].
7+
PSIS.jl implements the Pareto smoothed importance sampling (PSIS) algorithm from [VehtariSimpson2021](@citet).
88

99
Given a set of importance weights used in some estimator, PSIS both improves the reliability of the estimates by smoothing the importance weights and acts as a diagnostic of the reliability of the estimates.
1010

@@ -35,7 +35,3 @@ As indicated by the warnings, this is a poor choice of a proposal distribution,
3535

3636
When running PSIS with many parameters, it is useful to plot the Pareto shape values to diagnose convergence.
3737
See [Plotting PSIS results](@ref) for examples.
38-
39-
[^VehtariSimpson2021]: Vehtari A, Simpson D, Gelman A, Yao Y, Gabry J. (2021).
40-
Pareto smoothed importance sampling.
41-
[arXiv:1507.02646v7](https://arxiv.org/abs/1507.02646v7) [stat.CO]

‎docs/src/references.bib

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
@article{VehtariSimpson2021,
2+
title = {Pareto Smoothed Importance Sampling},
3+
author = {Vehtari, Aki and Simpson, Daniel and Gelman, Andrew and Yao, Yuling and Gabry, Jonah},
4+
year = {2024},
5+
journal = {Journal of Machine Learning Research},
6+
volume = {25},
7+
number = {72},
8+
pages = {1--58},
9+
url = {http://jmlr.org/papers/v25/19-556.html},
10+
eprint = {1507.02646},
11+
archiveprefix = {arXiv}
12+
}
13+
14+
@article{ZhangStephens2009,
15+
title = {A New and Efficient Estimation Method for the Generalized Pareto Distribution},
16+
author = {Zhang, Jin and Stephens, Michael A.},
17+
year = {2009},
18+
journal = {Technometrics : a journal of statistics for the physical, chemical, and engineering sciences},
19+
volume = {51},
20+
number = {3},
21+
pages = {316--325},
22+
publisher = {ASA Website},
23+
doi = {10.1198/tech.2009.08017},
24+
}

‎docs/src/references.md

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# References
2+
3+
```@bibliography
4+
```

‎src/core.jl

+58-8
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ Result of Pareto-smoothed importance sampling (PSIS) using [`psis`](@ref).
3838
3939
The `pareto_shape` parameter ``k=ξ`` of the generalized Pareto distribution `tail_dist` can
4040
be used to diagnose reliability and convergence of estimates using the importance weights
41-
[^VehtariSimpson2021].
41+
[VehtariSimpson2021](@citep).
4242
4343
- if ``k < \\frac{1}{3}``, importance sampling is stable, and importance sampling (IS) and
4444
PSIS both are reliable.
@@ -55,9 +55,9 @@ be used to diagnose reliability and convergence of estimates using the importanc
5555
5656
See [`PSISPlots.paretoshapeplot`](@ref) for a diagnostic plot.
5757
58-
[^VehtariSimpson2021]: Vehtari A, Simpson D, Gelman A, Yao Y, Gabry J. (2021).
59-
Pareto smoothed importance sampling.
60-
[arXiv:1507.02646v7](https://arxiv.org/abs/1507.02646v7) [stat.CO]
58+
# References
59+
60+
- [VehtariSimpson2021](@cite) Vehtari et al. JMLR 25:72 (2021).
6161
"""
6262
struct PSISResult{T,W<:AbstractArray{T},R,L,D}
6363
log_weights::W
@@ -177,7 +177,7 @@ _pad_right(s, nchars) = "$s" * " "^(nchars - length("$s"))
177177
psis(log_ratios, reff = 1.0; kwargs...) -> PSISResult
178178
psis!(log_ratios, reff = 1.0; kwargs...) -> PSISResult
179179
180-
Compute Pareto smoothed importance sampling (PSIS) log weights [^VehtariSimpson2021].
180+
Compute Pareto smoothed importance sampling (PSIS) log weights [VehtariSimpson2021](@citep).
181181
182182
While `psis` computes smoothed log weights out-of-place, `psis!` smooths them in-place.
183183
@@ -204,9 +204,59 @@ While `psis` computes smoothed log weights out-of-place, `psis!` smooths them in
204204
A warning is raised if the Pareto shape parameter ``k ≥ 0.7``. See [`PSISResult`](@ref) for
205205
details and [`PSISPlots.paretoshapeplot`](@ref) for a diagnostic plot.
206206
207-
[^VehtariSimpson2021]: Vehtari A, Simpson D, Gelman A, Yao Y, Gabry J. (2021).
208-
Pareto smoothed importance sampling.
209-
[arXiv:1507.02646v7](https://arxiv.org/abs/1507.02646v7) [stat.CO]
207+
# Examples
208+
209+
Here we smooth log importance ratios for importance sampling 30 isotropic Student
210+
``t``-distributed parameters using standard normal distributions as proposals.
211+
212+
```jldoctest psis; setup = :(using Random; Random.seed!(42))
213+
julia> using Distributions
214+
215+
julia> proposal, target = Normal(), TDist(7);
216+
217+
julia> x = rand(proposal, 1_000, 1, 30); # (ndraws, nchains, nparams)
218+
219+
julia> log_ratios = @. logpdf(target, x) - logpdf(proposal, x);
220+
221+
julia> result = psis(log_ratios)
222+
┌ Warning: 9 parameters had Pareto shape values 0.7 < k ≤ 1. Resulting importance sampling estimates are likely to be unstable.
223+
└ @ PSIS ~/.julia/packages/PSIS/...
224+
┌ Warning: 1 parameters had Pareto shape values k > 1. Corresponding importance sampling estimates are likely to be unstable and are unlikely to converge with additional samples.
225+
└ @ PSIS ~/.julia/packages/PSIS/...
226+
PSISResult with 1000 draws, 1 chains, and 30 parameters
227+
Pareto shape (k) diagnostic values:
228+
Count Min. ESS
229+
(-Inf, 0.5] good 7 (23.3%) 959
230+
(0.5, 0.7] okay 13 (43.3%) 938
231+
(0.7, 1] bad 9 (30.0%) ——
232+
(1, Inf) very bad 1 (3.3%) ——
233+
```
234+
235+
If the draws were generated using MCMC, we can compute the relative efficiency using
236+
[`MCMCDiagnosticTools.ess`](@extref).
237+
238+
```jldoctest psis
239+
julia> using MCMCDiagnosticTools
240+
241+
julia> reff = ess(log_ratios; kind=:basic, split_chains=1, relative=true);
242+
243+
julia> result = psis(log_ratios, reff)
244+
┌ Warning: 9 parameters had Pareto shape values 0.7 < k ≤ 1. Resulting importance sampling estimates are likely to be unstable.
245+
└ @ PSIS ~/.julia/packages/PSIS/...
246+
┌ Warning: 1 parameters had Pareto shape values k > 1. Corresponding importance sampling estimates are likely to be unstable and are unlikely to converge with additional samples.
247+
└ @ PSIS ~/.julia/packages/PSIS/...
248+
PSISResult with 1000 draws, 1 chains, and 30 parameters
249+
Pareto shape (k) diagnostic values:
250+
Count Min. ESS
251+
(-Inf, 0.5] good 9 (30.0%) 806
252+
(0.5, 0.7] okay 11 (36.7%) 842
253+
(0.7, 1] bad 9 (30.0%) ——
254+
(1, Inf) very bad 1 (3.3%) ——
255+
```
256+
257+
# References
258+
259+
- [VehtariSimpson2021](@cite) Vehtari et al. JMLR 25:72 (2021).
210260
"""
211261
psis, psis!
212262

‎src/generalized_pareto.jl

+6-10
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,6 @@
77
88
The generalized Pareto distribution.
99
10-
This is equivalent to `Distributions.GeneralizedPareto` and can be converted to one with
11-
`convert(Distributions.GeneralizedPareto, d)`.
12-
1310
# Constructor
1411
1512
GeneralizedPareto(μ, σ, k)
@@ -20,8 +17,8 @@ parameter ``σ`` and shape parameter ``k``.
2017
!!! note
2118
2219
The shape parameter ``k`` is equivalent to the commonly used shape parameter ``ξ``.
23-
This is the same parameterization used by [^VehtariSimpson2021] and is related to that
24-
used by [^ZhangStephens2009] as ``k \\mapsto -k``.
20+
This is the same parameterization used by [VehtariSimpson2021](@citet) and is related to
21+
that used by [ZhangStephens2009](@citet) as ``k \\mapsto -k``.
2522
"""
2623
struct GeneralizedPareto{T}
2724
μ::T
@@ -56,7 +53,7 @@ end
5653
5754
Fit a [`GeneralizedPareto`](@ref) with location `μ` to the data `x`.
5855
59-
The fit is performed using the Empirical Bayes method of [^ZhangStephens2009].
56+
The fit is performed using the Empirical Bayes method of [ZhangStephens2009](@citet).
6057
6158
# Keywords
6259
@@ -67,10 +64,9 @@ The fit is performed using the Empirical Bayes method of [^ZhangStephens2009].
6764
- `min_points::Int=30`: The minimum number of quadrature points to use when estimating the
6865
posterior mean of ``\\theta = \\frac{\\xi}{\\sigma}``.
6966
70-
[^ZhangStephens2009]: Jin Zhang & Michael A. Stephens (2009)
71-
A New and Efficient Estimation Method for the Generalized Pareto Distribution,
72-
Technometrics, 51:3, 316-325,
73-
DOI: [10.1198/tech.2009.08017](https://doi.org/10.1198/tech.2009.08017)
67+
# References
68+
69+
- [ZhangStephens2009](@cite) Zhang & Stephens, Technometrics 51:3 (2009)
7470
"""
7571
function fit_gpd(x::AbstractArray; prior_adjusted::Bool=true, kwargs...)
7672
tail_dist = fit_gpd_empiricalbayes(x; kwargs...)

0 commit comments

Comments
 (0)