One-sided bf_pointnull vs bf_rope with range 0 to Inf

[jackdolgin]

Hi, first thanks for this wonderful package. I'm not sure if this question is directly about bayestestR vs. being a slightly more broad conceptual question, but with regard to bayesfactor_parametersand its sister functions, I'm wondering about the difference between setting a null point vs. a null interval. Specifically, I'm running a brms fit and my experimental hypothesis is that the coefficient of interest will be negative. I wasn't sure whether it would be just as appropriate to run bayesfactor_rope(myfit, null = c(0, Inf)) as it would be to run bayesfactor_pointnull(myfit, direction = 'left'). I assume it wouldn't be, because I obtain quite different results when running the two. I suppose I'm specifically asking why setting a ROPE range over 0 and the entire positive range yields a different bf than running a one-sided bf test with null of simply 0. Thanks in advance!

[mattansb]

This is a great questions, and I think it has to do with one of the differences between BFs and p-values in hypothesis testing; While for p-values we have a null hypothesis (e.g, b=0) and the alternative is implied from it (b≠0), for BFs the specification of the two hypotheses is completely independent of one another.

E.g., we can have any of the following sets of hypotheses we might want to compare:

• H_A: b = 0; H_B: 1 < b < 2; (do not complement each other)
• H_A: -1 < b < +1; H_B: -2 < b < 0; (have an overlap)
• Etc...

In practice, bayestestR dose constrain the hypotheses to (1) not overlap and (2) no have gaps between them (as we will see).

So what do the two specifications you made do?

The default for bayesfactor_parameters() is:

• H₀ b=0
• H_A b≠0 (what ever the null hypotheses is not)

Setting direction = 'left' only affects the alterative, so we now have:

• H₀ b=0
• H_A b<0

We can see this in the footer notes in the output - the null is only 0.

library(bayestestR)

prior <- rt(4e3, df = 3)
posterior <- rnorm(4e3, mean = -.1, sd = .4)

(bf1 <- bayesfactor_pointnull(posterior, prior, direction = 'left'))
#> Bayes Factor (Savage-Dickey density ratio)
#> 
#> BF   
#> -----
#> 0.459
#> 
#> * Evidence Against The Null: 0
#> *                 Direction: Left-Sided test

When setting null = c(0, Inf) two things happen:
The null is now:

• H₀ b>0

But also, because how we've implemented to BFs, the alternative cannot overlap, so anything that is not the null is now:

• H_A b<0

Again, we can see this in the footer of the output:

(bf2 <- bayesfactor_rope(posterior, prior, null = c(0, Inf)))
#> Bayes Factor (Null-Interval)
#> 
#> BF  
#> ----
#> 1.53
#> 
#> * Evidence Against The Null: [0.000, Inf]

Plotting the two can also help:

(The marked / shaded areas are the null - everything else is the alterative):

plot(bf1)

plot(bf2)

So we can see that the alternative in both cases is b<0, but the nulls are different!

Hope this helps!

[jackdolgin]

That makes sense! Thank you very much. I can see that my research question aligns with bf1 because indeed I'm interested in 0 as the null rather than a whole range as the null. On a sort of related note, even though I think we're in agreement about h0 equaling 0 when the alternative hypothesis is b < 0, if I opted for ROPE as my dv instead of bf, would it then be appropriate to set the ROPE range as 0-Inf (since by definition a range is required, not a null point)? Thanks again.

[mattansb]

If I understand your question, you asking about ROPE tests, and whether it is okay to set "all positive values" as you ROPE?

If the ROPE is all positive values, this is a test for the probability that the parameter/estimate is negative. In that case, this will essentially give something close to the p_d (or 1-p_d):

library(bayestestR)

x <- rnorm(1e4, 0.3)

pd(x)
#> Probability of Direction: 0.61

rope(x, range = c(0, Inf))
#> # Proportion of samples inside the ROPE [0.00, Inf]:
#> 
#> inside ROPE
#> -----------
#> 61.44 %

It might be more informative to set the rope to include all positive values + small negative values that are not 'interesting':

rope(x, range = c(-0.5, Inf))
#> # Proportion of samples inside the ROPE [-0.50, Inf]:
#> 
#> inside ROPE
#> -----------
#> 79.92 %

^{Created on 2021-10-10 by the reprex package (v2.0.1)}

[jackdolgin]

That makes sense! I had considered just sticking with pd, since indeed I'm more interested in 'existence' than 'significance' in this case. One of the reasons I wanted to go with ROPE is because, from what I've read, ROPE can identify evidence in favor of the null hypothesis but pd can't. I'm basing that off of what I've read from the easystats authors like yourself, though I might have misinterpreted the text. For example, on page 4 of https://doi.org/10.21105/joss.01541, "If the ROPE completely covers the HDI, i.e., all most credible values of a parameter are inside the ROPE, the null hypothesis is accepted."

[mattansb]

p_d and ROPE are both doing essentially the same under the hood: They give the probability over some range of values.

A small p_d cannot be used to support the point null - this is true.

But you are not looking for a small p_d, and you are not testing a point null - you are looking for a large p_d to show that most of the posterior is not positive.

[jackdolgin]

Gotcha. Much appreciated!