Specifying correct priors for factors

[rbcavanaugh]

Question and context
First, I want to say thank you for developing such a fantastic package which is so well documented and accessible.

I just have a hopefully simple clarification question on an issue raised by the bayes factor vignette which states:

Notice that, though the prior estimate for all 3 pairwise contrasts is ~0, the scale or the HDI is much narrower for the prior of the setosa - versicolor contrast!

What happened???

This is caused by an inherent bias in the priors introduced by the effects coding (it’s even worse with the default treatment coding, because the prior for the intercept is usually drastically different from the effect’s parameters). And since it affects the priors, this bias will also bias the Bayes factors over / understating evidence for some contrasts over others!

The solution is to use orthonormal factor coding, a-la contr.orthonorm, which can either specify this factor coding per-factor:

Does it follow that, for situations where treatment coding is preferred (i.e. one would like to separately contrast two "experimental" conditions to a "control" condition), that another potential solution is to place the same prior on the intercept and effect's parameter?

Cheers -

[mattansb]

Hmm... Re-reading what I wrote, I think that can be better clarified. Let's try...

We need to keep in mind that Var(x±z) = Var(x) + Var(z), assuming that x and z are independent.
I will assume that the various prior distributions are independent (as this is easier + what is implemented in brms/stan by default).

Okay, let's see:

Treatment coding

If you are only interested in is just those two contrasts (comparing each group to the "control"), treatment coding would be fine - it is a reasonable parameterization of your problem / question.

However, if you wanted to compare the two experimental conditions to each other, this difference will have a different prior shape.

In trt coding we have a (intercept = control's mean) b1 (diff to group2) and b2 (diff to group3).
If we reconstruct the means:

• Control: a
• Exp1: a + b1
• Exp2: a + b2

The pairwise diffs are:

• Exp1 - Control = (a + b1) - (a) = b1, has Var(b1) on the prior.
• Exp2 - Control = (a + b2) - (a) = b2, has Var(b2) on the prior.
• Exp1 - Exp2 = (a + b1) - (a + b2) = b1 - b2, has Var(b1) + Var(b2) on the prior.

That is, the prior on {Exp1 - Exp2} is more spread out than the other two contrasts, leading to a bias in any prior-based metric - Bayes factors are esp sensitive to this, but to an extent so are the posteriors and any posterior-based metric.

(It can be shown that a similar bias happens with effects coding as well.)

Solutions for modelling factors

1. Using the orthonormal coding in the model fitting (the Rouder approach) and obtaining contrasts with emmeans.
2. Using index coding (the McElreath approach) - put priors directly on the means.
3. Treating the factor as a random grouping variable (the Kruschke approach) - thus putting a prior on the between-group variance instead of any specific differences.

You can read about the latter two and how to do them in brms in Solomon Kurz's great post.

[rbcavanaugh]

Fantastic - thank you for taking the time to write such a thorough response.