Matched-pairs rank biserial correlation for the wilcoxon signed-rank test

[rhazn]

Hello!

Thank you for creating this great tool :). I have difficulty parsing the results from a Wilcoxon signed-rank test using Pingouin and would love to hear if I am misunderstanding something or if I should send a patch.

From my understanding, the matched-pairs rank biserial correlation according to Kerby 2014 is calculated as the rank sum difference between favorable and unfavorable evidence for the Wilcoxon signed-rank test (see for example Table 2 on p. 6). This means wether or not a difference in value is considered favorable or unfavorable should depend on the hypothesis.

If I interpret the calculation in Pingouin correctly (from https://github.com/raphaelvallat/pingouin/blob/4e24aa730f70dcc1afea3efd929d31c3af004f2d/src/pingouin/nonparametric.py#L483C21-L483C74), differences larger than zero are always considered favorable evidence.

This is also shown in the docs (https://pingouin-stats.org/build/html/generated/pingouin.wilcoxon.html). The same test with different alternatives has different CLES (because for alternative of less it is calculated as 1 - CL), but the same RBC. Should the different alternative not change which evidence is considered favorable for the hypothesis?

From the docs:

pg.wilcoxon(x, y, alternative='greater')
          W-val alternative     p-val       RBC      CLES
Wilcoxon   20.5     greater  0.876244 -0.378788  0.395833

pg.wilcoxon(x, y, alternative='less')
          W-val alternative     p-val       RBC      CLES
Wilcoxon   20.5        less  0.142883 -0.378788  0.604167

In fact, there was a recent issue with the mwu test that seems to mirror this issue, even though of course it is not the same test: #417.

I am unsure if my understanding of the RBC for the Wilcoxon signed-rank test is flawed or if this is an issue with Pingouin?

[raphaelvallat]

Hi @rhazn,

Thanks for flagging this. I believe you are correct and the calculation of the RBC should be updated to take into account the hypothesis. At least, Kerby (2014) states that:

In words, the formula states that the nonparametric correlation equals the simple difference between the proportion of favorable and unfavorable evidence; in the case of the Mann-Whitney U test, the evidence consists of pairs.
An advantage of the simple difference formula is that it expresses one effect size (the rank-biserial correlation) in terms of another easily understood measure of effect size (the common language effect size). The other formulas for the rank-biserial express it in terms that are less easily interpreted. A second advantage of the simple difference formula is that it gives meaning to the direction of the sign. A positive correlation means that the data were in the predicted direction; a negative correlation, that they were against the predicted direction.

Feel free to submit a PR if you can. Thanks again.