p-direction for circular data

[tardipede]

Hello!
I recently got stuck in how to calculate the p-direction (or an equivalent index of existence of an effect) of the difference between bivariate posterior samples (i.e. each treatment is described by 2 parameters that can be correlated).
As an example in this image, each posterior sample is plotted with a different color for each treatment (red and black), whereas the two parameters are the two axis (x & y).

Calculating the p-direction of the difference between each parameter is feasible, but in this case, even if it is obvious that the treatments occupy different parameter spaces, when taken individually the parameters do not show an effect (and p-direction are quite low).

My idea to obtain a "general" index of existence that consider both parameters and an eventual correlation between them is to check the angles of vectors connecting points (see image below) in the posterior (given an adequate posterior size/ESS/convergence) to have a vector of data suitable for p-direction calculation.

By using those "posterior" angles I could then test the existence of an effect (difference) between my treatments. My main doubt is however how to define the absence of an effect (with standard univariate variables it is 0) as the "0" angle is not equivalent to the absence of effect.

I tried to first compute the median angle, and then "cut" the circle in half perpendicularly to the median angle (black arrow in the figure below). By calculating the proportion of angles lying in the median half of the circle (divided by the red line in the figure below) over all the angles I could obtain an index of existence equivalent to p-direction for circular data.

I am scratching my head for a while to understand if it is an appropriate and meaningful way to test the presence of an effect between two groups described by two potentially correlated parameters and I´m kinda stuck, so any input/idea/criticism is very welcome!

Thanks,
Matteo

[bwiernik]

I wouldn't use the angle approach here. Instead, I would calculate the multivariate distance between the two distributions, such as using Mahalanobis' D

[tardipede]

I tried to figure out how to use Mahalanobis´ D but as far as I know it measures the distance between a point and a distribution, and here I have two distributions.

One alternative I had in mind was to measure the overlapping between the two distributions, but I liked the idea of having a p-direction that can be converted to a p-value and be "appealing" or at least more understandable to frequentists.

[mattansb]

If you are looking for a p-value-like measure... Why not use a p-value? ^_^

Make some data

d1 <- MASS::mvrnorm(30, c(.0, .0), matrix(c(1, 0.5, 0.5, 1), 2))
d2 <- MASS::mvrnorm(30, c(.1, .1), matrix(c(1, 0.5, 0.5, 1), 2))

data <- rbind(d1, d2) |> 
  as.data.frame() |> 
  transform(g = rep(letters[1:2], each = 30))

Multivarite model

mod1 <- lm(cbind(V1, V2) ~ g,
          data = data)

emmeans::emmeans(mod1, ~ g + rep.meas) |> 
  emmeans::mvcontrast("pairwise", mult.name = "rep.meas")
#>  contrast T.square df1 df2 F.ratio p.value
#>  a - b       1.073   2  57   0.527  0.5932
#> 
#> P value adjustment: sidak

Same data, but with HLM:

data_long <- data |> 
  transform(ID = 1:nrow(data)) |> 
  datawizard::data_to_long(select = c("V1", "V2"))

mod2 <- lme4::lmer(value ~ name * g + (1 | ID),
                  data = data_long)

# similiar results
emmeans::emmeans(mod2, ~ g + name) |> 
  emmeans::mvcontrast("pairwise", mult.name = "name")
#>  contrast T.square df1      df2 F.ratio p.value
#>  a - b       1.025   2 90.98864   0.507  0.6040
#> 
#> P value adjustment: sidak

^{Created on 2022-08-31 by the reprex package (v2.0.1)}

[bwiernik]

I tried to figure out how to use Mahalanobis´ D but as far as I know it measures the distance between a point and a distribution, and here I have two distributions.

It's a distance between 2 points (generally the means of the two groups), like a Euclidean distance or Cohen's d. It accounts for multiple dimensions and their correlations.

For your purposes, describing the mean differences is probably sufficient (and has advantage that there are well-defined significance tests for null or equivalence testing with Mahalanobis D). If you really want to describe the differences between the whole distribution (tails, etc), you could compute the D between each quantile of the distributions and average them.

Other options are to compute a QQ correlation between the two distributions (if the shape is all that matters) or to compute the average pointwise distance of each data point in one distribution to its nearest neighbor in the other. The latter is related to the Hopkins statistic often used in clustering (​Hopkins, B. & Skellam, J. G. A new method for determining the type of distribution of plant individuals. Ann. Bot. 18, 213–227 (1954).)