Is it possible to apply cluster-robust standard errors to correlations?

[jmgirard]

I'm still new to the sandwich estimator and other population-average methods, but it seems like enabling the ability to combine these with correlation coefficients would be a nice feature for the package. But, if for some reason that's not possible, it would also be helpful for me to know that. Thanks!

[DominiqueMakowski]

@jmgirard I'm not sure I have a good idea of what you mean, how you would see that working?

[jmgirard]

An example would be if you have data where the observations may not be fully independent. Could we correct for the SEs being too small using something like the sandwich estimator (as you can in a linear model)?

[DominiqueMakowski]

Do you have an example on how it would work with say cor.test?

(currently correlation() runs sequentially a given cor_test() method, so I'm not sure whether your feature would involve tweaking the existing cor_test methods or implementing a new one)

[bwiernik]

This is an interesting and difficult problem. There are several things to consider here.

First, outside of some work specifically applicable to the context of dyads (e.g., https://doi.org/10.1037/0033-2909.118.3.430) there isn't really much theoretical work on cluster-robust variance estimation for Pearson correlations. This is admittedly a difficult problem to search for because in the robust inference literature "correlation" usually refers to the error correlation structure.

Second, derivations of the standard error of the correlation coefficient are typically done using the elements of the correlation matrix itself (e.g., https://doi.org/10.1016/0024-3795(85)90191-0), rather than the raw data matrix, so methods for cluster-robust SEs from OLS or similar models aren't straightforward to apply to these correlation SEs.

Third, adjusted standard errors for the correlation coefficient aren't particularly useful because they can't be used directly to construct confidence intervals due to the [-1, 1] bounds. Adjusting the correlation SE would have a highly eccentric impact on the SE of the Fisher-z transformed correlation.

All that said, a better approach to cluster-robust inference for correlation coefficients is to do a cluster bootstrap to construct a confidence interval (e.g., https://www.r-bloggers.com/2013/01/the-cluster-bootstrap/)

[DominiqueMakowski]

I have no idea how to translate that to a code commit 😅

[jmgirard]

I could help create code to implement cluster bootstrapping. I was hoping that the sandwich estimator might work because that doesn't require you to specify the clustering explicitly (which is nice), but @bwiernik makes a good point about the boundedness of correlations. I was wondering if there was some way to maybe transform the r to z or something, adjust the SE, and then transform back (similar to what you do in meta-analysis), but it sounds like this hasn't really been explored yet.

[DominiqueMakowski]

I could help create code to implement cluster bootstrapping

By all means!

but it sounds like this hasn't really been explored yet.

feel free to try, we like to explore the push the boundaries in easystats :)

[jmgirard]

By all means!

Ok, I'll work on that. I actually have an experimental package that does this already (albeit in a different context). So it should go quickly. https://github.com/jmgirard/mlboot/blob/master/R/bootstrap.R

[bwiernik]

The issue with the Fisher-z interval is that it is only approximate and relies pretty heavily on the variables being iid normally distributed. That's by definition not the case if there is meaingful clustering. It's possible that a sandwich estimator with the Fisher z might perform okay, but given the eccentricity of this transformation, I'm not confident about that.

[jmgirard]

Cool, thanks for sharing your expertise @bwiernik. Let me know if you have any tips for the cluster bootstrap (or notice anything weird about my implementation).