fwildclusterboot 0.14.3

• Fixes a bug with CI inversion when r was set to be close to the estimated parameter, in which case the confidence interval inversion failed. See #138. Thanks to Achim Zeileis & team for raising this issue!
• This might lead to insubstantial differences in computed bootstrap confidence intervals compared to prior versions - the reason is that starting values for the root finding procedure used in inverting the CIs might have changed.

v0.14

fwildclusterboot 0.14

Performance

Version 0.14 ...

• sparsifies the "fast and reliable" bootstraps - bootstrap types 31, 33, 13 (which leads to good speed gains for problems with high dimensional fixed effects)
• allows to project out cluster fixed effects when running the "fast and reliable" algorithms "11" and "31"
• computes the generalized inverse pinv via rcpp eigen instead of MASS::ginv() whenever Matrix::solve() fails (thanks to @kylebutts)
• unlocks parallelization (nthreads was internally set to 1 for some reason)

Here is a performance benchmark for the "31" bootstrap between version 0.13 and 0.14: the runtime of the bootstrap decreases from 150s to around 10 seconds!

library(fwildclusterboot)
library(fixest)

df <- fwildclusterboot:::create_data(
  N = 1000000,
  N_G1 = 50, 
  icc1 = 0.2, 
  N_G2 = 50, 
  icc2 = 0.5, 
  numb_fe1 = 100, 
  numb_fe2 = 100, 
  seed = 123
)

nrow(df)
# [1] 1000000
sapply(df[, c("group_id1", "Q1_immigration", "Q2_defense")], function(x) length(unique(x)))
# group_id1 Q1_immigration     Q2_defense 
# 50            100            100 

fit <- feols(
  proposition_vote ~ treatment | group_id1 + Q1_immigration + Q2_defense, 
  data = df
)

# fwildclusterboot 0.13: 
pracma::tic()
boottest(
  fit, 
  param = ~ treatment,
  B = 9999, 
  clustid = ~group_id1, 
  bootstrap_type = "31"
)
pracma::toc()
# elapsed time is 158.550000 seconds 

# fwildclusterboot 0.14: 
pracma::tic()
boottest(
  fit, 
  param = ~ treatment,
  B = 9999, 
  clustid = ~group_id1, 
  bootstrap_type = "31"
)
pracma::toc()
# elapsed time is 11.460000 seconds 

Breaking Changes

• the print.boottest() and print.mboottest() method have been deprecated, as both did not really do anything that the summary() method would not do. Note: this was a bad decision, and I will revert it in version 0.14.1.
• Bugfix: boottest() should never have run with fixest::feols() and
  varying slopes syntax via var1[var2]. Unfortunately it did for the heteroskedastic bootstrap - it's a bug. I am very sorry if you are affected by this! This version adds an error message for this case.

rOpenSci Review feedback link

• update docs:
  • add a vignette on wild bootstrap concepts (wild bootstrap 101)
  • better explanation of plot method in docs and vignette
  • some guidelines on how to turn messages and warnings off
• reorganization of ropensci ssr tags into code
• it is now possible to interrupt rcpp loops

Misc

• throws a clear error message when the subcluster bootstrap is tried for the fast and reliable algos (currently not supported)
• bumps the required WildBootTests.jl version to 0.9.7 (version 0.9.6 contained a small bug when fixed effects where used within the bootstrap via the fe argument of boottest().

v0.13

Potentially Breaking Changes:

• boottest(), mboottest() and boot_aggregate()no longer have a dedicated seed argument. From version 0.13, reproducibility of results can only be controlled by setting a global seed via drqng::dqset.seed() and set.seed(). For more context, see the discussion below.

• As one consequence, results produced via old versions of fwildlcusterboot are no longer exactly reproducible. For more background on this change, see the changelog.

• When the bootstrap is run via engine = "WildBootTests.jl", the bootstrapped t-statistics and the original t-statistic are now returned as vectors (to align with the results from other enginges). Previously, they were returned as matrices.

Other Changes:

• boottest() receives a new argument, sampling, which controls if random numbers are drawn via functions from base or the dqrng package.
• Some code refactoring. All bootstrap algorithms and their associated files have been renamed (e.g. boot_algo2.R is not called boot_algo_fastnwild.R, etc.).
• Much nicer error and message formatting, via rlang::abort(), warn() and inform(). rlang is added as a dependency.

fwildclusterboot v0.12.3

• Bump required version of WildBootTests.jl to 0.8.3

fwildclusterboot v0.12 (CRAN release)

fwildclusterboot 0.12

This is the first CRAN release since version 0.9. It comes with a set of new features, but also potentially breaking changes. This section summarizes all developments since version 0.9.

Potentially breaking changes:

• boottest()'s function argument boot_algo has been renamed to engine
• the setBoottest_boot_algo() function was renamed to setBoottest_engine()

Bug fixes and internal changes

• When a multi-parameter hypothesis of the form R beta = r was tested, the heteroskedastic wild bootstrap would nevertheless always test
  "beta_k = 0" vs "beta_k != 0", with "beta_k = param". I am sorry for that bug!
• The Matrix.utils has been removed from CRAN - it has been replaced by custom functions for internal use.

New features and Improvements

• A new function argument has been added - bootstrap_type. In combination with the impose_null function argument, it allows to choose between different cluster bootstrap types - WCx11, WCx13, WCx31, WCx33. For more details on these methods, see the working paper by MacKinnon, Nielsen & Webb (2022). Currently, these new bootstrap types only compute p-values. Adding support for confidence intervals is work in progress.
• A boot_aggregate() method now supports the aggregation of coefficients in staggered difference-in-differences following the methods by Sun & Abraham (2021, Journal of Econometrics) in combination with the sunab() function from fixest. Essentially, boot_aggregate() is a copy of aggregate.fixest: the only difference is that inference is powered by a wild bootstrap.
• The heteroskedastic bootstrap is now significantly faster, and WCR21 and WCR31 versions are now supported (i.e. HC2 and HC3 'imposed' on the bootstrap dgp.)

v0.11.2

• fixes a bug for new bootstrap variants and engine = "R" when a fixed effects is specified via fixest::feols()
• drops the dependency on Matrix.utils as the package might be taken off CRAN
• add dependency on summclust, which is now available on CRAN

Release of versions 0.10 and 0.11

fwildclusterboot 0.11

• This release introduces new wild cluster bootstrap variants as described in MacKinnon, Nielsen & Webb (2022). The implementation is still quite bare-bone: it only allows to test hypotheses of the form $\beta_k = 0$ vs $\beta_k \neq 0$, does not allow for regression weights or fixed effects, and further does not compute confidence intervals.

You can run one of the 'new' variants - e.g. the "WCR13", by specifying the boot_algo function argument accordingly:

boottest(
  lm_fit, 
  param = ~treatment, 
  clustid = ~group_id1,
  B = 9999, 
  impose_null = TRUE,
  boot_algo = "WCR13"
)

fwildclusterboot 0.10

• introduces a range of new methods: nobs(), pval(), teststat(), confint() and print()
• multiple (internal) changes for ropensci standards alignment
• drop the t_boot (teststat_boot) function arguments -> they are now
  TRUE by default
• fix a bug in the lean algorithms - it always tested hypotheses of the form
  beta = 0 instead of R'beta = r, even when R != 1 and r != 0
• enable full enumeration for R-lean tests
• enable deterministic 'full enumeration tests' - these are exact

fwildclusterboot v0.9

fwildclusterboot 0.9

• v0.9 moves data pre-processing from model.frame methods to model_matrix methods. I had wanted to do so for a while, but issue #42, as raised by Michael Topper, has finally convinced me to start this project.

• Moving to model_matrix methods unlocks new functionality for how boottest() plays with fixest objects - it is now possible to run boottest() after feols() models that use syntactic sugar:

library(fwildclusterboot)
library(fixest)

data(voters)
feols_fit <- feols(proposition_vote ~ i(treatment, ideology1) ,
    data = voters
)
boot1 <- boottest(feols_fit,
    B = 9999,
    param = "treatment::0:ideology1",
    clustid = "group_id1"
)

feols_fits <- fixest::feols(proposition_vote ~ treatment | sw(Q1_immigration, Q2_defense), data = voters)
res <- lapply(feols_fits, \(x) boottest(x, B = 999, param = "treatment", clustid = "group_id1"))  

voters$split <- sample(1:2, nrow(voters), TRUE)
feols_fits <- fixest::feols(proposition_vote ~ treatment, split = ~split, data = voters)

res <- lapply(feols_fits, \(x) boottest(x, B = 999, param = "treatment", clustid = "group_id1"))  

Interacting fixed effects via ^ still leads to errors - this remains work in progress:

feols_fit2 <- feols(proposition_vote ~ treatment | Q1_immigration^Q2_defense,
    data = voters
)

boot1 <- boottest(feols_fit2,
    B = 9999,
    param = "treatment",
    clustid = "group_id1"
)

• The release further fixes a multicollinearity bug that occured when lm() or fixest() silently deleted multicollinar variable(s). Thanks to Kurt Schmidheiny for reporting! (see issue #43)

• The na_omit function argument has been dropped. If the cluster variable is not included in the regression model, it is now not allowed to contain NA values.

• Several function arguments can now be fed to boottest() as formulas (param, clustid, bootcluster, fe).

data(voters)
feols_fit <- feols(proposition_vote ~ treatment ,
    data = voters
)
boot <- boottest(feols_fit,
    B = 9999,
    param = ~ treatment,
    clustid = ~ group_id1
)

fwildclusterboot v0.8

Two new bootstrap algorithms: 'WildBootTests.jl' and 'R-lean'

boot_algo = 'WildBootTests.jl'

• fwildclusterboot now supports calling WildBootTests.jl, which is a very fast Julia implementation of the wild cluster bootstrap algorithm. To do so, a new function argument is introduced, boot_algo, through which it is possible to control the executed bootstrap algorithm.

# load data set voters included in fwildclusterboot
data(voters)
# estimate the regression model via lm
lm_fit <- lm(proposition_vote ~ treatment + ideology1 + log_income + Q1_immigration , data = voters)
boot_lm <- boottest(
  lm_fit, 
  clustid = "group_id1", 
  param = "treatment", 
  B = 9999, 
  boot_algo = "WildBootTests.jl"
)

• WildBootTests.jl is (after compilation) orders of magnitudes faster than fwildclusterboot's native R implementation, and speed gains are particularly pronounced for large problems with a large number of clusters and many bootstrap iterations.

• Furthermore, WildBootTests.jl supports a range of models and tests that were previously not supported by fwildclusterboot: most importantly a) wild cluster bootstrap tests of multiple joint hypotheses and b) the WRE bootstrap by Davidson & MacKinnon for instrumental variables estimation. On top of the cake ... the WRE is really fast.

library(ivreg)
data("SchoolingReturns", package = "ivreg")
# drop all NA values from SchoolingReturns
SchoolingReturns <- SchoolingReturns[rowMeans(sapply(SchoolingReturns, is.na)) == 0,]
ivreg_fit <- ivreg(log(wage) ~ education + age + ethnicity + smsa + south + parents14 |
                           nearcollege + age  + ethnicity + smsa + south + parents14, data = SchoolingReturns)

boot_ivreg <- boottest(
  object = ivreg_fit,
  B = 999,
  param = "education",
  clustid = "kww",
  type = "mammen",
  impose_null = TRUE
)
generics::tidy(boot_ivreg)
#              term  estimate statistic   p.value    conf.low conf.high
# 1 1*education = 0 0.0638822  1.043969 0.2482482 -0.03152655 0.2128746

• For guidance on how to install and run WildBooTests.jl, have a look at the associated article.

• Also, note that running the wild cluster bootstrap through WildBootTests.jl is often very memory-efficient.

boot_algo = 'R-lean'

A key limitation of the vectorized 'fast' cluster bootstrap algorithm as implemented in fwildclusterboot is that it is very memory-demanding. For 'larger' problems, running boottest() might lead to out-of-memory errors. To offer an alternative, boottest() now ships a 'new' rcpp- and loop-based implementation of the wild cluster bootstrap (the 'wild2' algorithm in Roodman et al).

boot_lm <- boottest(
  lm_fit, 
  clustid = "group_id1", 
  param = "treatment", 
  B = 9999, 
  boot_algo = "R-lean"
)

Heteroskeadstic Wild Bootstrap

It is now possible to run boottest() without specifying a clustid function argument. In this case, boottest() runs a heteroskedasticity-robust wild bootstrap (HC1), which is implemented in c++.

boot_hc1 <- boottest(lm_fit, param = "treatment", B = 9999)
summary(boot_hc1)

boottest() function argument beta0 deprecated

For consistency with WildBootTests.jl, the boottest() function argument beta0 is now replaced by a new function argument, r.

Frühjahrsputz

I have spent some time to clean up fwildclusterboot's internals, which should now hopefully be more readable and easier to maintain.

Testing

fwildclusterboot is now pre-dominantly tested against WildBootTests.jl. Tests that depend on Julia are by default not run on CRAN, but are regularly run on Mac, Windows and Linux via github actions.

fwildclusterboot v0.7

• A bug fix release, see issues #26 and #27 regarding preprocessing for fixest when weights are passed to feols() as a formula or when cluster is specified in fixest as a column vector.