Tune an xgboost model with early stopping and #TidyTuesday childcare costs | Julia Silge

[utterances-bot]

Tune an xgboost model with early stopping and #TidyTuesday childcare costs | Julia Silge

A data science blog

https://juliasilge.com/blog/childcare-costs/

[JorgeRuRe]

Hi Julia! Thank you for this screencast! Is it possible to calculate the G-mean as a metric for evaluation in Tidymodels? I have been looking at the documentation of Yardstick, but G-mean is not included. Do you know if this is possible?

[juliasilge]

So this is the geometric mean like sqrt(sensitivity * specificity)? This is not currently implemented in yardstick but it would be a pretty straightforward custom metric. You can see an example of how to approach that here, and since you are using sensitivity and specificity, you can use that code pretty directly.

[HanLum]

Hi Julia,

This is a great tutorial, thank you!
I'm just wondering, is the validation set for early stopping selected randomly from the training set? I have a highly imbalanced dataset (84%) and am concerned the validation set may not include enough of the minority class (is there a way to stratify?)

Best wishes,
Hannah

[juliasilge]

@HanLum It is chosen randomly, yep, I don't believe there is currently a way to specify more details for the validation set used for early stopping. You can check out details on this here: https://parsnip.tidymodels.org/reference/details_boost_tree_xgboost.html#early-stopping

You might want to chime with your use case here: tidymodels/parsnip#765

[HanLum]

Thank you Julia!!