Skip to content

Conversation

topepo
Copy link
Member

@topepo topepo commented Feb 14, 2022

All of these plots have figure captions.

@topepo
Copy link
Member Author

topepo commented Feb 14, 2022

The hash for the SA results appears to have changed.

Locally, I get the same iteration results at the website:

#> Optimizing roc_auc
#> Initial best: 0.86627 
#>  1 ♥ new best           roc_auc=0.87293   (+/-0.007638) 
#>  2 ♥ new best           roc_auc=0.89656    (+/-0.006779) 
#>  3 ─ discard suboptimal roc_auc=0.88247    (+/-0.00731) 
#>  4 ─ discard suboptimal roc_auc=0.8812  (+/-0.006122) 
#>  5 ◯ accept suboptimal  roc_auc=0.88226    (+/-0.007343)

website:

#> Optimizing roc_auc
#> Initial best: 0.86627
#>  1 ♥ new best           roc_auc=0.87293  (+/-0.007638)
#>  2 ♥ new best           roc_auc=0.89656  (+/-0.006779)
#>  3 ─ discard suboptimal roc_auc=0.88247  (+/-0.00731)
#>  4 ─ discard suboptimal roc_auc=0.8812   (+/-0.006122)
#>  5 ◯ accept suboptimal  roc_auc=0.88226  (+/-0.007343)

Perhaps something else in the object has changed. @juliasilge Can you send your results?

@juliasilge
Copy link
Member

I do see the error if I run locally:

> verify_consistent_bo(svm_bo)
Error in `verify_consistent_bo()`:
! These Bayesian optimization results don't match the previous values.
Run `rlang::last_error()` to see where the error occurred.

I have the same results for the first five iterations:

Optimizing roc_auc
Initial best: 0.86627
 1 ♥ new best           roc_auc=0.87293	(+/-0.007638)
 2 ♥ new best           roc_auc=0.89907	(+/-0.007041)
 3 ─ discard suboptimal roc_auc=0.88247	(+/-0.00731)
 4 ─ discard suboptimal roc_auc=0.8812	(+/-0.006122)
 5 ◯ accept suboptimal  roc_auc=0.88226	(+/-0.007343)

But the last ten are different:

40 ◯ accept suboptimal  roc_auc=0.89	(+/-0.005141)
41 ◯ accept suboptimal  roc_auc=0.8877	(+/-0.007133)
42 + better suboptimal  roc_auc=0.89898	(+/-0.00658)
43 ◯ accept suboptimal  roc_auc=0.89895	(+/-0.006654)
44 ─ discard suboptimal roc_auc=0.87502	(+/-0.007454)
45 ─ discard suboptimal roc_auc=0.88489	(+/-0.007197)
46 ◯ accept suboptimal  roc_auc=0.88445	(+/-0.007064)
47 x restart from best  roc_auc=0.87697	(+/-0.007277)
48 ─ discard suboptimal roc_auc=0.84918	(+/-0.008598)
49 ─ discard suboptimal roc_auc=0.88926	(+/-0.005202)
50 ─ discard suboptimal roc_auc=0.87209	(+/-0.006013)

topepo added a commit that referenced this pull request Feb 16, 2022
@topepo
Copy link
Member Author

topepo commented Feb 16, 2022

Changes moved to #231 and 9a04f7e

@topepo topepo closed this Feb 16, 2022
@topepo topepo deleted the graph-updates branch February 16, 2022 22:00
@github-actions
Copy link

github-actions bot commented Mar 3, 2022

This pull request has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.

@github-actions github-actions bot locked and limited conversation to collaborators Mar 3, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants