Skip to content

Conversation

@MHiabu
Copy link
Contributor

@MHiabu MHiabu commented Aug 8, 2025

Some Enhancements/modifications.

Three new parameters added

max_candidates (default=50):
After many splits the number of split-option for which one needs to calculate the resulting loss explodes. Now the number of possible options is changed from (t_try * possible options) to max(max_candidates , t_try * possible options ).
With this change the splits parameter can be set much higher because computational cost now only grows linearly instead of quadratic.

split_decay_rate (default=0.1):
Possible splits are initiated with age=0. Whenever a possible split becomes a split_candidate (i.e. it has been drawn when drawing max(max_candidates , t_try * possible options ) times) it ages by +1. The age of the single split-candidate that has actually minimal loss is reset to zero. A high decay rate means faster aging. split_decay_rate=0 results in no aging and therefore mimics the old behaviour.

delete_leaves (default=1):
Originally if a leaf is split with respect to a variable that is already part of the leaf, then the leaf is deleted/replaced by the newly created children. This is also the default behaviour now.
Additionally, one can choose delete_leaves=0. In this case leaves are never deleted. I.e., a split results always in two new leaves while still keeping the parent.

New behaviour of split_try

Before, split_try was determining how many split points to try out in each leaf of every split_candidate. Therefore the number of splits that have been tried out in a split_candidate was (number of leaves in split_candidate)*split_try.
With this change in every split_candidate, split_try combinations of leaves and split points are chosen. Hence the number of splits that are being tried out in a split_candidate is reduced to be just split_try.

@MHiabu MHiabu requested review from jemus42, jyliuu and mnwright August 8, 2025 13:53
@MHiabu MHiabu closed this Aug 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants