Blueprint strategy training does not fully use the CPU

[andyafter]

As the last time when I was training blueprint strategy, perhaps because I set the tree size as a very small number, the training phase was actually done within hours.

However, within these hours, the CPU did not spike at all, it was constantly hanging around below 10%.

After which the data writing phase actually lasted for 4~5 days.

Is it possible that we could maximize the CPU usage of the training? Or is it simply because of my tree size is too small.

[andyafter]

Sorry my bad, it is occupying 80% CPU now, I just ran it

[krukah]

the writing phase taking several days is certainly a point of possible optimization. i'm almost certain there's a more efficient way of serializing to disk rather than iterating over a BTreeMap<_, _> and using u64::from(*) impls. i eagerly welcome such a contribution !

however, with CPU utilization on blueprint training, there exists a fundamental tradeoff that you must understand:

• the training process involves parallelized reads
• these reads each yield 1 update (Counterfactual<_>)
• the updates require mutation to the in-memory Profile struct
• the mutation must be synchronized, while the reads do not need to be
• fully utilizing CPU is possible for the reads, and is not, as far as i am aware, possible for the profile mutation
• so during the mutation step, i can't imagine how to utilize more CPU and i would chalk this up to being a fundamental constraint of "rust makes it hard for multiple threads to write to the same object"
• there may very well be some optimization using a concurrent HashMap/BTreeMap (see dashmap crate), which i welcome you to look into since it would provide quite a large speedup for blueprint training

[andyafter]

Although, memory usage is constantly small though. So yeah, recently I took a look at the paper again. Just want to confirm with you:

• In each iteration of blueprint training, can I split the training task into smaller chunks so that I can give it to different workers.

My understanding is that we can, if so I can actually do some concurrency changes on the codebase.

[krukah]

yeah, i do believe this is possible, but im not certain. at the very least, i don't know how it would impact TouchedNodes/sec, which might be the best metric for training speed. but i have to assume it would be better off

the first step would be assuming that all BTree lookups become asynchronous, probably just postgres lookups. however there are real tradeoffs to requiring async lookups to:

• map Observation-> Abstraction
• query our blueprint profile, to build our CFR trees
• update our blueprint profile.

all of these operations would lose at least an order of magnitude of performance compared to their in-memory-lookup equivalent, even if they can now be partitioned an order of magnitude more.

thoughts?

[andyafter]

I definitely agree with you on this, the in memory operations needs to keep "in memory", as each computation task is too simple to be "sent out" to do.

My thoughts came from the literatures actually since most of the papers I read from CPRG mentions about the heavy reliance of these AIs they created on RAM. As a matter of fact the several tests that I ran so far, doesnt matter the training or the clustering, the RAM was never really used much.

So I was wondering if there is a way for us to try these things:

1. Somehow make the data structures "flattened", where we exploit as much as memory and in return get a trade off for efficiency, or
2. Have multiple training running on different machines, each randomly touch certain leaves, and then combine them together with some sort of combining rules

This way we can savage both Computation and RAM power.

[andyafter]

Loop tiling won't work if the matrixes we work on are too big on one dimension and too small on the other, however perhaps making the training step serielized for the predictors in the CPU to fast loading them would probably help.

And even if loop tiling works for blueprint training, my guess is that it's gonna be 4 times faster at most. There still needs to be a good 5~10 times optimization for robopoker to be able to make 6 people tables economically viable.