Is there a way to improve citnames build speed

[SunJun8]

As the title,

it takes a long time to build json for a 300M compiled database, and citnames seems to use only one cpu core

[rizsotto]

Sure, that process is a single threaded one. And currently does not do anything which would block parallel processing.

Are you interested to implement it? (I can guide you what's needed to make it.)

[SunJun8]

Sure, that process is a single threaded one. And currently does not do anything which would block parallel processing.

Are you interested to implement it? (I can guide you what's needed to make it.)

ok i would like to try

[rizsotto]

Thanks @SunJun8 !

The function which needs a change is the transform function in the source/citnames/source/Application.cc. As you can see, it does transform each event into one (or more) compilation database entries, and merge the together.

I've tried to use std::transform_reduce to run it parallel, but looks like parallel-STL functions are still not ready. So, I would recommend to write a similar map-reduce like higher-order function, which is written by the existing C++17 constructs, and can be use to implement the transform method.

[rizsotto]

Looking for help to make this work. #455 is a PR where I use parallel STL, but see no improvements. If anyone knows how to fix it, I would be glad.