CPU time

[cyber42]

Hi,

I'm using timeout_decorator for machine learning workloads, ensuring that parameter optimizations terminate below pre-defined timeout. This helps in my process, but my workflow became sensitive to any other workload happening on the same host.

I wonder if terminating the function by CPU time is technically possible with the approach followed by timeout_decorator. This would ensure that the optmization step is independent of other processes using the host.

Would this make sense, I could fork / create a new branch or send a pull request with proposed changes.

Let me know if this makes sense, and if you'd be interested in intergrating this in the existing codebase.

Thanks, Mark

[bitranox]

Dear Mark,
I think thats a good idea.
In Python, you can measure CPU time using the time module. The time.process_time() function returns the current process time in seconds, which can be used to measure CPU time.
I guess my own timeout decorator can be easily adopted to archive that, since there is already a polling mechanism in there.
Check : https://github.com/bitranox/wrapt_timeout_decorator
I would simply add a parameter use_process_time which defauls to False to use wallclock time.
if that parameter is set to True we use process_time

I created a new issue on my repo : bitranox/wrapt_timeout_decorator#49

yours sincerely
Bitranox

[causalitygroup]

Sounds good, thanks! We're double checking the idea with my colleagues and get back to you.

Mark

[bitranox]

Dear Mark,
also check out the meaning of our hard_timeout parameter, for more accurate timing, since that considers also the time needed to fork (or to simulate a fork on windows) - so You get get a more accurate timing.
(the default is hard_timeout = False, so the timeout starts to be calculated when the spawned process is really starting.
Of course the timeout method multiprocessing is needed to archive that.
If You spawn many processes, the spawning itself might take a considerable time, especially on windows - that will be considered here, so Your learning processes will all get the same process_time.

yours sincerely
Bitranox