Question: set up time limit for solvers

[Rogerpi]

Hello,

Is it possible to set a maximum execution time for solvers? I have generated a set of domains and problems and made a script to run them all through server requests, but I would need to set a maximum time so it does not block the benchmarks. I have not been able to find any information about it.

Thanks!

[haz]

Entirely planner dependent. For example, the suite of Fast Downward planners would let you set it via the command line:

usage: fast-downward.py [-h] [-v] [--show-aliases] [--run-all] [--translate] [--search]
                        [--translate-time-limit TRANSLATE_TIME_LIMIT]
                        [--translate-memory-limit TRANSLATE_MEMORY_LIMIT] [--search-time-limit SEARCH_TIME_LIMIT]
                        [--search-memory-limit SEARCH_MEMORY_LIMIT] [--validate-time-limit VALIDATE_TIME_LIMIT]
                        [--validate-memory-limit VALIDATE_MEMORY_LIMIT] [--overall-time-limit OVERALL_TIME_LIMIT]
                        [--overall-memory-limit OVERALL_MEMORY_LIMIT] [--alias ALIAS] [--build BUILD] [--debug]
                        [--validate] [--log-level {debug,info,warning}] [--plan-file FILE] [--sas-file FILE]
                        [--keep-sas-file] [--portfolio FILE] [--portfolio-bound VALUE] [--portfolio-single-plan]
                        [--cleanup]
                        INPUT_FILE1 [INPUT_FILE2] [COMPONENT_OPTION ...]

Converted the issue to a Q&A discussion, as this is kind of out of the scope for planutils implementation.

[Rogerpi]

Thanks for the fast answer!

Still, would it be possible to add a timeout that stops the solver process? Then would not depend on the planner.

Right now, I'm facing some pddl problems that make some solver run for hours without a solution, and I end up stopping the docker and restarting again. I see I could update the manifest of each planner that allows it and add this parameter to the JSON request, but I feel that limiting the amount of time might be a desirable feature regardless of the solver used.

I wonder your opinion about it and if you could provide some guidance on how to do it, if possible.

Thanks!

[ctpelok77]

So, this is actually an interesting question.
For different paradigms, it might make sense to use the internal planner option as opposed to an external resource constraint (ulimit).
With planners that produce a single plan, there is usually very little overhead once a plan was found.
There are any-time planners that improve plan quality with time (e.g., lama) and there as well, the last plan found is the one you want.
But there are planners that produce a set of plans as their solution, e.g., top-k planners. Now, the question is, what is the expected behavior with a timeout? All plans found so far? If you asked for 1000 plans, and only 100 were found, should these 100 be returned?
If the answer is yes, then you need to use the internal mechanism of the planner (in K* planners, the mechanism of fast downward mentioned in the previous comment is reused, forbid iterative I think has a similar option). If the answer is no, you should use an external constraint.

[haz]

Ya, and diverse planning is just one example of the variety in planutils (there's also macq, and tarski, and planning.domains scripts, and cpddl, and and and...). Not all of these exactly make sense for a time-out to be given.

@Rogerpi : there is a closely related Planning-as-a-Service project that aims at wrapping a custom subset of the planutils functionality. Over there, we do have an environment variable that will try to nix things after a certain number of seconds (see here). Maybe a server wrapping of planutils is what you'd want?

As a planutils core feature, I still think it's a bit mis-aligned. The planning-as-a-service or lab or similar projects are far more appropriate for such things.

As a related aside, @nirlipo : how does the time-limit on the PaaS actually get enforced? This is the core line I see it in action.