async web agent

[NicolasAG]

Description by Korbit AI

What change is being made?

Introduce an asynchronous and parallelized framework for a web-based reinforcement learning agent, including new modules for async task execution, environment orchestration, and step management.

Why are these changes being made?

The changes are designed to improve the scalability and efficiency of the web agent by allowing parallel processing of tasks and asynchronous interactions with remote environments. This update optimizes the agent's task execution loop and introduces mechanisms to handle repetitive steps and prevent redundant retries, offering a more robust system for managing complex web interactions.

Is this description stale? Ask me to generate a new description by commenting /korbit-generate-pr-description

[korbit-ai]

Based on your review schedule, I'll hold off on reviewing this PR until it's marked as ready for review. If you'd like me to take a look now, comment /korbit-review.

Your admin can change your review schedule in the Korbit Console

[ollmer]

Looks good so far! Let's plug your miniwob agent structure into client_main_loop.yaml and check how it's going. Then when we're sure that it works correctly, lets measure the throughput by spawning lots of coroutines with agents against 16 or 32 remote environments on a single node.

[ollmer]

@NicolasAG lets merge it to main maybe? The agent and the env server are pretty stable now.

[korbit-ai]

Korbit doesn't automatically review large (3000+ lines changed) pull requests such as this one. If you want me to review anyway, use /korbit-review.

[ollmer]

LGTM! Please take a look at my changes at the remote_environment.py and lets merge.

[rizar]

took a quick look at RemoveEnvironment, looks very goood!