Replace Nginx based session proxy with orchestrator proxy

[sitole]

Implements http proxy in orchestrator that replaces current nginx-based session proxy that is using dns server from orchestrator to route traffic.

• Legacy Nginx session-proxy is intact and can still be used as it is, next step will be remove it when traffic migrated
• Orchestrator proxy for now runs on port 5007 allowing smooth deployment
• Support for pretty (for browsers) and json error messages when port in sandbox not listening
• Proxy settings is taken from currently used client-proxy, so should be already battle-tested

[linear]

E2B-1752 Merge session proxy to orchestrator

[ValentaTomas]

Right now (for the deployment) we will have both proxies running at the same time?

[sitole]

Right now (for the deployment) we will have both proxies running at the same time?

Yea, both are running but for traffic new one is used. Actually it's just change of the port in client-proxy.

Comes to my mind, if you think its somehow invasive we can for example route traffic to now one just for some teams / enable it with some feature flag to enable quick revert. Request proxy is done in context where we know about exact client so gives us big space.

[sitole]

Related to 96c3425 commit

Reverts change that routed traffic to orchestrator proxy. After internal discussion we decided that we want to implement orchestrator proxy and then try to add feature-flags to enable it, but its dedicated to separated request.

Just if you want to test changes its needed to revert this commit, without it traffic will float via session-proxy instead.

[jakubno]

I think there's some problem with RPC after this change (I changed the port in client proxy). I can write / read files from sandbox (uses HTTP server). But I can't start command etc..

[ValentaTomas]

@sitole Building up to what @jakubno encountered—running the E2B SDK test suite (and also potentially the code interpreter tests) against your staging with might be a good early test for similar functionality.

[sitole]

With commit 3330b9f i fixed issue were envd randomly responds with connection reset by peer.
Disabling connections keep-alive on new proxy resolved issue.

I noticed current nginx session-proxy is missing keepalive setup (https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) and it looks like default value for missing keep-alive statement is empty, so keep-alive is disabled. That makes sense why current solution worked until now.

I don't like idea to disabling it just because envd complains. Iam mostly concerned about how this can affect traffic. Sure, keepalive was already disabled, but iam not sure if this behave exactly matches nginx vs golang http-proxy one.

I will continue with finding some relevant docs about golang http.server and handling idle connections.

cc @ValentaTomas @jakubno