Forks and spawns (VS Code on httptap yields nothing)

[jvenge]

Built from src, ran the sysctl commands:

$ ./httptap -- /usr/bin/code
$

That's it. VS Code actually starts, but ps auxf tells me that the command I issued to cli calls a launcher or wrapper, the actual instance is initiated there. Can I make httptap follow the process invocation?

Notably, this instance of VSC won't see the net at all, the Extension browser will tell me that it is offline. U24.04, I ran the sysctl mods before.

[alexflint]

Hey - it looks like the issues is that httptap exits when the top-level process exits, leaving the "real" vscode stranded in an isolated network namespace with nobody to proxy traffic to the outside world. A simple workaround is this:

httptap -- bash -c "code; sleep infinity" 

What's happening here is that vscode forks and detaches itself as is standard for GUI apps. Luckily, that still doesn't let it escape from the network namespace that httptap creates. All that is needed is to force httptap's proxy code to keep running even when the direct subprocess returns, which can be done with "sleep infinity". You will have to hit ctrl+c manually to exit manually when you close down vscode.

Separately, for vscode you may want to add --ignore-certificate-errors because electron does not seem to pick up httptap's CA file. So the full invocation becomes

httptap -- bash -c "code --ignore-certificate-errors; sleep infinity" 

Amazingly, when I do this at my end, the vscode instance that starts does not accept any keyboard input! That's really crazy. Do you see that too? I have no idea why that would be.

But going back to the original issue, for your interest and for the benefit of others who come across this issue, you can reproduce the fork-to-background phenomenon in a more minimal way by running two levels of setsid like this:

$ httptap -- setsid setsid curl http://httpbin.org/get
curl: (6) Could not resolve host: httpbin.org

This only applies to programs that explicitly daemonize themselves (typically by forking, then putting themselves into a new process group, then forking again), not to programs that simply start normal subprocesses. httptap has no trouble with normal subprocess creation. For example:

$ httptap -- python -c 'import subprocess; subprocess.run(["curl", "http://httpbin.org/get"])'                 
---> GET http://httpbin.org/get
<--- 200 http://httpbin.org/get (285 bytes)
...

Interested to hear if this resolves your issue.

[alexflint]

Just adding some further notes here from my own research. The fork-detach-fork thing that GUI apps do can be reproduced via python like this:

$ httptap -- python -c 'import os; import sys; import subprocess
if os.fork(): sys.exit()
os.setsid()
if os.fork(): sys.exit()
subprocess.run(["curl", "http://httpbin.org/get"])'

Output:

curl: (6) Could not resolve host: httpbin.org

In contrast, if the top-level process just sleeps long enough for the job to be done then it all works out perfectly:

$ httptap -- python -c 'import os; import sys; import subprocess; import time       
if os.fork(): time.sleep(1)
os.setsid()
if os.fork(): sys.exit()
subprocess.run(["curl", "http://httpbin.org/get"])'
---> GET http://httpbin.org/get
<--- 200 http://httpbin.org/get (285 bytes)

I think I should add a --no-exit argument to httptap that causes it to keep proxying until the user kills it with ctrl+c.

[jvenge]

Appreciate your efforts. I cannot reproduce the keyboard issue, but on the other hand, while browsing and installing extensions the amount of reported sites amounts to zilch. This either means VSC does not relay http calls to the system or they get lost somehere else. I'll have to talk to the VSC poeple on this (or dissect the src). Or see if wireshark sees something that I don't here.

[alexflint]

When I search extensions with vscode running under httptap, I do see the HTTP calls. For example searching "axolotl" in vscode under httptap gives me this output:

---> POST https://marketplace.visualstudio.com/_apis/public/gallery/extensionquery
<--- 200 https://marketplace.visualstudio.com/_apis/public/gallery/extensionquery (4332 bytes)
---> POST https://mobile.events.data.microsoft.com/OneCollector/1.0?cors=true&content-type=application/x-json-stream
<--- 200 https://mobile.events.data.microsoft.com/OneCollector/1.0?cors=true&content-type=application/x-json-stream (9 bytes)
---> GET https://kappadoo.gallerycdn.vsassets.io/extensions/kappadoo/axolotl-vsnote/0.1.1/1646479180048/Microsoft.VisualStudio.Services.Icons.Default
<--- 200 https://kappadoo.gallerycdn.vsassets.io/extensions/kappadoo/axolotl-vsnote/0.1.1/1646479180048/Microsoft.VisualStudio.Services.Icons.Default (7350 bytes)
---> POST https://mobile.events.data.microsoft.com/OneCollector/1.0?cors=true&content-type=application/x-json-stream
<--- 200 https://mobile.events.data.microsoft.com/OneCollector/1.0?cors=true&content-type=application/x-json-stream (9 bytes)

The invocation that I eventually came to, making use of the --no-exit flag that I just very recently merged to main, and using --no-new-user-namespace to work around the keyboard issue I had, is this:

sudo $(which httptap) --no-exit --no-new-user-namespace -- sudo -u $USER code --ignore-certificate-errors .

If you don't have the same keyboard issue, then you should be able to do (again, with the latest from main):

httptap --no-exit -- code --ignore-certificate-errors .

If it's working then right away when vscode starts you'll see some HTTP calls such as:

---> OPTIONS https://default.exp-tas.com/vscode/ab
<--- 204 https://default.exp-tas.com/vscode/ab (0 bytes)
---> GET https://update.code.visualstudio.com/api/update/linux-x64/stable/903b1e9d8990623e3d7da1df3d33db3e42d80eda
<--- 200 https://update.code.visualstudio.com/api/update/linux-x64/stable/903b1e9d8990623e3d7da1df3d33db3e42d80eda (430 bytes)
---> GET https://default.exp-tas.com/vscode/ab
<--- 200 https://default.exp-tas.com/vscode/ab (4184 bytes)

By the way, are you by any chance running vscode using snap? Might be worth installing via rpm or deb or tar.gz if you possibly can as I've had lots of issues running httptap with snap apps on account of the way that snap uses a lot of complex containerization tech that interacts quite badly with being run in an isolated network namespace.

[jvenge]

Built lastest: the plain httptap ... call starts code but does not yield any http call tracking.

The sudo call, however, makes it work, so yay, success. I just don't understand what makes it tick now.

This all is happening in a fresh install of ubuntu desktop 24.04LTS in VirtualBox, so no idea if the virtual input devices add shenanigans in this particular scenario.

[alexflint]

Good to hear!

The sudo call does it without creating a user namespace, which is a bit of containerization tech in the linux kernel that lets you create (among other things) network namespace without being root. There seems to be a lot of work happening in the kernel and elsewhere that changes where you can create user namespaces, and how they work, so I guess it's not surprising that excising that bit of tech and just going with sudo works.

The reason there are two sudo calls in there is that vscode refuses to run as root, so you have to sudo once to create the network namespace (the first sudo in the command), then sudo again to switch back to the original user to actually run vscode.