Devtools doesn't work if served through VsCode remote proxy

[jakemac53]

When using VsCode remote, it has the ability to automatically proxy ports that were launched by a sub-process of VsCode through the code server port. It does this by serving them under a subdirectory, so for example if your code server port was 12345, and the local devtools port was 67890, it would launch devtools at http://127.0.0.1:12345/proxy/67890/.

However, devtools doesn't work when served under a subdirectory like this. It assumes that it is served at the root of whatever server has launched it. This is likely due to the <base href="/"> tag in the header.

Would it be possible to get devtools working when served under a subdirectory to support VsCode remote port proxying?

[kenzieschmoll]

@DanTup can you provide some input on this? I think this is something that you've looked at before.

[DanTup]

@jakemac53 do you have exact steps to reproduce this? We've fixed something related to paths in the past by rewriting the base href (some details in #3585), but I presume that's not happening for your setup. I've not seen the path part of the URL changing before though, previously it seemed to always forward entire ports as-is (usually just putting the port number into another hostname or something).

[jakemac53]

I am just trying to do ctrl+shift+p -> open devtools, from VsCode remote (using code server on the remote box). Maybe the difference is that I am using "code server" and not actual VsCode remote? I remember there is some subtle difference there... I am on a chrombox so I don't have an installed version of VsCode, just the web version.

[DanTup]

Maybe the difference is that I am using "code server" and not actual VsCode remote?

Ah, got it - I'll take a look. I think code-server is an unofficial web version of VS Code not created by Microsoft (I think a lot of their web infra parts are not open source?). However, as long as we're rewriting the base href as I thought we were, I would still expect this to work.

I'll try to repro and see what's up.

[DanTup]

I could reproduce this by running code-server via Docker:

docker run -it --name code-server -p 127.0.0.1:8080:8080 `
  -v "C:\Dev\Temp Projects\code-server-project\local:/home/coder/.local" `
  -v "C:\Dev\Temp Projects\code-server-project\config:/home/coder/.config" `
  -v "C:\Dev\Temp Projects\code-server-project\project:/home/coder/project" `
  codercom/code-server:latest

Then setting up Dart SDK + extensions and trying to open DevTools in the browser.

I looked up where we rewrite the base href and found it was less generic than I remembered:

https://github.com/dart-lang/sdk/blob/953b1a9b0b09ddf467ff71702597fdb7920b15f2/pkg/dds/lib/src/devtools/handler.dart#L51-L56

What we do is support when DevTools is hosted at /devtools via DDS. We don't support arbitrary other URLs. The problem is, we don't even know what that URL is because the code-server proxy is accepting a request for /xxx/proxy/ and then rewriting it back to / before it gets to the DevTools server.

I looked through the docs to see if maybe the proxy is passing through the client version of the path that we could rewrite into base href, but I can't see anything.

Out of interest, are you accessing this in a browser of from a real VS Code frontend? If VS Code via SSH, then maybe disabling the proxy would work, and VS Code's SSH connection would automatically forward the ports to the remote.

Otherwise, if you have a wildcard hostname that resolves to the remote, maybe setting proxy-domain: in the config.yaml will work (because then you should get x.yourdomain instead, and it might use HOST headers to route the traffic without needing multiple ports?).

I found an issue at coder/code-server#6504 which is about Dart-Code in code-server. I'll ping on there to see if maybe the devs have some other ideas that would work (or can confirm if there's a way for us to get at the real client path that we could write into base href).

[DanTup]

I did some testing with proxy-domain and it seems to work as required, except that DevTools needs to run in a secure context (so localhost is fine over HTTP, but other domains are not).

For example if in the code-server config.yaml you put:

proxy-domain: localtest.me:8080

(localtest.me is a wildcard domain resolving to localhost)

Then when I try to launch DevTools, it uses http://9100.localtest.me:8000/ as the hostname, which gets proxied correctly to the DevTools server. However it's not a secure context so there are lots of errors and nothing loads.

If you happen to have a wildcard domain and HTTPS certificate pointing at your remote, probably that would work though.

I commented on the code-server issue linked above... Right now, besides the wildcard domains, I don't think we have enough information to solve this another way.

[DanTup]

Haaacktually, I think I have a simpler fix - although unfortunately it still fails further.. It appears that Flutter supports relative base hrefs (I'm fairly sure it didn't when we implemented path routing in DevTools, but I may be mistaken).

If I put <base href="./"> into index.html then DevTools loads in subfolders (see caveat below). @jakemac53 I think you can test this by going into the Dart SDK being used and modify bin\resources\devtools\index.html to contain:

<base href="./">

However, there is a big caveat... When accessing DevTools, you must have a trailing slash for this path to resolve correctly. When you use dart run --observe, the current URLs printed do NOT have a trailing slash:

PS C:\Dev\Temp Projects> dart run --observe .\foo.dart
The Dart VM service is listening on http://127.0.0.1:8181/nvTlbI5kMVI=/
The Dart DevTools debugger and profiler is available at: http://127.0.0.1:8181/nvTlbI5kMVI=/devtools?uri=ws://127.0.0.1:8181/nvTlbI5kMVI=/ws

This means those links are broken. Adding a trailing slash after devtools/ will fix them.

However, with that fixed, we hit the next problem, which is that DevTools tries to connect to DTD using its local URI and that fails:

This is related to some discuss in Dart-Code/Dart-Code#5158 which is that DevTools has a single URI for DTD which is one that works from the DevTools server, but has not been rewritten for access from the client. We could fix this by having Dart-Code pass the client-facing URI for DTD to the DevTools server (eg. we'll ask code-server to map it to something that ends with /port/proxy/) but if the DevTools server (or anything else on the remote) tries to connect to it, it will fail.

My feeling is that IDEs maybe be able to pass two URIs to the DevTools server - both the "local" URI (used for services running where the DevTools server is) and a "user facing" URI (used from wherever the URI is). For non-remote workspaces they are the same, but for this kind of setup they will be different (the IDE should be responsible for knowing this and providing one or both to DevTools). @kenzieschmoll any thoughts on this? (some additional notes about it in the context of IDX at Dart-Code/Dart-Code#5158 (comment))

[kenzieschmoll]

Still trying to catch up on this issue, but this may be related @DanTup, https://dart-review.googlesource.com/c/sdk/+/376762. CC @bkonyi

[DanTup]

Ah, nice - that fixes part of the issue I noted above. It still leaves the question issue about having two DTD URIs though (one for things connecting from the same machine and one that are on the user/client machine).

Something that also occurred to me is that if we use a relative base href, we still need to rewrite it but now it would be based on the incoming request URI.

Eg.:

• if the incoming request is / then ././ is correct
• if the incoming request is /inspector then ./ is correct
• if the incoming request is /inspector/ then ../ is correct

This is essentially the relative path to get from the request URI to the appRoot (/ for non-DDS and /devtools for DDS).

[DanTup]

I've a change at https://dart-review.googlesource.com/c/sdk/+/376682 that should change to relative base hrefs (that I believe should not change any behaviour for anything).

(this solves one of the issues noted above, but isn't enough to get things working on its own)

[DanTup]

I found a few places in Flutter suggesting relative base hrefs might not work, although I'm failing to understand why. I did some further testing by serving up DevTools at /proxy/1234 with a relative base href and it all seems fine (see flutter/flutter#154620 (comment)).

I've opened flutter/flutter#154620 to see if anyone knows of why we can't/shouldn't do this and if we can make it clearer in the code. If something won't work, it should be clearer (or ideally, fixed).. but as far as I can tell, it works.

[DanTup]

This is fixed by dart-lang/sdk@560adc7 in the SDK which switches to using a relative base href that the server computes based on the incoming request URL.

Running in a remote web context will also require Dart-Code/Dart-Code#5271 though which was included in todays pre-release of the Dart extension (v3.97.20240926).

@jakemac53 once you have both of these, please let me know if you still see any issues with this.

[jakemac53]

Doesn't seem to be working for me still - I see a base href tag like this <base href="&#47;">

[DanTup]

@jakemac53 what version of SDK are you using? dart-lang/sdk@560adc7 only landed in the SDK today (so probably it's not in Flutter master yet, and I'm not sure about internal). Can you run it without the proxy and see what you see?

I would expect to see a . rather than a / with the change. I just downloaded a bleeding-edge SDK from https://gsdview.appspot.com/dart-archive/channels/main/raw/latest/sdk/ and ran dart devtools from it and this is what I saw:

[jakemac53]

I do see a . if I run dart devtools, but if I do ctrl->shift->p and select "Dart: Open DevTools in Browser" then I see the /

[jakemac53]

If I do dart devtools from my remove VsCode I do see the proper . as well (I was previously trying on real VsCode). However now I get a lot of net::ERR_INCOMPLETE_CHUNKED_ENCODING 200 (OK) issues :/. No idea what that is about.

[DanTup]

I do see a . if I run dart devtools, but if I do ctrl->shift->p and select "Dart: Open DevTools in Browser" then I see the /

I can't repro this - could it be browser cache, or your VS Code isn't using the same Dart SDK as you're using from the terminal? (or you haven't reloaded since updating the SDK, so the old devtools server process is still around?).

However now I get a lot of net::ERR_INCOMPLETE_CHUNKED_ENCODING 200 (OK) issues :/. No idea what that is about.

hmm, that's odd - though while searching for it I found https://stackoverflow.com/a/24175704 which mentions a Chrome feature "Predict network actions to improve page load performance" was causing this and it depending on relative vs absolute paths (which sounds very similar to what's happening here). Does clearing the cache make any difference, and if not, can you check the paths of the requests being made that fail with this?

Does it only happen in the remote session? I should try to get something set back up so I can test it exactly as you are.

[bilogic]

I'm trying to get started with flutter using code-server with the following setup:

1. Laptop A with code-server running as a PWA (Chrome)
2. Android device connected via USB to laptop A

Does this mean that the code-server PWA can communicate with the Android device? e.g. upload compiled flutter app or step debugging etc

[DanTup]

@bilogic this issue is specifically about using DevTools in the browser when using code-server.

I think what you're asking about should work in general, and is much less complex. In your setup, all of the extension backend code and invocations of flutter are running on the laptop (only the editor UI is on the client), so the devices and functionality should be largely the same as if you were just running VS Code directly on the laptop. With code-server (or VS Code's own web functionality), only the UI is being pushed to the browser, the extension host process is still a real backend NodeJS process where the Code server (or code-server) is running.

[bilogic]

@DanTup thanks sorry if I'm asking in the wrong place, if you could point me in the right direction, I will remove my irrelevant messages.

To clarify:

1. code-server running on ubuntu server
2. VSCode editor UI pushed to Mac/Windows laptop
3. Android device connected to Mac/Windows laptop

I tried and couldn't quite manage to get code-server to communicate with the Android device.

[DanTup]

I tried and couldn't quite manage to get code-server to communicate with the Android device.

Could you expand on what "communicate" means here. Did the device show up as one that can be selected and something just didn't work, or does the device not show up?

If you file an issue at https://github.com/Dart-Code/Dart-Code/issues with more details, I'll try to provide some more info to help debug. Depending on exactly what the issue is, it may or may not be fixable in the extension though.

Thanks!

[jakemac53]

Does clearing the cache make any difference, and if not, can you check the paths of the requests being made that fail with this?

Sorry I somehow missed this comment/question, but yes clearing the cache does tend to help. I have this issue with many things being forwarded through the vscode remote server so it definitely isn't devtools specific.