Skip to content

Provide better diagnostics than "The server shut down unexpectedly." #362

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
AArnott opened this issue Apr 28, 2025 · 8 comments · May be fixed by #376
Open

Provide better diagnostics than "The server shut down unexpectedly." #362

AArnott opened this issue Apr 28, 2025 · 8 comments · May be fixed by #376
Assignees
Labels
bug Something isn't working

Comments

@AArnott
Copy link

AArnott commented Apr 28, 2025

When an MCP server fails to initialize, McpClientFactory.CreateAsync throws InvalidOperationException: 'The server shut down unexpectedly.'

No additional information is provided in the exception. We need to do better. We need to explain to the user what went wrong so that users can understand and fix the issue.

In one case, for example, the local server config was to run the following command:

docker run -i --rm -e GITHUB_PERSONAL_ACCESS_TOKEN ghcr.io/github/github-mcp-server

All I got back was the above exception. But when I run it from the command line, I get much more useful data:

docker: error during connect: Head "http://%2F%2F.%2Fpipe%2FdockerDesktopLinuxEngine/_ping": open //./pipe/dockerDesktopLinuxEngine: The system cannot find the file specified.

Run 'docker run --help' for more information

That actually gives me a clue that somehow my docker engine isn't running. I launched Docker Desktop, and retried. Sure enough, it works.

But at the moment, this info is inaccessible to users running in Visual Studio.

@AArnott AArnott added the bug Something isn't working label Apr 28, 2025
@stephentoub
Copy link
Contributor

Do you have a recommendation for what to do here? I assume the Process.Start actually successfully starts the process, which means the output you're seeing, coming on either stdout or stderr, would need to be interpreted as error information rather than something meaningful from a running server (some of which may redirect their logs to stderr).

@AArnott
Copy link
Author

AArnott commented Apr 28, 2025

Ah, the woes of using STDIO as an RPC channel. I hate it for this reason, and because it's so dang easy for code running in that process to inadvertently pollute the stream by console logging.

At a minimum, I think you should collect the STDERR stream content and include it as a property on the exception you throw so we can share it with the user.

@stephentoub
Copy link
Contributor

stephentoub commented Apr 29, 2025

At a minimum, I think you should collect the STDERR stream content and include it as a property on the exception you throw so we can share it with the user.

We could. The reason we haven't is, because stdout can't be used for logging, MCP actually suggests that stderr be used not just for errors but for general purpose logging. It's why examples like in our README:
https://github.com/modelcontextprotocol/csharp-sdk?tab=readme-ov-file#getting-started-server
do things like:

consoleLogOptions.LogToStandardErrorThreshold = LogLevel.Trace;

So we'd be tracking potentially unbounded amounts of state that might then be included in an exception error message. We could try to limit this, maybe putting a limit on how much we store, and only doing so until the initialization handshake has been successfully completed...

(It's also a bit challenging to factor well, as the knowledge of the handshake and the failure happen at the client/session layer while the tracking of stderr happens at the transport layer.)

@AArnott
Copy link
Author

AArnott commented Apr 29, 2025

because stdout can't be used for logging, MCP actually suggests that stderr be used not just for errors but for general purpose logging

gah. Another reason to add to my distaste of STDIO for IPC. (not your fault, I know.)

Can we attach our own logger to stderr of the server then, so we can point the user to the log?

@stephentoub
Copy link
Contributor

stephentoub commented Apr 29, 2025

Can we attach our own logger to stderr of the server then, so we can point the user to the log?

The stdio client automatically logs all of the server process' stderr, so if you provide your own logger to the client, you should be getting all of that information already. The challenge I expect would be filtering it in a robust/maintainable manner. We might be able to improve that, though, especially if we can't come up with anything better, e.g. adding a callback to the stdio client constructor that would receive all of the stderr redirected output.

@halter73
Copy link
Contributor

I don't think it would be a bad idea to include the last 10 lines of stderror output in at the end of the "The server shut down unexpectedly" exception message. It might be generic logs, but if the server shutdown unexpectedly, that could still be very useful information.

Furthermore, this should also be an IOException rather than an InvalidOperationException, because the caller didn't necessarily do anything wrong to cause the stdio server to crash before attempting to send a message or response.

@stephentoub
Copy link
Contributor

stephentoub commented Apr 29, 2025

I don't think it would be a bad idea to include the last 10 lines of stderror output in at the end of the "The server shut down unexpectedly" exception message

Thoughts on how to structure this? The exception is coming from McpSession.ProcessMessagesAsync, which only has access to an arbitrary ITransport. I don't think we want to hardcode knowledge of that being a stdio transport, which suggests the answer is to expose something from ITransport, which is kind of gross, but I don't have a better answer. Another option would be to expose something from StdioClientTransport, and then type test / special-case in ProcessMessagesAsync, which is also kind of gross.

@halter73
Copy link
Contributor

halter73 commented May 1, 2025

Another option would be to treat any process exit in StdioClientSessionTransport prior to disposal as ungraceful and complete the ChannelWriter with an exception including the last 10 lines of stderr (and stdout if we haven't gotten past initialization) if any.

I'm a little surprised this isn't how we handled things initially. Not necessarily the including the 10 lines in the exception message part but throwing an exception in the first place. It's not like HTTP where server authors are more likely to want to kill idle sessions to save on backend costs. We could consider only raising an exception for non-zero exit codes if we worried throwing on any early exit will be too noisy, but I don't think it will be. I think we should raise an exception for any early exit and include the status code in the exception message even if it's 0.

And I want to reiterate that it doesn't make sense for StreamClientSessionTransport.SendMessageAsync to throw an InvalidOperationException if it's not connected. You cannot guard against it as a caller. That only makes sense if it was possible to call that method before connecting, but that isn't possible. StreamClientSessionTransport sets IsConnected to true in its constructor. If the transport isn't connected because it's disposed, it should be an ODE. And if it isn't connected because the peer/server disconnected unilaterally, that should be an IOException.

@stephentoub stephentoub self-assigned this May 1, 2025
@stephentoub stephentoub linked a pull request May 2, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants