You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
I started working on supporting streaming on our UI with the new road-core/service code. I found that when media_type is set to application/json, the /streaming_query API returns JSON data like:
I would like to use the fetch-event-source library on our React UI and it is enabled with this change. OpenAI API streaming support is also using the Event stream format (I tested my local ollama server) and I think it is reasonable to support that format on the /streaming_query API. Thanks.
Describe the solution you'd like
Add an option to /streaming_query API to enable the Event stream format.
Describe alternatives you've considered
We (Ansible Lightspeed) are using road-core/service as a backend service. We can convert the streaming format on our service that directly communicates with UI, but we think it is desirable that this is implemented on road-core/service.
Additional context
Chat completions API streaming sample (tested with local ollama server):
Is your feature request related to a problem? Please describe.
I started working on supporting streaming on our UI with the new road-core/service code. I found that when media_type is set to application/json, the /streaming_query API returns JSON data like:
Can we have an option to receive data in the Event stream format, i.e. adding data: at the beginning and two new lines at the end of each chunk:
I would like to use the fetch-event-source library on our React UI and it is enabled with this change. OpenAI API streaming support is also using the Event stream format (I tested my local ollama server) and I think it is reasonable to support that format on the /streaming_query API. Thanks.
Describe the solution you'd like
Add an option to
/streaming_query
API to enable the Event stream format.Describe alternatives you've considered
We (Ansible Lightspeed) are using road-core/service as a backend service. We can convert the streaming format on our service that directly communicates with UI, but we think it is desirable that this is implemented on road-core/service.
Additional context
Chat completions API streaming sample (tested with local ollama server):
The text was updated successfully, but these errors were encountered: