Letâs learn about using LangChain in JavaScript for streaming structured output responses from the backend to the frontend while building a chatbot application that interacts with the OpenAI API to generate responses to user prompts. In this tutorial we will be using Next.js (with React.js) for the frontend (UI) and backend (API). The finished project can be found on GitHub .
When streaming structured data, we have to go from coarse-grained HTTP streaming to more fine-grained Server-Sent Events (SSE) . I tried everything to make it work without SSE, but wasnât able to make it work. So if you have a better solution, please let me know!
The API endpoint for streaming structured data is very similar to returning one instance for the structured output. The difference is that the backend has to use SSE to stream the structured output to the frontend:
Working with Server-Sent Events in the frontend is a bit more involved than with a regular HTTP request. We have to use the EventSource API to listen for incoming events from the server. The EventSource API is a browser API that allows you to open a connection to a server and receive events from it. However, we want to use the native fetch API in combination with SSE, so we resort to a library called @microsoft/fetch-event-source :
Then we can use it the following way in the form submit handler:
Do not forget to import the new library in your frontend:
Congratulations! You have learned how to stream structured output in JavaScript with LangChain. If you want to learn more about LangChain, check out the official documentation .