Streaming AI responses

A shop owner fills in product pages, and a Write button asks a model, the AI that writes text, for each description. A model writes a few words at a time, so a long answer takes ten seconds or more. A page that waits shows nothing all that time, and people click again or leave. A page that shows each piece as it arrives, called streaming, feels fast, lets people stop a bad answer, and costs nothing extra.

The Code tab holds two pages of a Nuxt app (Nuxt is a Vue framework that also runs server code): a description writer that reads its stream by hand, and a chat with a Vue tutor built on the AI SDK, a library for calling models. Both get a Stop button that really stops. You need one idea from the track's first lesson, AI for Vue developers: the map. A page never calls a paid model itself. It calls a route, a URL your own server answers, and the server holds the secret key that pays for each call.

Read a stream by hand

The description writer reads its stream with no library, which shows what every streaming tool does underneath. You type a product, press Write, and the route /api/draft sends the description back as plain text. The page, app/pages/draft.vue, gets it from useTextStream(), a composable: a function that gives a page refs and the code that updates them.

Inside the composable, fetch, the browser's call for a URL, returns a response, res. Its body, res.body, is a ReadableStream, which hands out data as it arrives. The composable takes a reader from it with getReader() and reads one chunk after another in a loop:

const reader = res.body.getReader()
const decoder = new TextDecoder()
while (true) {
  const { value, done } = await reader.read()
  if (done) break
  text.value += decoder.decode(value, { stream: true })
}

Each reader.read() gives the next chunk of bytes, until done is true. A TextDecoder turns the bytes into text, and the text ref, printed by the page, grows with every chunk.

{ stream: true } matters. A character such as é takes two bytes, and a chunk can end between them. Without it, each chunk is decoded on its own, and "Café" arrives as "Caf��". With the option, the decoder holds the first byte until the second one comes.

The fetch also takes the signal of an AbortController, a built-in object that cancels requests, so stop(), behind the page's Stop button, can cancel it.

The server streams, the key stays there

The tutor chat, the Code tab's second page, needs a stream that carries more than text: whose message it is, when an answer starts and ends, whether something failed. The AI SDK sends that as a UI message stream, and its useChat() composable reads it. The page posts the conversation to /api/tutor, a route written in server/api/tutor.post.ts. The route calls the model with the key from private runtime config, settings only server code can read.

The route makes one streamText() call, which asks the model and streams its answer:

const result = streamText({
  model: gateway('anthropic/claude-sonnet-5.5'),
  instructions: 'You are a friendly Vue tutor. Keep answers short.',
  messages: await convertToModelMessages(messages.slice(-20)),
  maxOutputTokens: 800,
  abortSignal: controller.signal,
})
return createUIMessageStreamResponse({ stream: toUIMessageStream({ stream: result.stream }) })

streamText() is the streaming form of generateText(), which waits for the whole answer. instructions is the system prompt, the standing rules for every answer. maxOutputTokens caps the answer in tokens, the pieces of words a provider bills. messages.slice(-20), the last twenty messages converted for the model, caps how much history you pay to send.

In the browser's Network tab the response is a list of server-sent events, one JSON object per line, such as data: {"type":"text-delta","id":"t1","delta":"A ref"}. Near the end, a finish event carries a finishReason: "stop", or "length" when maxOutputTokens cut the answer.

The abortSignal is for a person who leaves, with Stop or a closed tab. Nuxt does not tell the AI SDK that the browser left, so without it the route reads the answer to the end for nobody, and you pay for every token. The route's own AbortController fires when event.node.res, the response it sends, closes.

useChat in a page

The tutor page, app/pages/tutor.vue, gets the whole chat from one useChat() call, from the @ai-sdk/vue package. A DefaultChatTransport points it at your route instead of its default, /api/chat:

const { messages, sendMessage, status, stop, error, clearError, regenerate } = useChat({
  transport: new DefaultChatTransport({ api: '/api/tutor' }),
})

messages grows as text streams in, and the template prints the text parts of each message. status is 'submitted' while waiting for the first word, 'streaming' while words arrive, 'ready' when done, and 'error' when the request failed. The page shows Thinking for the first, a Stop button for the first two, and enables the input only when 'ready'.

When a request fails, the message in error is the raw response body, so the page shows its own sentence, "The tutor could not answer", and a Try again button that calls clearError(), then regenerate(), which sends the last question again.

stop() cancels the request and keeps the words already shown. Leaving the page does not call it: useChat stops updating its refs, but the request runs on and the server writes for nobody. onBeforeUnmount(() => stop()), which runs just before Vue removes the page, closes it.

What goes wrong

On the description writer, a Stop button without a catch throws on every click.

const { value, done } = await reader.read() // rejects when stop() aborts

The text stops, as it should, but in development the console shows [Vue warn]: Unhandled error during execution of native event handler followed by Uncaught (in promise) AbortError: signal is aborted without reason. Aborting rejects the waiting read(), and nothing catches it. Catch it as the normal end it is:

try {
  // fetch and the read loop
}
catch (error) {
  if (error instanceof DOMException && error.name === 'AbortError') return
  status.value = 'error'
}

Stop then ends the text quietly, and any other failure still sets status to 'error'.

A reader that skips res.ok, which is false when the route answers with an error, shows the error as the answer.

const res = await fetch(url, { method: 'POST', body: JSON.stringify(body), signal: controller.signal })
const reader = res.body!.getReader()

Clear the product name and press Write: the route answers with a 400 error, and the page prints { "error": true, "url": ".../api/draft", "statusCode": 400, "statusMessage": "Validation Error", ... where the description should be. An error response has a body too, and the loop reads it. Check first:

if (!res.ok || !res.body) {
  status.value = 'error'
  return
}

The page then says "The draft could not be written. Try again."

What to remember

• A streamed body is read with getReader() in a loop and decoded with { stream: true }, so characters split between chunks survive.
• The route holds the key, sets instructions and maxOutputTokens, and returns a UI message stream.
• useChat() gives messages, status, stop, error, clearError and regenerate; DefaultChatTransport points it at your route.
• Cancel in both places: stop() when the page unmounts and an abortSignal on the server, or the model writes on and you keep paying.
• Check res.ok before reading, and catch AbortError as the normal end of Stop.