Streaming
All Octavus responses stream in real-time using Server-Sent Events (SSE). This enables responsive UX with incremental updates.
Stream Response
When you execute a request, you get an async generator of parsed events:
import { toSSEStream } from '@octavus/server-sdk';
// execute() returns an async generator of StreamEvent
const events = session.execute({
type: 'trigger',
triggerName: 'user-message',
input: { USER_MESSAGE: 'Hello!' },
});
// For HTTP endpoints, convert to SSE stream
return new Response(toSSEStream(events), {
headers: {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
Connection: 'keep-alive',
'X-Accel-Buffering': 'no',
},
});
// For sockets, iterate events directly
for await (const event of events) {
conn.write(JSON.stringify(event));
}The X-Accel-Buffering: no header disables proxy buffering on Nginx-based infrastructure (including Vercel), ensuring SSE events are forwarded immediately instead of being batched.
Heartbeat
toSSEStream automatically sends SSE comment lines (: heartbeat) every 15 seconds during idle periods. This prevents proxies and load balancers from closing the connection due to inactivity - particularly important during multi-step executions where the stream may be silent while waiting for tool processing or LLM responses.
Heartbeat comments are ignored by all SSE parsers per the spec. No client-side handling is needed.
Event Types
The stream emits various event types for lifecycle, text, reasoning, and tool interactions.
Lifecycle Events
// Stream started
{ type: 'start', messageId: '...', executionId: '...' }
// Stream completed
{ type: 'finish', finishReason: 'stop' }
// Possible finish reasons:
// - 'stop': Normal completion
// - 'tool-calls': Waiting for server tool execution (handled by SDK internally)
// - 'client-tool-calls': Waiting for client tool execution
// - 'length': Max tokens reached
// - 'content-filter': Content filtered
// - 'error': Error occurred
// - 'other': Other reason
// Error event (see Error Handling docs for full structure)
{ type: 'error', errorType: 'internal_error', message: 'Something went wrong', source: 'platform', retryable: false }A generation the model did not finish is never delivered as a finish: a safety refusal or a response cut off at the output token limit surfaces as an error event (provider_error with code CONTENT_FILTER or OUTPUT_TRUNCATED) after the configured backup model, if any, has had its attempts - see maxOutputTokens.
Block Events
Track execution progress:
// Block started
{ type: 'block-start', blockId: '...', blockName: 'Respond to user', blockType: 'next-message', display: 'stream', thread: 'main' }
// Block completed
{ type: 'block-end', blockId: '...', summary: 'Generated response' }Step Events
A block that calls tools runs several model steps in one turn. Step events mark the boundaries between them and let the platform retract a step it has to redo:
// A later model step is starting (never sent for a block's first step).
// The live counterpart of the `step-start` message part, so a message built
// from the stream has the same structure as one loaded from history.
{
type: 'step-start';
}
// The current step's partial output is abandoned: drop everything streamed
// since the step began (after the last `step-start`, or since the block's
// output began). A fresh attempt at the same step streams next.
{
type: 'step-discard';
}step-discard is sent when the platform rolls a step back to re-issue it - for example when a transient provider failure interrupted the stream before anything from the step had executed. Only the interrupted step is affected; output committed by earlier steps stays. A client that ignores the event keeps the abandoned output and sees the retried step appended after it, which is why the Client SDK handles it for you.
Persisted Marker
A turn that pauses for tools runs in rounds: each tool-request pauses the execution, and the matching continuation resumes it. The platform saves the session's state at the end of every round and marks the stream so a consumer knows how far the saved state reaches:
// Everything streamed before this event is now in saved session state.
// Sent once per round, after the round's state write and before the
// tool request that pauses the execution.
{
type: 'persisted';
executionId: string;
round: number; // rounds of this execution saved so far (1 after the first)
messageId?: string; // the in-progress assistant message the round extended
}The whole execution - every round until it finishes - is one assistant message in session state, so a message built from the stream has the same shape as the message a later session read returns. Each round's start event carries the id of the message it extends as messageId (see the Sessions API for the one case where a turn continues in a new message). A consumer that renders a running session from saved state can use the marker to tell which part of the live stream its copy already includes; the Client SDK uses it to resume an in-progress message rather than rebuilding it.
Text Events
Streaming text content:
// Text generation started
{ type: 'text-start', id: '...' }
// Incremental text (most common event)
{ type: 'text-delta', id: '...', delta: 'Hello! How' }
{ type: 'text-delta', id: '...', delta: ' can I help?' }
// Text generation ended
{ type: 'text-end', id: '...' }Delta granularity is provider-dependent. Each text-delta is forwarded exactly as the model provider frames it. Some providers emit small, word-sized deltas; others (notably some Claude models) emit larger multi-token chunks, so text can arrive in lumps rather than a steady token-by-token stream. Do not assume a fixed delta size. To render text as smooth, continuous typing regardless of provider framing, see Delivery Cadence below.
Reasoning Events
Extended reasoning (for supported models like Claude):
// Reasoning started
{ type: 'reasoning-start', id: '...' }
// Reasoning content
{ type: 'reasoning-delta', id: '...', delta: 'Let me analyze this request...' }
// Reasoning ended
{ type: 'reasoning-end', id: '...' }Delivery Cadence
Raw text-delta (and reasoning-delta) granularity is set by the model provider, so it can be coarse - large multi-token lumps rather than a steady stream. Octavus gives you two independent, opt-in ways to get smooth, continuous rendering. They can be used alone or together:
- Client-side smoothing (recommended for most UIs). If you render through
@octavus/client-sdk/@octavus/react, enable the built-in typewriter pacer. It paces already-received text onto the screen at a steady cadence with no wire cost and no added latency (it is flushed the moment a turn finishes). See Client SDK -> Streaming. - Server-side (wire) smoothing. If you consume the raw SSE stream in your own backend and render it elsewhere, you can have Octavus re-chunk the stream before it reaches you by declaring a streaming cadence in your agent config. This normalizes every provider uniformly.
Server-side cadence (agent config)
Declare a streaming cadence on the agent (or a thread / a next-message block). When set, the runtime re-chunks visible text and reasoning into word- or line-sized deltas before sending them:
agent:
model: anthropic/claude-sonnet-4-5
system: system-prompt
streaming:
chunking: word # off (default) | word | line
delayMs: 15 # optional inter-chunk delaychunking: off(the default) forwards provider deltas unchanged - identical to today's behavior, no extra frames.chunking: word/lineemit more, smaller SSE frames so downstream rendering looks like continuous typing. This is the expected trade-off of finer wire granularity: more (smaller) events over the wire.delayMsis an optional pause between emitted chunks, an integer between 0 and 100. Keep it small so a burst drains within the model's natural idle gap (avoids adding latency on fast models). Omit it for the built-in default.
Cadence is resolved with the standard block > thread > agent precedence, the same as speed. It does not apply to structured output (responseType) blocks, whose deltas carry JSON rather than prose.
Tool Events
Tool call lifecycle:
// Tool input started
{ type: 'tool-input-start', toolCallId: '...', toolName: 'get-user-account', title: 'Looking up account' }
// Tool input/arguments streaming
{ type: 'tool-input-delta', toolCallId: '...', inputTextDelta: '{"userId":"user-123"}' }
// Tool input streaming ended
{ type: 'tool-input-end', toolCallId: '...' }
// Tool input is complete and available
{ type: 'tool-input-available', toolCallId: '...', toolName: 'get-user-account', input: { userId: 'user-123' } }
// Tool output available (success)
{ type: 'tool-output-available', toolCallId: '...', output: { name: 'Demo User', email: '...' } }
// Tool output error (failure)
{ type: 'tool-output-error', toolCallId: '...', error: 'User not found' }Resource Events
Deprecated: Resources (and resource-update events) are superseded by tools. Persist state with a consumer-defined tool instead. Still emitted for now.
Resource updates:
{ type: 'resource-update', name: 'CONVERSATION_SUMMARY', value: 'User asked about...' }Display Modes
Each block/tool specifies how it should appear to users:
| Mode | Description |
|---|---|
hidden | Not shown to user (background work) |
name | Shows block/tool name |
description | Shows description text |
stream | Streams content to chat |
Note: Hidden events are filtered before reaching the client SDK. Your frontend only sees user-facing events.
Stream Event Type
type StreamEvent =
// Lifecycle
| StartEvent
| FinishEvent
| ErrorEvent
// Text
| TextStartEvent
| TextDeltaEvent
| TextEndEvent
// Reasoning
| ReasoningStartEvent
| ReasoningDeltaEvent
| ReasoningEndEvent
// Tool Input/Output
| ToolInputStartEvent
| ToolInputDeltaEvent
| ToolInputEndEvent
| ToolInputAvailableEvent
| ToolOutputAvailableEvent
| ToolOutputErrorEvent
// Octavus-Specific
| BlockStartEvent
| BlockEndEvent
| ResourceUpdateEvent
| ToolRequestEvent
| ClientToolRequestEvent;Client Tool Request
When a tool has no server handler registered, the SDK emits a client-tool-request event:
{
type: 'client-tool-request',
executionId: 'exec_abc123', // Use this to continue execution
toolCalls: [ // Tools for client to handle
{
toolCallId: 'call_xyz',
toolName: 'get-browser-location',
args: {}
}
],
serverToolResults: [ // Results from server tools in same batch
{
toolCallId: 'call_def',
toolName: 'get-user-account',
result: { name: 'Demo User' }
}
]
}After the client handles the tools, send a continue request with all results:
session.execute({
type: 'continue',
executionId: 'exec_abc123',
toolResults: [
...serverToolResults, // Include server results from the event
{
toolCallId: 'call_xyz',
toolName: 'get-browser-location',
result: { lat: 40.7128, lng: -74.006 },
},
],
});See Client Tools for full client-side implementation.
Error Events
Errors are emitted as structured events with type classification:
{
type: 'error',
errorType: 'rate_limit_error', // Error classification
message: 'Rate limit exceeded', // Human-readable message
source: 'provider', // 'platform' | 'provider' | 'tool'
retryable: true, // Whether retry is possible
retryAfter: 60, // Seconds to wait (rate limits)
code: 'ANTHROPIC_429', // Machine-readable code
provider: { // Provider details (when applicable)
name: 'anthropic',
statusCode: 429,
requestId: 'req_...'
},
backup: { // How the backup model failed too (when one was tried)
model: 'openai/gpt-5.5',
errorType: 'rate_limit_error',
message: 'You have no credits remaining. ...',
code: 'PROVIDER_ACCOUNT_ERROR'
}
}retryable is the platform's verdict once its own recovery (the configured backup model, same-provider retries) is spent: whether a fresh run of the turn can succeed. backup is present only when the agent configures a backup model and it failed as well; the event itself describes the primary model's failure.
Error Types
| Type | Description |
|---|---|
rate_limit_error | Too many requests |
authentication_error | Invalid API key |
provider_error | LLM provider issue |
provider_overloaded | Provider at capacity |
tool_error | Tool execution failed |
internal_error | Platform error |
Tool Errors
Tool errors are captured per-tool and don't stop the stream:
{ type: 'tool-output-error', toolCallId: '...', error: 'Handler threw exception' }The stream always ends with either finish or error, with one exception: worker executions emit a final usage event (the per-execution cost summary) after finish, just before the stream closes. See Cost reporting.
For client-side error handling patterns, see Error Handling.