WebSockets

WebSockets provides a persistent, full-duplex communication channel: the client and server can independently send messages over an established connection. In a typical HTTP/1.1 deployment, the client starts with an HTTP upgrade handshake, after which communication uses WebSocket frames rather than ordinary HTTP request-response messages. Production applications normally use wss://, which protects the connection with TLS. WebSockets supports text and binary messages; application code defines their meaning.

In cloud applications, a managed WebSocket gateway or a load balancer in front of application servers accepts connections and routes traffic. A managed gateway can retain the client connection while invoking short-lived backend handlers for individual messages. This means a persistent client connection does not necessarily require a continuously running function. With self-managed servers, engineers must instead manage connection capacity, draining during deployments, and routing updates to the server holding each connection.

A practical generative AI workflow begins when a browser sends a prompt with a request identifier over its WebSocket connection. A backend worker calls a foundation model streaming API, such as Amazon Bedrock ConverseStream or InvokeModelWithResponseStream, and reads incremental output events. The worker translates those events into application messages containing text deltas, sequence numbers, and the request identifier. It sends an explicit completion or error message when generation ends. Streaming improves perceived responsiveness and time to first visible output; it does not necessarily reduce total generation time.

The model stream and the client connection are separate interfaces. Bedrock streaming responses are not browser WebSocket connections, and a gateway does not automatically translate model events into client messages. A backend must perform that adaptation, preserve relevant metadata, and handle errors after partial output has already reached the user. Clients can send cancellation commands over the same channel, but the application must implement cancellation and stop upstream generation where supported.

WebSockets, server-sent events, and HTTP response streaming are related alternatives, not interchangeable protocols. Server-sent events delivers server-to-client text events over HTTP and is often simpler when the browser only needs generated text updates; prompts can arrive through separate HTTP requests. WebSockets is useful when the same channel must also carry frequent client commands or interactive events. HTTP/1.1 chunked transfer encoding frames an HTTP response of initially unknown length; it does not create a WebSocket connection. When using API Gateway for HTTP response streaming, select an API type and integration that explicitly support streaming rather than assuming every gateway integration forwards chunks immediately. HTTP/2 uses its own framing rather than HTTP/1.1 chunked transfer encoding.

Reliability requires application-level design. Message order is preserved on a live connection, but reconnecting does not automatically replay missed output or guarantee exactly-once processing. Use request identifiers, sequence numbers, bounded replay buffers when needed, and retry-safe handlers. Apply bounded queues and output batching so slow clients do not cause unlimited memory growth. Respect connection lifetimes, idle timeouts, message-size limits, and gateway quotas, and reconnect with exponential backoff and jitter.

Authenticate connection establishment and authorize each sensitive action against the current user and tenant. Validate browser origins, message schemas, and payload sizes; do not treat an open connection as permanent authorization. Avoid exposing cloud credentials to browsers. Monitor connection failures, time to first output, inter-message latency, delivery errors, and abandoned generations. Consider connection-duration and message charges alongside backend inference costs when choosing between WebSockets and simpler one-way streaming.