Stop Sequences | Cloud Concepts

What is Stop Sequences

Stop sequences define an explicit boundary for generated output, helping applications prevent unwanted continuations and avoid unnecessary output tokens. They complement output-token limits and concise prompting, but require careful delimiter selection and model-specific validation to avoid cutting off useful content.

Stop sequences are inference controls that terminate text generation when the generated output matches a configured pattern. For example, an application can instruct a model to finish its answer with the marker END_RESPONSE and configure that marker as a stop sequence. When the inference system recognizes the marker, it stops generating further text. Many APIs omit the matched sequence from the returned response, but exact behavior, supported sequence counts, and length restrictions depend on the model and endpoint.

In cloud applications, stop sequences are typically supplied as inference request parameters. They are useful when a prompt establishes a clear output boundary, such as the end of a classification result or the transition to another speaker in a text-based conversation template. Merely mentioning a stopping rule in the prompt is not equivalent to configuring an inference-level stop sequence: prompt instructions depend on model compliance, while a supported stop parameter is enforced by the serving system when a match occurs.

Consider a support application that asks for a short answer followed by END_RESPONSE. Without a configured stop sequence, the model might continue with additional examples or commentary after that marker. Stopping at the marker can avoid those extra tokens and reduce completion latency. However, the model must actually produce the marker for this mechanism to help. Set a maximum output-token limit as a separate backstop, and prompt for the required level of detail rather than relying on stopping alone.

Delimiter selection is a correctness decision. Common strings such as a period, a newline, or the word Done can appear in legitimate answers and cause premature termination. Prefer a distinctive marker that is unlikely to occur naturally, and test it against realistic inputs, multilingual text, code, and quoted material. Matching is generally literal rather than semantic; do not assume regular-expression support, automatic case normalization, or that one configured string corresponds to one tokenizer token.

Stop sequences control the output side of token efficiency. They do not compress prompts, remove irrelevant retrieved passages, or expand the context window. Combine them with input-token estimation, prompt compression, context pruning, and output-token budgeting. Reserve enough context capacity for the expected answer, and use actual usage measurements to assess savings. Shortening displayed text after generation does not undo inference work or output-token charges already incurred.

Streaming introduces another implementation concern: a stop pattern can span multiple streamed chunks. Prefer provider-side stopping when available. A client-side matcher needs a rolling buffer to avoid exposing a partial delimiter, and closing a client connection does not necessarily guarantee immediate server-side cancellation or reduced billing.

Monitor output-token usage, latency, stop reasons, completeness, and retry rates together. A lower token count is not an improvement if truncated answers cause additional requests. Stop sequences are neither a security boundary nor a substitute for structured-output validation; stopping at a closing brace, for example, can truncate nested JSON. Validate required fields and response syntax, distinguish configured stopping from token-limit exhaustion where the API permits, and test each model or endpoint configuration before deployment.

Key aspects of Stop Sequences

Effective stop sequences combine explicit completion boundaries, careful delimiter selection, compatible inference controls, and quality-aware token measurement.

Explicit generation boundaries

A configured pattern ends generation when the serving system recognizes it in generated output. Use a delimiter aligned with the task’s natural completion point.

Complementary output controls

Combine stop sequences with a maximum output-token limit and concise response instructions. The sequence provides a content-based boundary, while the token limit caps generation even if the delimiter never appears.

Collision-resistant delimiters

Choose markers unlikely to appear in valid answers. Test ordinary text, code, quoted documents, and multilingual responses to detect accidental early stopping.

Model-specific behavior

Check whether the selected model supports custom stop sequences and how its API handles matching, returned delimiters, and stop reasons. Do not assume settings are portable unchanged between endpoints.

Measured token efficiency

Track actual output-token usage alongside answer completeness, latency, and retries. Savings depend on avoiding otherwise-generated output, not simply removing text from the user interface.

Streaming-aware enforcement

Prefer server-side stop handling to client-side truncation. If matching streamed text locally, retain enough buffered text to detect patterns split across chunks and verify cancellation behavior separately.

Cloud services for Stop Sequences

Managed inference services such as Amazon Bedrock and Vertex AI can enforce supported stop sequences within the generation request, while SageMaker AI allows engineers to deploy a serving stack with its own stopping implementation. An application should map its intended completion boundary to each endpoint’s supported configuration rather than assume identical behavior across providers. Pair stopping with output-token caps, input-context optimization, and response validation. Collect usage, latency, termination reasons, and retry rates through application telemetry to determine whether the configuration reduces total cost without sacrificing completeness. For streaming responses, test delimiter handling and cancellation separately from nonstreaming requests.

Amazon Bedrock

Amazon Bedrock exposes stopSequences in the Converse inference configuration for models that support the setting; native invocation parameters vary by model. Combine supported stop sequences with maxTokens, and inspect response usage and stopReason to evaluate termination and token efficiency.

Open Amazon Bedrock →

Amazon SageMaker

Models hosted on SageMaker AI endpoints can implement stop sequences through their inference container or model-serving framework. Request fields, matching behavior, and cancellation semantics depend on the deployed serving stack rather than a universal endpoint-level stop-sequence setting.

Open Amazon SageMaker →