Task Completion Rate

Task completion rate is an outcome-based metric for evaluating applications, including those powered by foundation models. Calculate it as successfully completed tasks divided by total eligible attempted tasks, multiplied by 100. If an assistant successfully completes 850 of 1,000 eligible tasks, its task completion rate is 85%. The measurement unit is a task, which may involve several messages, model invocations, tool calls, and backend operations.

Define success before collecting results. For an appointment assistant, success might require creating the correct booking, receiving confirmation from the scheduling system, and communicating the confirmed details to the user. A fluent response saying that an appointment was booked is not evidence that the task succeeded. For a summarization application, completion might require producing a summary that satisfies an agreed rubric for factual correctness, required information, and format.

In cloud applications, instrument the complete workflow rather than only the model endpoint. Assign a task identifier and record task initiation, relevant actions, terminal outcomes, and the application version. Verify transactional outcomes against authoritative systems, such as an order database or booking API. For less deterministic tasks, use human review or a validated evaluation rubric. Model-based judges can help scale evaluation, but their judgments require calibration and periodic checks against human assessments.

The denominator must remain consistent. Document which tasks are eligible, how retries are grouped, when abandoned tasks become unsuccessful, and how tasks still in progress are handled. Count a retry within the same task as another attempt to achieve that task's outcome, not automatically as a new user task. Use a defined observation window so recently started tasks do not appear to fail simply because their outcomes have not arrived. If no eligible tasks exist, report the rate as unavailable rather than inventing a percentage.

Distinguish autonomous completion from assisted completion. If a customer-service assistant transfers a case to a person who resolves it, the overall service may have completed the task, but the AI did not complete it independently. Report these outcomes separately when automation is a business objective. Similarly, refusing an unauthorized action is correct safety behavior, but it should not silently inflate the completion rate for legitimate user tasks. Track refusal correctness separately and document how out-of-scope requests are classified.

Task completion rate is not the same as model accuracy, API availability, or response generation success. A model can answer individual questions correctly while failing a multistep workflow because of missing permissions or unreliable tools. An HTTP success response only confirms a technical interaction, not that the user's business goal was achieved. Conversely, task completion does not guarantee satisfaction: users may finish a task but dislike the delay or effort involved.

Use representative offline scenarios before release and production measurements afterward. Segment results by task type, complexity, language, and application version to expose weaknesses hidden by an aggregate percentage. Compare changes using consistent task definitions and report sample sizes. Optimizing completion alone can encourage excessive retries, expensive model calls, or inappropriate actions, so pair it with latency, user satisfaction, safety measures, and cost per interaction. Cost per successfully completed task is also useful because inexpensive interactions can still produce expensive outcomes when many tasks fail.