User Satisfaction

User satisfaction is an outcome-oriented measure of how users perceive an application, rather than a property of its underlying foundation model alone. An AI assistant can produce grammatically excellent answers while frustrating users because it is slow, overlooks context, or cannot complete their requests. Conversely, a short answer with clear instructions and an appropriate handoff may produce high satisfaction even when the model cannot resolve the entire task.

Start by defining the intended users, their tasks, and the business objective. For a customer-support assistant, the objective might be to resolve eligible requests accurately while maintaining customer satisfaction and controlling support costs. For an engineering assistant, it might be to help developers complete changes safely with less effort. Evaluation should use representative tasks and users rather than assuming that strong general model benchmarks imply a satisfying application.

Direct measurement commonly uses post-interaction surveys, rating scales, or thumbs-up and thumbs-down feedback. A customer satisfaction score, often called CSAT, can be calculated as the number of satisfied survey responses divided by all valid survey responses, multiplied by 100. Define the scale and threshold explicitly: on a five-point scale, an organization might classify ratings of four or five as satisfied. Record the response rate as well, because a high score from a small, self-selected group may not represent the wider user population.

Behavioral signals provide supporting evidence but do not directly establish satisfaction. Repeated questions, abandoned sessions, requests for human assistance, and user edits can reveal friction. However, abandonment could mean either frustration or successful information retrieval, and escalation may be the correct outcome for a sensitive request. Combine these signals with explicit feedback and verified task outcomes instead of treating any one signal as definitive.

In cloud applications, correlate feedback with a pseudonymous session identifier, model and prompt versions, retrieval configuration, latency, errors, and usage cost. This makes it possible to investigate whether poor ratings come from model responses, unavailable tools, stale retrieved information, or infrastructure delays. Minimize retained personal information, restrict access to interaction data, and apply appropriate retention policies.

User satisfaction, task completion rate, and cost per interaction are complementary business-alignment metrics. Task completion rate measures successful eligible tasks divided by attempted eligible tasks; cost per interaction measures defined operating costs divided by interactions. A cheaper model may lower cost per interaction but increase repeat contacts and reduce satisfaction. Measure cost per successfully completed task as an additional check, and use consistent definitions when comparing alternatives.

Test changes through controlled experiments with comparable user groups and adequate sample sizes. For example, compare two retrieval configurations while tracking satisfaction, verified resolution, response latency, and cost. Do not maximize ratings at the expense of truth or safety: confident misinformation can initially please users, while a necessary refusal can receive a poor rating. Select configurations that improve the intended business outcome while meeting correctness, privacy, and safety requirements.