AIF-C01FOUNDATIONAL

AWS Certified AI Practitioner (AIF-C01) Course

Understand AI, ML and generative AI on AWS for AIF-C01, from foundation models to responsible AI, in short lessons with exercises and no videos.

AIF-C01 examcurrent
Domains5
Tasks14
Difficulty1 / 5
ReleaseOct 22, 2024 Last dayNot announced
All exam dates →
246 lessons
Updated
Start learning free
COURSEInteractive AIF-C01 courseLessons built around exercises with feedback on every answer, no videos.Start learning →PRACTICE EXAMAIF-C01 exam simulatorCheck your readiness with full-length or custom practice exams.See the practice exam →EXAM GUIDEOfficial AIF-C01 exam guideThe AWS document listing every domain, task and service in scope.Open the guide →
AT A GLANCE

Inside this course

246lessons
849exercises
6free lessons
40services
139concepts

Exercises you will solve

Sort the statements413
Fill the blanks232
Put the steps in order95
Trace a request36
Build the diagram36
Find the flaw24
Complete the diagram12
Pick the services1

Services with the most lessons

  • Amazon Bedrock10
  • Amazon SageMaker6
  • Amazon Transcribe3
  • AWS Config3
  • Amazon Aurora2
  • Amazon Bedrock AgentCore2
  • Amazon Comprehend2
TRY IT NOW

Try a lesson

Amazon QuickSight: business insights from AI/ML results

Restore the presentation service

Fill the gap with the service where an author prepares the dataset, creates visuals, and publishes an interactive business dashboard.

Prediction recordsDataset rowsInteractive dashboard
Amazon SageMaker
Amazon S3
Users
ONE LESSON, STEP BY STEP

How a lesson works

  1. ExplanationFrom model results to business insights
  2. Sort the statementsSeparate prediction from presentation
  3. ExplanationFollow the results into a dashboard
  4. Trace a requestTrace a prediction to its readers
  5. ExplanationMake rows useful to business readers
  6. Complete the diagramRestore the presentation serviceThe exercise you just tried
  7. ExplanationExplore results without changing the model
  8. Build the diagramRebuild the results-to-insights flow
  9. ExplanationWhat to remember
  • 5 to 10 short steps, one at a time.
  • Feedback at once: an exercise says what was wrong as soon as you answer.
  • No videos: you read and solve at your own pace.
WHAT THE EXAM COVERS

Course scope

The course follows the official AIF-C01 exam guide and covers all 5 domains and 14 tasks.

DOMAIN 120% of the exam · 75 lessons
Fundamentals of AI and ML
Explain basic AI concepts and terminologies2 free26 lessons
Machine learning: models that learn from dataFreeHow training produces models for inference, what fit and bias mean, and how machine learning relates to AI, deep learning, neural networks, and agentic systems.

Machine learning (ML) learns patterns from data instead of requiring a person to write every decision rule. To recognize unwanted email, for example, we can learn from examples of unwanted and legitimate messages. A model represents the learned patterns and uses them to turn inputs into outputs.

Neural Networks vs Machine Learning: model family and broader approachWhy neural networks are one family within machine learning, how other model families differ, and where deep learning fits in this relationship.

Machine learning is an approach within artificial intelligence in which models learn patterns from data rather than relying entirely on hand-written decision rules. It is a broad category, not the name of one model design. A model family is a group of models that share a basic structure, such as decision trees or neural networks.

Neural Networks: how a network learns an input-to-output mappingHow layers use weights, biases, and activation functions to transform inputs into outputs, with training adjusting parameters and inference using them.

A neural network is a mathematical model that transforms numerical inputs into outputs. For example, it can take numbers representing image pixels and produce a score for a manufacturing defect. Learning an input-to-output mapping means learning how those input values should influence the result, rather than storing a separate answer for every possible image.

Amazon Rekognition: computer vision with pretrained modelsHow Rekognition uses pretrained deep learning models to infer labels from images and videos, illustrating computer vision, neural networks, and the distinction between training and inference.

Computer vision is the field of AI that extracts information from images and videos. A computer receives numbers representing pixels, not an understanding of what a picture shows. Computer vision connects those visual patterns to useful descriptions.

Amazon Comprehend: natural language processing with pretrained modelsHow Comprehend uses trained machine learning models to infer structured insights from unstructured text without requiring users to train a model.

Amazon Comprehend is a natural language processing service. Natural language processing (NLP) applies computing techniques to human language. Comprehend uses machine learning to analyze text such as reviews, messages, and documents.

Machine learning and generative AI: learning method versus content generationWhy learning from data and generating content describe overlapping categories rather than interchangeable terms or competing alternatives.

Machine learning (ML) describes learning patterns from data rather than relying only on hand-written rules. It answers a question about how a model gains its capabilities. A model can learn from past emails to recognize unwanted messages, for example.

Foundation Models vs. traditional machine learningHow broadly pretrained foundation models differ from task-specific models, while remaining part of machine learning rather than an alternative to it.

Machine learning produces models that learn patterns from data. A foundation model is a machine learning model with broad initial training that provides a starting point for multiple applications. Foundation models are therefore part of machine learning, not an alternative to it.

Amazon Comprehend and Amazon Bedrock: NLP analysis versus generative AIHow Comprehend's specialized text analysis differs from the generative foundation model capabilities available through Bedrock.

Natural language processing (NLP) is AI concerned with human language. An NLP task might identify the sentiment in a review or produce a summary of it. Both involve language, but they ask for different kinds of results: an analysis of existing text versus newly generated content.

Bias and Fairness: understanding unequal outcomesHow systematic bias can disadvantage people or groups, and how fairness criteria help judge whether those differences are unjustified in context.

In AI, systematic bias means patterns in data, predictions, or decisions that can disadvantage particular people or groups. A speech recognition model may repeatedly make more errors for speakers with certain accents. That pattern matters because it can make the service less useful or accessible to those speakers. One isolated mistake does not, by itself, establish systematic bias.

Generative AI: content generation from learned patternsHow generating content from learned patterns and supplied context fits within AI, machine learning, and deep learning, rather than defining those broader fields.

Artificial intelligence (AI) is the broad field of building systems that perform tasks associated with intelligence, such as reasoning, perception, planning, and language processing. Producing content is one possible use of AI, not its defining purpose. AI also includes systems that follow explicitly programmed rules.

Agentic AI: goal-directed actionHow agents choose actions, use tools, and react to results to complete real-world tasks within defined limits.

An AI agent pursues a goal by choosing and taking actions. A goal describes the desired outcome; an action is a step toward it. For example, “help this customer request an eligible return” is a goal. Looking up the order is one possible action, not the whole task.

Agentic AI and Generative AI: acting versus generatingHow goal-directed action differs from content generation, and why agentic AI describes system behavior rather than a model architecture or learning method.

Generative AI produces content, such as text, images, or code. A system might draft a refund explanation or write a plan for arranging a meeting. The output is content: writing a plan does not mean the system has carried it out.

Batch Inference: production prediction jobsFreeHow managed or self-hosted batch jobs apply a trained model to collected inputs and save predictions for later use, independently of API access or serverless hosting.

Batch inference applies a trained model to a collected dataset and saves predictions for later use. For example, a job can score customer records for a campaign. The campaign uses the saved scores after the job finishes.

Online Inference: serving application requestsHow deployed models handle on-demand requests through synchronous or asynchronous responses, with serverless hosting independent of response timing.

Online inference means using a deployed model to handle incoming application requests on demand. The application sends input to a model-serving endpoint, an address through which it can request a prediction. The output might be a fraud score, a product ranking, or a generated answer. Running inference normally does not retrain the model.

Real-Time Inference vs. Batch InferenceHow response-time requirements distinguish interactive prediction requests from offline processing of accumulated inputs.

Inference means using an already-trained model to produce a prediction or generated output. Real-time and batch inference both do this; neither is model training. Their main difference is when the result is needed.

Batch Inference vs Online InferenceHow response-time requirements and prediction freshness distinguish batch workloads from interactive online predictions.

Inference means applying a trained model to input to produce an output, such as a recommendation or a risk score. Batch inference processes a collected dataset as a job, with predictions available for later use. Online inference processes incoming application requests on demand.

Serverless: on-demand inference computeHow serverless inference shifts compute management to the provider, with service-specific scaling, usage charges, and startup latency rather than a particular response timing or batching model.

Inference means using a trained model to produce an output from an input, such as predicting a category for an image. Serverless inference runs that work without requiring you to provision and maintain individual compute servers.

Structured Data: schemas and recordsHow consistent schemas organize tabular and time-series records into defined fields, independently of storage location or whether the data is labeled.

Structured data organizes information into defined fields with consistent meanings. A schema describes those fields and their data types, such as text, number, or timestamp. This gives applications a shared way to interpret the data.

Structured Data vs Unstructured DataHow predefined fields differ from free-form content, and why storing text or image references in a table does not structure the underlying content.

Structured data organizes information into predefined fields with consistent meanings and types. For example, each product record might have a product identifier, a price, and a category. You can compare prices directly because the price field has the same meaning in every record.

Unstructured Data: inputs to AI modelsHow text, images, and audio become inputs to AI models, with or without target labels.

Unstructured data includes the meaningful content of reviews, photographs, and recorded conversations. These examples have different modalities: text, images, and audio. Modality means the form in which information is expressed.

Data Labeling: recognizing task-specific labelsHow prediction targets and annotations make tabular, time-series, image, and text examples labeled for a particular task, independently of whether the data is structured or unstructured.

A machine learning task defines what a model should predict. The information given to the model is its input; the answer it is meant to predict is the prediction target. In supervised learning, training examples include known targets so the model can learn relationships between inputs and answers.

Machine learning: learning approachesHow learning from labeled examples, unlabeled data, or rewards leads to different capabilities, including classification, regression, and clustering.

Supervised learning learns from examples that pair input information with a known answer, called a label. For example, an email can be paired with its known category: unwanted or legitimate. These answers guide learning so the model can predict answers for new inputs.

Supervised Learning: predictions from labeled examplesHow models learn from features paired with known targets to predict categories through classification or numeric quantities through regression.

Supervised learning learns from examples that pair inputs with known answers. The inputs are described by features. The answer the model should learn to predict is the target, also called a label. A labeled example contains both.

Supervised Learning vs. Unsupervised LearningHow the availability of known target values distinguishes supervised prediction from unsupervised pattern discovery, including classification versus clustering.

Both approaches learn from data, but they learn different things. Supervised learning uses examples that pair inputs with known outputs. It learns a relationship that can help predict an output for a new input. Unsupervised learning looks for structure in the inputs without being given target outputs to learn from.

Reinforcement Learning: learning through rewardsHow an agent improves its policy through actions and reward feedback to pursue long-term outcomes while balancing exploration and exploitation.

Reinforcement learning trains a decision-maker through the consequences of its actions. The agent is the decision-maker, and the environment is the world it interacts with. For a delivery robot, the agent controls the robot and the environment includes the warehouse, obstacles, and delivery locations.

Reinforcement Learning vs Supervised LearningHow reward feedback from actions differs from labeled training examples, and why that distinction determines which learning approach fits a problem.

Supervised learning learns from examples that pair an input with a known answer, called a label or target. For example, a support ticket can be paired with its correct department. Training compares the model's predicted department with that supplied answer and adjusts the model to reduce prediction error.

Identify practical use cases for AI21 lessons
Machine learning: real-world applicationsHow learned patterns turn inputs into useful outputs in vision, language, speech, recommendations, fraud detection, and forecasting, including ML's contribution to knowledge retrieval and agentic applications.

Machine learning applications use learned patterns to turn new inputs into useful outputs. In computer vision, the input is an image or video. Patterns in visual features help a model identify objects, locate damage, or recognize printed text. For example, a package photograph can become a list of detected objects or an extracted shipping address.

Machine learning: choosing a suitable model-based solutionHow prediction uncertainty, business value, explainability, regulation, and operational constraints determine whether rules, traditional ML, or foundation models suit a problem.

A suitable solution starts with the required behavior, not with a model. Rules suit decisions whose logic is already known: calculate a delivery charge from an approved price list, or reject a form with a missing required field. These tasks require following a specification, not predicting an unknown answer.

Human-in-the-Loop: meaningful human reviewHow people use evidence to assess, challenge, and correct AI outputs in assisted decisions and responsible AI audits.

Human-in-the-Loop means people deliberately participate in an AI workflow rather than treating its output as automatically authoritative. AI can help by sorting work, extracting information, or recommending an action. A person then applies evidence and judgment where the workflow calls for review.

Amazon Textract: extracting data from documentsHow Textract extracts printed text, handwriting, and structured form and table data to automate manual data entry while retaining human review where needed.

Amazon Textract is a managed machine learning service that extracts text and data from documents, including scanned pages, images, and PDF files. Instead of a person reading each page and retyping its contents, an application can use Textract’s output.

Determinism: repeatability versus exact requirementsWhy repeatable outputs do not guarantee correctness, and when exact, rule-defined requirements call for conventional code rather than ML predictions.

Determinism means that identical inputs, initial state, and execution conditions produce identical results. Think of a calculator applying a fixed tax rate and rounding rule: with everything held constant, it returns the same total.

AWS Lambda: rule-based event processingHow event-triggered Lambda functions run explicit business rules, and how their execution costs inform choosing conventional code over AI for prescribed outcomes.

AWS Lambda runs your code without requiring you to provision or manage servers. An event is data describing something that happened, such as an order being placed. A configured trigger causes Lambda to run your function with that event.

Cost-Benefit Analysis: evaluating AI against simpler alternativesHow to weigh AI's measurable benefits against lifecycle and error costs, using simpler alternatives as a baseline and recognizing when exact outcomes make prediction unsuitable.

Cost-benefit analysis asks whether an option creates enough value to justify its resources and risks. For AI, the useful question is not just “Does it work?” but “Does it improve on a simpler way of doing the job?”

Unsupervised Learning: discovering groups with clusteringHow clustering discovers groups from similarities in unlabeled data and why those groups require interpretation before use.

Clustering is an unsupervised learning technique that discovers groups of similar records. It uses unlabeled data: examples without supplied target categories for the model to learn. Named columns do not make data labeled; a customer dataset can contain purchase frequency and average order value without containing customer segment labels.

Natural Language Processing: turning language into useful informationHow NLP uses the meaning of human language to power applications such as ticket routing, sentiment analysis, and document summarization.

Natural language processing (NLP) applies computational techniques to human language. It helps applications turn messages, reviews, and documents into useful information. Language is not a set of tidy database fields: context, spelling variations, and phrases such as “not bad” affect what words mean.

Amazon Personalize: personalized recommendationsHow Amazon Personalize learns from user interactions to recommend and rank relevant products or content for each user.

Amazon Personalize is a managed machine learning service that helps applications recommend products or content relevant to individual users. Instead of developers writing a separate set of recommendation rules for every person, the service learns patterns from data. A shopping app can suggest products, while a video app can suggest something to watch.

Amazon Bedrock: Knowledge BasesHow Knowledge Bases retrieves relevant source material to ground model responses without retraining the model.

A foundation model may write fluent answers without knowing your organization’s current policies or product manuals. Knowledge Bases for Amazon Bedrock connects those sources to model requests so answers can use information beyond the model’s training.

Amazon SageMaker: the managed ML lifecycleHow SageMaker AI supports the path from preparing training data to training and deploying a custom model within the broader SageMaker platform.

Amazon SageMaker is the broader platform for data, analytics, and AI. SageMaker Unified Studio brings together tools for working with data and building AI applications. Within that platform, Amazon SageMaker AI provides managed capabilities for preparing data, building and training models, and deploying them. SageMaker AI is also available as a standalone service.

Amazon SageMaker: foundation models with JumpStartHow JumpStart provides pretrained models that learners can select, customize, and deploy using SageMaker AI.

A foundation model is a model trained on broad data that can be adapted to different tasks. Starting with a pretrained model means you do not have to build and train that model from scratch. Depending on its capabilities, it can help with tasks such as generating text, summarizing documents, or creating images.

Amazon Bedrock and Amazon SageMaker: choosing the serviceHow to distinguish Bedrock's managed foundation-model application capabilities from SageMaker's broader model development and hosting capabilities.

Amazon Bedrock provides managed access to foundation models: models pretrained on broad datasets that can support tasks such as generating text, summarizing documents, or creating images. Supported tasks depend on the model you choose.

Amazon Transcribe: converting speech into textHow batch and streaming transcription turn speech into readable text with timestamps and speaker labels for uses such as captions and meeting transcripts.

Amazon Transcribe is a managed automatic speech recognition service. It turns speech in audio into text, adding punctuation to help make the result readable. You use the service without building or training your own speech recognition model.

Amazon Translate: translating text across languagesHow managed neural machine translation supports multilingual content through real-time requests and batch document jobs, and when human review is needed.

Amazon Translate is a managed service that translates text from a source language into a target language. It helps make product descriptions, support messages, websites, and documents accessible to people who use different languages.

Amazon Transcribe and Amazon Translate: transcription versus translationHow converting speech into text differs from translating text between languages, and which service fits each need.

Transcription turns speech into written words. Amazon Transcribe performs this speech-to-text task: for example, English speech becomes an English transcript. This makes spoken information available to read rather than listen to.

Amazon Lex: voice and text conversational botsHow Lex uses intents, utterances, and slots to guide voice or text conversations toward fulfilling user requests.

Amazon Lex is an AWS service for building conversational bots that accept voice or text. It provides speech recognition for spoken input and natural language understanding to help determine what the user wants. A bot can guide a conversation toward a task, rather than simply return the user's words.

Amazon Lex vs Amazon Transcribe: conversations or transcriptionWhen to choose Lex for interactive conversations rather than Transcribe for converting speech into text.

Amazon Lex helps you build interactive conversations using voice or text. Choose it when an application needs to understand a user's request and respond as part of a back-and-forth exchange.

Model Explainability: explanation requirements and model choiceHow intrinsic interpretability and the limits of post-hoc explanations influence whether traditional ML or foundation models can meet a use case’s explanation requirements.

Model explainability helps people understand why a model produced an output and what patterns influence its behavior. Before choosing a model, identify who needs the explanation and what they must understand. An engineer investigating errors needs different detail from a person asking why an application was declined.

Foundation Models: workload suitabilityHow reusable pretrained capabilities fit a workload’s data and outputs while meeting explainability, regulatory, latency, and cost requirements.

A foundation model is broadly pretrained, so an application can reuse capabilities such as understanding language, recognizing visual patterns, or producing text. This makes it a useful candidate for summarizing conversations, drafting descriptions, or extracting information from varied documents. Not every foundation model accepts the same inputs or produces the same outputs: check its supported modalities, such as text, images, or audio.

Describe the AI/ML development lifecycle28 lessons
Train-Validation-Test Split: dataset roles and separationHow separate dataset partitions support model training, guide development decisions, and preserve an independent final evaluation in an ML pipeline.

A train-validation-test split divides a dataset into separate collections of examples called partitions. Each partition has a different job. Keeping them separate helps distinguish learning from examples from performing well on examples not used for learning.

Feature Engineering: turning data into model inputsHow feature engineering bridges data preparation and model training by transforming raw attributes into useful inputs that can also be produced at prediction time.

A feature is an input variable a model uses. A label is the outcome a supervised model learns to predict. For customer churn prediction, features might describe subscription length and recent usage; the label records whether the customer later canceled.

Hyperparameter Tuning: selecting training configurationsHow tuning compares configurations through repeated training and validation to select a candidate for final evaluation on untouched test data.

Hyperparameter tuning compares training configurations to find a promising candidate model. A configuration is a set of choices that controls training or the model's structure, such as a learning rate or a decision tree's maximum depth.

Pre-Training: building a reusable foundation modelHow learning from broad datasets creates foundational capabilities, and why reusing pre-trained weights avoids the cost of building those capabilities from scratch.

Pre-training is the learning stage that builds a model's foundational capabilities. A model repeatedly processes examples and adjusts its weights: numerical parameters that shape its predictions. The learned weights are a central part of what training produces, not just settings for accessing the original dataset.

Open-Source Models: understanding model releasesHow a pretrained model release’s artifacts, documentation, and licenses establish what can be reused and inspected, without guaranteeing explainability.

A pretrained model has already learned patterns from training data. Reusing it provides a starting capability without training from scratch. Research institutions, community repositories, model hubs, and commercial publishers distribute particular versions as model releases.

Fine-Tuning: adapting a pretrained modelHow a supported pretrained model becomes a reusable task-specific adaptation through targeted data, parameter updates, evaluation, and deployment.

Fine-tuning adapts a pretrained model to a more specific task using targeted data. It updates learned parameters, or trains additional adaptation parameters, to produce a reusable adaptation. For example, a support team could adapt a model to turn customer messages into consistently structured issue summaries.

Fine-Tuning vs. Pre-TrainingHow adapting existing capabilities differs from building broad capabilities or extending them through continued pretraining, especially in objectives, data, and compute requirements.

A model’s parameters, often called weights, are the learned values that shape its responses. Pre-training from scratch starts with newly initialized parameters and learns broad capabilities from a large, varied dataset. For a language model, these capabilities include patterns in language, code, and other training text.

APIs: model inference endpointsHow applications send inputs and receive predictions through an API contract, and who operates the endpoint when it is managed versus self-hosted.

Inference means using a trained model to produce an output from new input. The output might be a prediction, a classification, or generated text. An inference endpoint is a network address through which an application requests that work, commonly over HTTPS.

Real-Time Inference: managed and self-hosted APIsHow trained models serve application requests through managed or self-hosted APIs, and how operational responsibilities differ.

Inference uses a trained model to produce a prediction or generated output. An application sends input to an API endpoint, an address that accepts requests, and receives the result. Receiving a request does not automatically train or update the model.

Amazon Bedrock: managed model inferenceHow Bedrock provides managed foundation-model inference and how inference capacity and pricing choices affect cost and responsiveness.

Amazon Bedrock provides managed access to foundation models: models already trained on broad data that can perform tasks such as generating text, summarizing, creating embeddings, or generating images. Inference means using a model to produce a result from an input; it is different from training the model.

Amazon EC2: self-hosted model inferenceHow a model-serving application runs on an EC2 instance and exposes an inference API using CPU or GPU capacity you manage.

Amazon EC2 provides virtual machines called instances. With self-hosted inference, you run a model-serving application on an instance. Inference means using a trained model to produce an output from new input; it is not training the model.

Amazon EC2 vs. Amazon SageMaker: model hostingHow infrastructure control and operational responsibilities differ between self-hosting models on EC2 and using SageMaker managed inference endpoints.

Model hosting means running a trained model so an application can send input and receive a prediction. This use of a model is called inference. An endpoint is the address an application calls to request that prediction.

Kiro: specification-driven developmentHow Kiro turns application requirements into designs, implementation tasks, and reviewable code, and where this workflow fits alongside model training and application deployment.

Kiro is an AI-assisted development tool. Engineers use it to understand source code, develop features, and prepare changes for review. Its specification-driven development workflow makes the intended behavior explicit before moving into implementation.

AWS Glue: discovering data formats and schemasHow Glue crawlers and the Data Catalog describe structured and unstructured data sources, and why inferred schemas alone do not identify labels or the meaning of timestamps, text, and images.

AWS Glue helps discover and prepare data. For discovery, a crawler inspects a configured data source, such as files in Amazon S3. It uses classifiers, which recognize supported data formats and help infer their schemas.

AWS Lake Formation vs AWS Glue: governance and data preparationHow Lake Formation's data access governance differs from AWS Glue's data discovery, cataloging, and transformation responsibilities.

AWS Glue and AWS Lake Formation solve different problems in a data lake: a central store of datasets, often held in Amazon S3. Glue helps discover, describe, and prepare data. Lake Formation helps govern who can access it. They are complementary services, not replacements for each other.

Amazon Bedrock: Prompt ManagementHow reusable prompt templates, tested variants, and saved versions make application prompts manageable and reproducible.

A prompt tells a foundation model what to do. If an application copies that text into several places, changing the instructions can leave different parts of the application using different wording. Amazon Bedrock Prompt Management lets you create, organize, test, and version reusable prompts instead of managing scattered copies.

Amazon Bedrock: model evaluationHow automated and human evaluations provide evidence about model quality, bias, and trustworthiness without making model internals explainable.

Model evaluation checks a foundation model's responses against defined criteria. Amazon Bedrock supports automated and human evaluations that help compare models and assess their suitability for a task. The results are evidence about observed behavior, not a guarantee about every future response or an explanation of the model's internal workings.

Amazon SageMaker: bias analysis with ClarifyHow Clarify measures bias in datasets and model predictions, explains prediction contributions, and supports monitoring for bias drift.

Amazon SageMaker Clarify, a capability of SageMaker AI, helps teams detect potential bias in data and model predictions. It compares groups using bias metrics: numerical measures of differences in representation, outcomes, or errors. You choose the group-defining attribute, called a facet, such as age group, and the groups to compare.

Amazon QuickSight: business insights from AI/ML resultsHow QuickSight turns datasets and AI/ML outputs into interactive dashboards that help business users explore and interpret results without building or training models.

Amazon QuickSight is a managed business intelligence service. It turns data into charts, tables, and interactive dashboards so people can explore business questions without writing model-training code.

Experiment Tracking: run records and reproducibilityHow persistent run records link model results to data, code, configuration, and artifacts so teams can compare experiments and reproduce selected candidates.

Experiment tracking preserves evidence from repeated attempts to build and evaluate models. An experiment groups related work; a run is one execution, such as a training job or evaluation. Each run needs a unique identifier so parallel attempts stay separate and teammates can refer to the same result.

CI/CD: repeatable ML release pipelinesHow automated build, test, and release stages validate versioned ML code and model artifacts and control their promotion to production, including after retraining.

Continuous integration (CI) means developers frequently merge small code changes into a shared codebase and receive automated build and test feedback. A pipeline runs a defined sequence of stages instead of relying on someone to remember release commands. In machine learning, those stages must check both the software and the model it will use.

AWS CodePipeline: repeatable release workflowsHow stages, actions, artifacts, and approval gates orchestrate repeatable releases of application and ML-related changes while integrated tools perform the underlying work.

AWS CodePipeline is a managed service that coordinates a release workflow. Instead of remembering a list of manual tasks for every change, a team defines a pipeline and runs changes through that workflow. A configured source-change trigger can start a run automatically.

Amazon CloudWatch: metric-based monitoringHow CloudWatch uses AWS service and custom metrics to track production workload health and raise threshold alarms, including latency, errors, and resource utilization for deployed ML models.

Amazon CloudWatch monitors resources and applications. A metric is a series of numerical measurements over time, such as response time or CPU usage. Looking at these measurements helps a team notice when a deployed model becomes slow, unreliable, or short of capacity.

Model Drift: the production monitoring feedback loopHow changes in production inputs and outcomes are monitored against a reference baseline to decide whether a model needs maintenance rather than automatic retraining.

Model drift describes changes in a model’s operating environment that can make its learned patterns less useful. The model itself can stay unchanged while customers, products, or behavior change. Good validation results before deployment do not guarantee good predictions forever.

Classification Metrics: measuring predictive performanceHow confusion-matrix counts yield accuracy, precision, recall, and F1, and why class imbalance and error costs affect their usefulness without guaranteeing business value.

A classification model predicts a category. To measure its performance, compare its predictions with ground-truth labels: the known actual outcomes. For binary classification, first choose the positive class. Positive means the outcome being detected, not necessarily something desirable.

Cost management: cost per AI outcomeHow development and operating costs translate into cost per user or successful interaction, helping compare AI models against business value and quality requirements.

Cost per AI outcome connects spending to useful work, such as a customer served or a task completed successfully. It helps a team judge whether an AI application delivers business value, rather than simply choosing the model with the lowest request price.

AWS Cost Explorer: analyzing project costsHow filtering and grouping historical costs by service and activated cost allocation tags reveals ML project spending that can inform cost per user and ROI calculations.

AWS Cost Explorer helps you explore historical AWS costs and usage. For an ML project, start with a defined date range, such as a completed month. Daily views reveal changes within that period; monthly views help compare spending over time. These are spending records, not measures of model accuracy or business value.

Return on Investment: measuring the financial value of AIHow to calculate and interpret AI investment returns by translating measured business improvements into financial benefits and comparing them with full lifecycle costs against a consistent baseline and evaluation period.

Return on investment (ROI) compares the financial benefits of an AI initiative with its investment costs. It answers a business question: does the value gained exceed what we spent? A higher accuracy score or faster response is useful evidence, but neither is a financial return by itself.

DOMAIN 224% of the exam · 50 lessons
Fundamentals of GenAI
Describe AWS infrastructure and technologies for building GenAI applications11 lessons
Amazon Bedrock: GuardrailsHow configurable checks screen model inputs and outputs for responsible-AI risks, and why those checks cannot guarantee safe or truthful responses.

Amazon Bedrock Guardrails provides configurable checks that help an application reduce responsible-AI risks. These checks examine content against policies you choose, such as restrictions on harmful content or sensitive information. They do not retrain the model or make it inherently trustworthy.

Amazon Q vs Amazon BedrockWhen to choose Amazon Q's ready-made assistants rather than Amazon Bedrock for building a custom generative AI application.

Amazon Q is in maintenance since 2026-06-30. It is closed to new customers; existing customers can keep using it. AWS provides this migration guide: https://docs.aws.amazon.com/amazonq/latest/qbusiness-ug/qbusiness-availability-change.html. This lesson explains Q as AWS offered it, not as a recommendation for new workloads.

Amazon Q Business and Amazon Bedrock: managed assistant or application building blocksWhen a managed enterprise assistant fits a business need versus building a custom generative AI application with Amazon Bedrock.

Amazon Q Business has been in maintenance since June 30, 2026. It is closed to new customers; existing customers can keep using it. This lesson explains the service as AWS offered it, not as a recommendation for new workloads. AWS's migration guide: https://docs.aws.amazon.com/amazonq/latest/qbusiness-ug/qbusiness-availability-change.html.

AWS shared responsibility model: assigning control ownershipHow service and AI deployment choices determine the AWS–customer boundary and which security and governance controls customers must implement and substantiate.

The AWS shared responsibility model divides security work between AWS and its customers. AWS provides security of the cloud: protecting the facilities, physical hardware, underlying networking, and virtualization layer that run AWS services. Customers provide security in the cloud: protecting their data, identities, permissions, applications, and customer-controlled settings.

Security: layered protection for generative AIHow AWS infrastructure and layered controls protect generative AI data and workloads while leaving customers responsible for secure access and configuration.

Generative AI applications handle valuable information: prompts, retrieved documents, model outputs, and sometimes training data. Security protects that information from unauthorized access or changes and helps keep the application available.

AWS IAM: granting permissions through rolesHow assumed roles provide temporary access and IAM policies limit allowed actions on AWS resources, including those used by AI applications.

AWS Identity and Access Management (IAM) controls access to AWS resources. An IAM role is an identity with permissions that an authorized person, application, or AWS service can use. A role does not have its own permanent password or long-term access keys.

Amazon VPC: network isolationHow isolated networks, traffic controls, and private service access reduce the exposure of generative AI workloads without guaranteeing application security or compliance.

Amazon Virtual Private Cloud (Amazon VPC) gives you a logically isolated network in AWS. You choose its IP address ranges and divide it into subnets, which are smaller network sections. For a generative AI application, this lets you separate application servers from resources intended for public access.

AWS KMS: managed encryption keysHow AWS services use KMS keys to protect stored data, including AI application data, while controlling and auditing key use.

AWS Key Management Service (AWS KMS) manages cryptographic keys and controls their use. Integrated AWS services use these keys to protect data at rest: data saved in storage. AI training datasets, saved prompts, and generated outputs need this protection just like other application data.

AWS Compliance versus SecurityHow demonstrating compliance with applicable requirements differs from reducing security risks, and why neither guarantees the other.

Compliance means meeting applicable requirements and demonstrating that the required controls work. Requirements can come from laws, industry standards, contracts, and internal policies. A control is a safeguard or practice, such as restricting access or reviewing permissions.

AWS Compliance: infrastructure assurancesHow AWS compliance reports and program coverage support GenAI workload assessments, and why their service and geographic scope does not establish customer workload compliance.

AWS compliance programs provide assurance about AWS controls against particular standards or requirements. For a generative AI workload, this evidence can support assessment of the infrastructure underneath the application, such as AWS-operated facilities and infrastructure operations. It reduces the need to investigate those controls from scratch.

AWS Artifact: AWS compliance reportsHow to find and download AWS compliance reports as evidence of AWS controls, and why those reports do not establish an AI application's compliance.

AWS Artifact provides access to AWS security and compliance documents, including SOC reports, ISO certificates, and PCI compliance reports. These documents help you assess AWS and supply evidence for reviews or audits. A control is a safeguard or practice, such as restricting physical access to a data center.

Explain the basic concepts of generative AI (GenAI)1 free27 lessons
Tokenization: how models count textFreeHow a model’s tokenizer turns text into token identifiers whose counts determine context usage, token-metered inference costs, and processing work.

A language model does not process text directly as words on a page. A tokenizer divides text into units called tokens and maps them to numerical token identifiers in its vocabulary. The model processes the resulting sequence of identifiers.

Embeddings: meaning in vector spaceHow models represent inputs as learned vectors whose similarity supports semantic search and retrieval.

An embedding is a numerical representation produced by a machine learning model. It commonly takes the form of a fixed-length vector: an ordered list of numbers. Depending on the model, the input can be a passage of text, an image, or another supported type of data.

Embeddings and tokenization: representations versus text unitsHow to distinguish tokenization's division of text into processing units from embeddings' numerical representation of inputs.

A language model needs text in a form it can process. Tokenization divides text into units called tokens and maps those units to numerical token identifiers from a vocabulary. This answers the question: “Which text units will the model process?”

Transformer Architecture: building context with attentionHow attention transforms token embeddings and positional information into context-dependent representations that transformer-based language models use to predict tokens.

A transformer is a neural network architecture that uses attention to build representations of relationships within a sequence. For text, a tokenizer first splits the input into tokens and assigns each one an identifier. A token can be a word, part of a word, or punctuation; it is not necessarily a whole word.

Text Generation: creating written outputsHow generating text supports summarization, translation, code creation, and conversational responses, and what makes each output useful.

Text generation uses generative AI to produce written output from instructions and context. It can create summaries, translations, code, and conversational replies. A prompt supplies the request and any supporting material. Clear instructions about the audience, purpose, and format help define what a useful result should look like.

Multimodal Generation: matching modalities to use casesHow a model’s supported input and output modalities determine which media-generation and media-grounded assistant use cases it can support.

A modality is a form of information, such as text, images, audio, or video. Multimodal generation involves more than one modality. Turning a written description into an image and answering a question about a photograph in text are both multimodal interactions.

Semantic Search: retrieving by meaningHow compatible query and passage embeddings retrieve relevant information despite different wording, helping users and generative AI assistants find source material.

Semantic search finds existing material by its relationship to the meaning of a request. An employee asking “Can I expense a train ticket?” could find a passage titled “Reimbursement for business travel,” even though the wording is different.

Text Generation versus Semantic SearchHow composing a new response differs from retrieving relevant existing material, even when both capabilities appear in one application.

Text generation composes a written response to instructions. Semantic search finds existing material whose meaning is relevant to a query. The main difference is the job being done: composing an output versus selecting material that already exists.

Retrieval-Augmented Generation and Semantic Search: generation versus retrievalWhy semantic search returns relevant information while RAG generates an answer using retrieved information, which need not come from semantic search.

Semantic search finds existing information by meaning. A question such as “Can I expense a train ticket?” can return a passage titled “Reimbursement for business travel,” even though the wording differs. Its output is relevant source material, such as passages or document links—not a newly written answer.

Amazon Q: generative AI assistantsHow Q Developer supports software development and AWS workflows, while Q Business answers employee questions using organizational content.

Amazon Q is in maintenance since June 30, 2026. It is closed to new customers; existing customers can keep using it. AWS provides a migration guide at https://docs.aws.amazon.com/amazonq/latest/qbusiness-ug/qbusiness-availability-change.html. This lesson explains the service as AWS offered it, rather than recommending it for new workloads.

Model Evaluation: defining evidence of successHow representative data, technical and business metrics, and human review establish whether a model meets acceptance criteria throughout its lifecycle.

Model evaluation asks whether a model is suitable for its intended task under real operating conditions. Start with the intended users, supported tasks, and unacceptable failures. Turn these into acceptance criteria: measurable conditions the application must meet. For a support assistant, these could cover correct answers, appropriate escalation, and protection of customer information. Set the criteria before inspecting results, rather than changing them to make a result look successful.

Model Evaluation: comparing candidate modelsHow to compare models under common evaluation conditions and weigh capabilities, quality, safety, latency, cost, compliance constraints, and environmental impact when selecting a suitable model.

Model selection means choosing a model that suits a particular task. Start with its capabilities: the kinds of input and output it supports, the languages it handles, and whether it can accept the required amount of context. A text-only model cannot meet a requirement to interpret images directly.

Usage-Based Pricing: generative AI inference costsHow input and output token charges compare with provisioned-throughput costs as model choice, demand, and performance and availability requirements change.

Inference means using a trained model to produce a result. For text generation, many Amazon Bedrock models offer token-based, on-demand pricing. A token is a unit of text processed by a model, often a word or part of a word. Tokens are not the same as words, and token counts can differ between models.

Inference Latency: cost–responsiveness tradeoffsHow token workload, model choice, and serving capacity affect time to first token, total response time, and the cost of meeting responsiveness targets.

Inference latency is the time needed to obtain an output from a deployed model. Always define what you measure: model execution covers processing within the serving system, while application end-to-end latency can also include network travel, authentication, retrieval, queueing, and response delivery.

Context Windows: assembling context within token limitsHow context engineering selects and organizes instructions, history, retrieved evidence, and tool results within a model's token budget while reserving response capacity and balancing relevance, cost, and latency.

A context window is the finite capacity a model can use during a request. Language models measure this capacity in tokens: words, word fragments, punctuation, or other encoded content. Limits and counting rules depend on the selected model. Images and audio also consume capacity under model-specific rules.

Conversation Memory: carrying context between requestsHow applications retain and select relevant interaction history to preserve continuity within and across sessions without resending everything.

Conversation memory is interaction history that an application retains and reuses. If a user says, “Compare that with the cheaper option,” earlier messages supply the missing meaning. In a typical invocation, the model uses the context it receives; the application or a managed conversation capability maintains continuity between invocations.

Context Windows vs. Conversation MemoryWhy retaining conversation information does not make it available to the model unless it is selected for the current context, and why persistent memory does not enlarge the context window.

Conversation memory is information an application or managed service retains from interactions. It can include earlier messages, saved preferences, or a conversation summary. Persistent memory keeps information beyond the request that created it, potentially across sessions.

Retrieval-Augmented Generation: external knowledge in model contextHow RAG selects external evidence and supplies it at request time to ground business answers without changing model weights.

A model's training does not give it reliable access to every company's current policies, manuals, or product details. Retrieval-Augmented Generation (RAG) helps by finding external information relevant to a question and supplying it to the model when that question is answered.

Tool Calling: from model request to external executionHow tool definitions enable a model to request an operation, while application code validates, authorizes, executes, and returns its result.

Tool calling lets a language model request an operation outside its text-generation process, such as checking an order's current delivery status. The application supplies a tool definition: a name, a description of its purpose, and an input schema describing the accepted arguments.

Model Context Protocol: connecting agents to external capabilitiesHow hosts, clients, and servers let an AI application discover and access external capabilities through a common interface without providing reasoning, orchestration, or authorization enforcement.

Model Context Protocol (MCP) standardizes communication between AI applications and external capabilities. A capability might look up an order, read a document, or provide a reusable prompt template. A common interface reduces the need to invent a different connection method for every application and external system. It does not remove the work of connecting those capabilities to their underlying systems.

Model Context Protocol and Tool Calling: integration versus invocationHow MCP's standardized capability interface differs from tool calling, in which a model proposes an operation for the application to execute.

Model Context Protocol (MCP) standardizes communication between AI applications and external capabilities. It answers an integration question: how can an application discover and access capabilities through a common interface rather than a separate custom interface for each system?

Multi-Agent Orchestration: coordinating specialist agentsHow coordination patterns guide specialist agents through a shared task using explicit handoffs, scoped memory, and controlled tool execution.

Multi-agent orchestration organizes specialist AI agents around a shared task. Each agent has a role, instructions, and permitted capabilities. Specialists can use the same underlying model; different models are not required.

Multi-Agent Orchestration vs. Model Context ProtocolWhy MCP connectivity to external capabilities does not replace agent coordination or workflow control.

Multi-agent orchestration organizes several AI agents around a shared task. It assigns work, manages handoffs, and decides when the application can move forward. Specialist agents might research options, check rules, or prepare a response.

AWS Step Functions: workflow orchestration with state machinesHow state machines coordinate agent and tool tasks through ordered, conditional, and parallel steps while tracking workflow data and handling task failures.

AWS Step Functions coordinates work through a state machine: a definition of steps and the rules for moving between them. Each step is a state. A Task state calls a service to perform work, such as running an agent or invoking a tool. Other states control which work happens next.

AWS Step Functions vs AWS Lambda: orchestration and code executionWhen coordinating an agent workflow calls for Step Functions rather than a Lambda function that executes an individual task.

An agent workflow is the wider process around an AI agent's work, including calls to tools and processing their results. Within that process, code execution does a task; orchestration coordinates how tasks fit together.

Amazon Bedrock AgentCore: managed infrastructure for AI agentsHow AgentCore's managed components support production agents while application code defines their reasoning and actions.

An AI agent uses a model and tools to work toward a goal. Moving it into production requires more than a working prompt: it needs somewhere to run, controlled access to tools, continuity between interactions, and a way to investigate failures. Amazon Bedrock AgentCore provides managed components for these needs.

Amazon Bedrock and Amazon Bedrock AgentCore: choosing the right serviceWhen to use Amazon Bedrock for model access and managed agent orchestration versus AgentCore for framework-independent agent infrastructure.

Amazon Bedrock provides managed access to foundation models. An application sends input to a selected model and receives output, such as a summary or generated text. AWS operates the underlying inference infrastructure. A model request does not, by itself, require an agent.

Understand the capabilities and limitations of GenAI for solving business problems12 lessons
Conversational AI: flexible dialogue with generative AIHow generative models support interactive dialogue by adapting replies to context, handling follow-up questions, and generating tailored content rather than only predefined responses.

Conversational AI lets people interact with a system through natural-language dialogue. A person can ask a question, add details, and ask again. The system responds through text or speech.

In-Context Learning: adapting responses through contextHow instructions, examples, and conversation history adapt responses without updating model weights, using zero-, one-, and few-shot prompts while balancing context quality, token cost, and latency.

In-context learning adapts a model’s response by changing the input it receives, not its weights, the learned parameters inside the model. Context can include instructions, examples, reference information, and conversation history. The same model can write a friendly reply or a formal summary when given different guidance.

Amazon Q Business: conversations with enterprise informationHow retrieval-grounded conversations help users find, summarize, and generate content from enterprise information, and why answers still need validation.

Amazon Q Business has been in maintenance since June 30, 2026. It is closed to new customers; existing customers can continue using it. This lesson describes the service as AWS offered it, not as a recommendation for new workloads. AWS's migration guide: https://docs.aws.amazon.com/amazonq/latest/qbusiness-ug/qbusiness-availability-change.html.

Hallucinations: factual reliability of generated answersWhy convincing generated claims can be unsupported, how they cause business and legal harm, and how grounding, output validation, and carefully interpreted confidence scores reduce—but do not eliminate—the risk.

A hallucination is generated information that is false, fabricated, or unsupported by the evidence the system should use. Examples include an invented API parameter, a nonexistent legal case, or a summary that attributes a statement to a document that never contained it. An unsupported claim is not necessarily false, but it has not been established by the available evidence.

Nondeterminism: limits of repeatable generationHow sampling and execution variability affect repeatability, and why controls can reduce variation without guaranteeing identical answers.

Nondeterminism means that repeating an operation with the same apparent inputs does not necessarily produce the same result. A generative model might use different wording, organize an answer differently, or make a different recommendation. This variation can help with brainstorming, but makes stable application behavior harder to achieve.

Hallucinations and nondeterminism: correctness versus variabilityHow unsupported claims differ from variation between repeated outputs, and why making answers more repeatable does not make them factually correct.

Hallucination concerns what an answer claims. A generative AI system hallucinates when it presents false, fabricated, or unsupported information. For example, a summary might attribute a statement to a document that never contained it. Fluent language does not establish that a claim is supported.

Model Interpretability: understanding model behaviorHow inspectable decision rules and feature-based explanations help people understand predictions, and why generated rationales may not faithfully explain generative model behavior.

Model interpretability helps people understand how inputs relate to a model's predictions. Knowing that a model predicts “high risk” is different from understanding which inputs led to that result. This understanding helps people investigate errors and spot reliance on unexpected information.

Model Interpretability vs Model TransparencyWhy making a model's internal decision process inspectable differs from making its behavior understandable to a particular audience.

Model transparency concerns visibility into a model and how it is developed and used. Here, we focus on transparency of its decision process: can someone inspect the rules or calculations that produce an output? A small decision tree can expose that process directly. Access to a complex model’s code or weights permits inspection, but does not necessarily make its decisions easy to follow.

Model Interpretability vs Post-Hoc ExplainabilityWhy producing an explanation after training is a technique for supporting understanding rather than a guarantee that the model's behavior is faithfully understood.

Model interpretability concerns whether people can understand how a model's inputs relate to its behavior and outputs. Post-hoc explainability is a way to support that understanding: an explanation method is applied to a model after training.

Data Residency: geographic boundaries across the data lifecycleHow residency requirements determine approved locations for data storage, processing, copies, and logs, and how to prevent unintended transfers outside those boundaries.

Data residency describes where data resides geographically. Requirements can come from laws, contracts, industry rules, or internal policy. Turn each requirement into an explicit rule: which data classes it covers and which locations are approved. Customer records might have a strict boundary while public website images do not.

Total Cost of Ownership: comparing GenAI model costsHow to compare the full costs of GenAI model options over the same period and workload, including usage charges, customization, infrastructure, and ongoing operational effort.

Total Cost of Ownership (TCO) is the complete cost of using a solution over an agreed period. For generative AI, it includes more than the price of a model call: it also includes preparing the model for the task, supporting infrastructure, and the people who keep it working.

Key Performance Indicators: measuring generative AI business valueHow to translate business objectives into measurable outcome KPIs and assess value against baselines, targets, review periods, and quality guardrails.

A key performance indicator (KPI) is a selected measurement that shows progress toward an important objective. Start with the change the business wants, then choose a measure of that change. For a generative AI support assistant, “help customers resolve problems” points to successful resolution, not the number of answers generated.

DOMAIN 328% of the exam · 60 lessons
Applications of Foundation Models
Describe design considerations for applications that use foundation models (FMs)1 free22 lessons
Multimodality: matching model inputs and outputsFreeHow to match an application's required modalities to a foundation model's input and output capabilities, accounting for media limits, task quality, cost, and latency.

A modality is a type of information, such as text, images, audio, or video. Multimodality means working with multiple information types. Start model selection by writing down what the application must receive and what it must produce—not by looking for a model labeled “multimodal.”

Context Window: input and output token budgetsHow request content and generated answers fit model-specific token limits, and how those limits affect model choice and response-length settings.

A model's context window is the information it can work with during one inference request. For text models, this capacity is measured in tokens: words, parts of words, punctuation, or other text fragments. Tokens are not the same as words, and the same text can have different token counts with different models.

Caching: prompt reuse in foundation modelsHow prompt caching reuses processing of repeated input prefixes, and how model support, cache hits, and expiration affect latency and cost savings.

A foundation model processes the input in a prompt before generating an answer. Prompt caching saves reusable processing of an input prefix: the beginning of the prompt. A later request with a matching prefix can reuse that work instead of processing the whole beginning again.

Temperature Sampling: controlling response variationHow temperature reshapes next-token probabilities to balance consistency and variety without guaranteeing accuracy, output format, or response length.

A language model generates a response one token at a time. A token can be a word, part of a word, or punctuation. For each position, the model scores possible next tokens using the prompt and the text generated so far.

Nucleus Sampling: controlling choices with top-pHow top-p selects an adaptive next-token candidate set and changes response variability without guaranteeing accuracy or controlling response length.

A language model generates text one token at a time. A token can be a word, part of a word, or punctuation. At each step, the model assigns probabilities to possible next tokens. Nucleus sampling, also called top-p sampling, uses those probabilities to build a shortlist of eligible choices.

Nucleus Sampling vs. Temperature SamplingHow truncating candidates by cumulative probability differs from reshaping their probabilities, and why the two controls can interact.

A language model generates text by repeatedly choosing a next token, which can be a word, part of a word, or punctuation. Temperature and nucleus sampling affect that choice in different ways: temperature reshapes the probabilities, while nucleus sampling cuts down the candidate set.

Text Embeddings: retrieving passages by meaningHow similarity between question and passage vectors retrieves relevant text for RAG applications, where the retrieved text—not the vectors—provides evidence for generated answers.

A text embedding is an array of numbers, called a vector, produced by a machine learning model. It represents learned patterns in the meaning of a word, sentence, or passage.

Amazon OpenSearch Service: vector storage and similarity searchHow OpenSearch stores embeddings in vector indexes and retrieves similar content for AI applications.

Amazon OpenSearch Service is a managed search and analytics service that can also store and search embeddings. An embedding is a list of numbers, called a vector, produced by a machine learning model to represent content such as text.

Vector Search: storing embeddings in AWS databasesHow embeddings and metadata support similarity retrieval in Amazon OpenSearch Service, Aurora PostgreSQL and RDS for PostgreSQL with pgvector, and Neptune Analytics.

An embedding is a list of numbers produced by a model to represent features of content, such as the meaning of a paragraph. Vector search compares a query embedding with stored embeddings to find nearby representations. A search for “recover account access” can therefore find “Reset your password” without matching the exact words.

Amazon RDS: storing embeddings with PostgreSQLHow RDS for PostgreSQL uses the pgvector extension to store application-generated embeddings alongside relational data and retrieve similar vectors.

An embedding is a list of numbers produced by a machine learning model to represent content, such as a document. Embeddings let an application compare content by meaning rather than only matching exact words.

Amazon Aurora: storing and searching embeddingsHow Aurora PostgreSQL uses the pgvector extension to store embeddings alongside relational data and retrieve similar vectors for foundation model applications.

An embedding is a list of numbers that represents features of content, such as the meaning of a text passage. An embedding model creates this list. Comparing embeddings can help an application find passages with similar meanings, even when they use different words.

Amazon Aurora and Amazon RDS: PostgreSQL vector database optionsHow Aurora PostgreSQL differs from RDS for PostgreSQL as a managed database option, while both support vector storage through pgvector.

Amazon RDS is a managed relational database service that offers several database engines. Amazon RDS for PostgreSQL runs the PostgreSQL engine. Amazon Aurora PostgreSQL-Compatible Edition runs an AWS-built engine designed to work with PostgreSQL applications and tools.

Amazon Neptune: vector search with Neptune AnalyticsHow Neptune Analytics stores embeddings on graph vertices and combines vector similarity search with graph relationships to retrieve context for foundation models.

Neptune Analytics is an analytics database engine in Amazon Neptune that supports graph analysis and vector search. It is separate from Neptune Database. A graph represents things as vertices and their relationships as edges: for example, an article vertex can connect to a product vertex through an ABOUT relationship.

Fine-Tuning vs. In-Context LearningHow persistent learned adaptations differ from examples supplied in each request, including where their training and inference costs arise.

Fine-tuning and in-context learning can both help a pretrained model follow a task’s conventions. The key difference is where the adaptation lives: in learned model parameters, or in the context supplied for a request.

Model Distillation: upfront costs and serving savingsHow training a student on teacher-generated examples trades upfront investment for potential serving savings, depending on request volume and acceptable task quality.

Model distillation transfers behavior from a teacher model to a student model. In the approach covered here, a larger teacher generates answers to representative application inputs. Those examples are reviewed and used to train a typically smaller student; the teacher's parameters are not simply copied.

Knowledge Distillation vs. Fine-TuningWhy distillation identifies learning from a teacher while fine-tuning identifies adapting an existing model, allowing the same training process to be both.

Knowledge distillation identifies learning from a teacher model. A student model is trained to approximate useful behavior from that teacher, using signals such as teacher-generated answers or output probabilities. The defining question is: What provides the guidance for learning? Distillation transfers behavior through training; it does not mean directly copying the teacher’s parameters.

Model Distillation and Fine-Tuning: learning signal versus training procedureWhy distillation identifies learning from a teacher while fine-tuning identifies a training procedure, and how training a student can be both.

Model distillation describes a student model learning selected behavior from a teacher model. The teacher provides the learning signal: information that guides what the student learns. For example, the teacher can generate answers that become training targets for the student.

Fine-Tuning: customization costs and tradeoffsWhen fine-tuning's data preparation, training, evaluation, and serving costs are justified compared with prompting, retrieval, pretraining, or distillation.

Fine-tuning adapts a pretrained model using targeted examples. Its value might be more consistent classifications, a required response format, or better task performance. The training bill alone does not tell you whether that improvement is worth buying.

AI Agents: goal-directed actionHow agents complete business tasks through a controlled cycle of choosing actions, using tools, and observing results, and when that flexibility is useful.

An AI agent is a software system that selects actions to achieve a goal, observes their results, and decides what to do next. The goal describes the desired outcome, such as resolving a delayed order. It does not have to specify every step in advance.

Tool Use: the tool-call cycleHow a model requests a defined tool, the application validates and executes the request, and returned results enable agents to use business data and perform actions.

Tool use lets a model request capabilities outside its learned knowledge. A tool can retrieve current inventory, calculate a total, or reserve an item. This gives business applications access to live information and actions without retraining the model.

Tool Use and AI Agents: capability versus orchestrationWhy calling a tool is not by itself agent behavior, and how coordinating actions toward a goal distinguishes an agent from a simple tool-enabled interaction.

Tool use lets a model-based application use capabilities outside the model’s learned knowledge. A tool can retrieve an order’s current status, calculate a total, or create a support ticket. The model requests an operation; application code executes it and returns the result.

Task Planning: from business goals to action plansHow an agent breaks a business goal into dependent actions and adjusts its plan using verified results, available tools, and approval boundaries.

Task planning connects a business goal to the actions needed to achieve it. An AI agent can interpret the request, identify missing information, and break the work into smaller tasks. Planning determines what work is needed and how actions depend on one another; scheduling determines when work runs.

Choose effective prompt engineering techniques14 lessons
Instruction Hierarchy: instruction priorityHow model- and API-supported roles determine instruction authority, rather than wording, prompt position, or commands embedded in reference context.

Instruction hierarchy determines which instruction takes precedence when directions conflict. Authority comes from the instruction roles or fields supported by the model and API, not from how important the text sounds. A role is part of the structured input supplied to the model; typing SYSTEM: inside an ordinary user message does not create a system message.

Prompt Context: supplying task-specific informationHow to supply relevant, trustworthy background within a model's context limits while keeping supporting information distinct from instructions and negative prompts.

Prompt context is background information a model receives for a particular task. It can include reference documents, purchase details, earlier conversation turns, or tool results. A support assistant answering a refund question needs the applicable policy and relevant purchase facts, not just the question. This helps it answer for the situation rather than rely only on patterns learned during training.

Negative Prompting: defining output exclusionsHow to express specific output exclusions alongside positive instructions, and why their effectiveness depends on the model rather than providing enforced safeguards.

Negative prompting tells a generative AI model what its output should avoid. The exclusion might concern content, such as customer names; behavior, such as guessing an unknown cause; or presentation, such as Markdown code fences. Here, negative means excluding something, not using an unfriendly tone.

Chain-of-Thought Prompting: structuring multistep reasoningHow instructions or worked examples guide intermediate reasoning, when this helps, and why generated explanations do not guarantee correct answers.

Chain-of-thought prompting guides a foundation model toward intermediate reasoning before a final answer. An intermediate step produces something needed later, such as a subtotal used in a comparison. The goal is a more structured approach to a problem, not simply a longer response.

Chain-of-Thought Prompting vs. In-Context LearningWhy prompting for intermediate reasoning differs from learning through supplied examples, so example count alone does not determine whether a prompt uses chain-of-thought.

In-context learning means adapting a model’s response through the context supplied with a request, without updating its model parameters. Supplied examples can demonstrate a task’s labels, answer format, or input-to-output pattern. For example, labeled support tickets can show how to classify a new ticket.

Prompt Templates: reusable model requestsHow fixed instructions and runtime placeholders form reusable model requests that can incorporate examples or other prompting techniques without retraining the model.

A prompt template is a reusable specification for constructing a model request. It combines fixed instructions with runtime placeholders: named spaces that an application fills with values for the current request. This lets an application repeat a task without rewriting its instructions each time.

Iterative Prompt Refinement: improving prompts through feedbackHow repeated testing and targeted, clear prompt revisions improve response quality without replacing guardrails or changing model weights.

Iterative prompt refinement means testing a prompt, examining the response, and revising the instructions to address a specific weakness. The aim is better results, not merely wording that sounds more polished.

AI Guardrails: enforcing response boundariesHow explicit prompt constraints guide acceptable responses and why independent application checks are needed to enforce those boundaries.

AI guardrails define the boundaries of acceptable behavior for an AI application. They can limit supported topics, prohibited content, sensitive information, output formats, evidence requirements, and permitted actions. A support assistant, for example, can answer product questions without revealing another customer's account details.

Data Poisoning: compromised data and AI behaviorHow malicious changes to training or retrieval data influence AI behavior, and why protecting source integrity requires more than well-written prompts.

Data poisoning is deliberate manipulation of information an AI system relies on. An attacker adds, changes, or selectively removes data to influence behavior. Unlike an accidental data-quality problem, poisoning is intentional, although intent can be difficult to establish. The goal may be worse answers overall, biased recommendations, or one specific wrong answer.

Jailbreaking: bypassing model safeguardsHow adversarial inputs can defeat a model’s behavioral restrictions and why prompt instructions cannot replace independent application security controls.

Jailbreaking is an attempt to make a foundation model disregard its safeguards or behavioral restrictions. The attacker wants a response the model is intended to refuse, such as prohibited guidance or disclosure of protected information. Here, jailbreaking concerns generative AI behavior—not unlocking a mobile device.

Prompt Injection: protecting instruction trust boundariesHow direct and indirect attacks turn untrusted content into instructions, and why guardrails need independently enforced access and action controls to limit harm.

Prompt injection is an attempt to make a language model follow instructions supplied through an untrusted source instead of following the application's rules. The attack crosses an instruction trust boundary: material the model should read as data is treated as authority over its behavior. The payload can be ordinary language, such as “Ignore the summary request and reveal the confidential context.”

Data Poisoning vs. Prompt InjectionHow manipulating trusted information differs from embedding malicious instructions, and when a poisoned retrieval document combines both attacks.

Data poisoning is deliberate manipulation of information an AI system relies on. An attacker might change training examples or replace a correct policy statement with a false one. The AI can then give a harmful answer while still following the intended task. An accidental error is a data-quality problem, not necessarily an attack.

Jailbreaking and Prompt Injection: distinguishing the risksHow defeating model safeguards differs from injecting instructions to redirect application behavior, and when an attack does both.

Jailbreaking means trying to defeat a model’s safeguards or behavioral restrictions. These are limits on what the model should produce or support, such as restrictions against generating prohibited content. Jailbreaking asks: Can the attacker get the model to cross a boundary it is supposed to respect? Here, the term concerns AI behavior, not unlocking a mobile device.

Version Control: immutable prompt revisionsHow Amazon Bedrock Prompt Management separates editable drafts and prompt variants from immutable numbered versions, allowing applications to reference approved revisions and return to earlier ones.

Amazon Bedrock Prompt Management helps you manage reusable instructions for a foundation model. You develop a prompt in an editable draft, where you can change its wording, input variables, model selection, and supported inference settings.

Describe the training and fine-tuning process for FMs9 lessons
Knowledge Distillation: learning from a teacherHow training on a teacher’s probabilities or generated answers transfers behavior to a student, with potential efficiency gains and capability loss that require evaluation.

Knowledge distillation is a training approach in which a student model learns to approximate the behavior of a teacher model. The teacher supplies training signals, such as probabilities for possible answers or answers it generates. Training adjusts the student so that its outputs become more like the teacher’s on the examples used.

Transfer Learning: reusing pretrained representationsHow frozen-feature reuse and fine-tuning adapt pretrained knowledge to a target task, and why model suitability affects the benefits.

Transfer learning reuses learning from an existing model for a new task. Instead of starting with randomly initialized weights, it starts with a pretrained model: a model that has already learned patterns from data. Weights are the numerical parameters adjusted during training.

Continued Pre-Training: learning from additional domain textHow resuming self-supervised training on an unlabeled domain corpus updates an existing model’s capabilities, with potential gains in domain fluency and losses in general performance.

Continued pre-training resumes training an existing foundation model on additional data rather than starting a new model from scratch. For a language model, that data can be an unlabeled domain corpus: a collection of documents about a field, such as equipment manuals and maintenance reports. Unlabeled means people do not need to supply a correct answer for each document.

Instruction Tuning: learning from task demonstrationsHow supervised instruction-response examples update a pretrained model’s parameters or adapters to improve task behavior rather than train capabilities from scratch.

Instruction tuning is supervised training that teaches a pretrained model how to respond to requests. Each demonstration contains an instruction, optional input or context, and a target response: the answer the model should learn to produce. It is supervised because the examples supply the desired answers rather than asking the model to discover what counts as a good response.

Continued Pre-Training vs. Instruction TuningHow the two methods differ in training examples, learning objectives, and intended outcomes, and why domain familiarity is different from instruction-following behavior.

Continued pre-training and instruction tuning both adapt an existing foundation model through further training. Their central difference is what the training examples demonstrate: domain content, or how to respond to a request.

Data Curation: fit-for-purpose fine-tuning datasetsHow to turn source material into a governed fine-tuning dataset with reliable labels, representative examples, and an appropriate size for the intended model behavior.

Data curation means making data fit for a specific purpose. For foundation model fine-tuning, first describe the behavior you want: classifying support requests, following a response format, or asking for clarification when information is missing. Choose examples that demonstrate that behavior. A large document collection is not automatically a useful training dataset.

AWS Lake Formation: governing access to data lake datasetsHow catalog-based permissions and fine-grained data filters control access to approved datasets for model training and fine-tuning.

AWS Lake Formation helps control who can read data lake datasets. For model training and fine-tuning, this lets a data owner make approved data available without giving every training workflow unrestricted access to the lake.

Amazon SageMaker Ground Truth: human labeling workflowsHow labeling workflows use workforce choices, clear instructions, and quality checks to produce annotations and human preference feedback for foundation model fine-tuning.

Amazon SageMaker Ground Truth helps organize data labeling: people receive tasks, follow instructions, and submit judgments that become training data. It supports human labeling workflows, including custom tasks. Creating this data is separate from training or fine-tuning a model.

Reinforcement Learning from Human Feedback: preparing preference dataHow to prepare representative, consistently labeled, and governed preference comparisons for RLHF reward training, prioritizing feedback quality over raw dataset size.

Reinforcement Learning from Human Feedback (RLHF) uses human judgments to help shape a model's behavior. In a common workflow, reviewers compare candidate answers, a reward model learns to predict those preferences, and reinforcement learning updates the language model using that reward. This lesson focuses on preparing the comparisons, before reward training begins.

Describe methods to evaluate FM performance15 lessons
Benchmark Datasets: repeatable model comparisonsHow representative, held-out test cases evaluated under consistent conditions support meaningful foundation-model comparisons—and where benchmark scores can mislead.

A benchmark dataset supplies a common set of test cases for comparing models. A case might contain a question and correct answer, a document and reference summary, or a prompt and scoring guide called a rubric. Shared cases make evaluation more repeatable than trying a few handpicked prompts.

Human-in-the-Loop Evaluation: structured human reviewWhen human judgment is needed to evaluate foundation model outputs and how representative test cases, clear rubrics, and reviewer calibration make that judgment reliable.

Human-in-the-loop evaluation means people assess model outputs using explicit criteria. It is especially useful for open-ended tasks: several answers may be acceptable, yet differ in usefulness, completeness, tone, or suitability for an audience. Matching one reference answer word for word would miss those differences.

N-Gram Overlap: measuring lexical agreementHow matching token sequences support BLEU and ROUGE-N scores, and why high overlap does not establish meaning or factual correctness.

An n-gram is a contiguous sequence of n tokens. Tokens are pieces of text; here, treat each space-separated word as one token. A unigram has one token, a bigram has two, and a trigram has three. Overlap compares these sequences between a model's candidate output and a reference answer.

BLEU: scoring generated text against referencesHow clipped n-gram precision and a brevity penalty produce a lexical-overlap score, and why that score suits translation better than open-ended answers or correctness checks.

BLEU, short for Bilingual Evaluation Understudy, compares generated text, called the candidate, with one or more approved references. It measures lexical overlap: matching tokens and sequences of tokens, not whether the text means the same thing. A token is a unit produced by splitting text; for these examples, treat each word as one token.

ROUGE: scoring reference overlapHow ROUGE-1, ROUGE-2, and ROUGE-L measure overlap with reference text, how to interpret their precision, recall, and F1 scores, and why overlap alone cannot establish answer quality.

ROUGE is a family of automatic text-evaluation metrics originally developed for summarization. It compares generated text, called the candidate, with a reference: an example of an acceptable output. It is useful for repeatable comparisons when a task has meaningful reference answers.

BLEU and ROUGE: precision versus recallHow BLEU's precision-oriented overlap differs from commonly used recall-oriented ROUGE measures when choosing metrics for translation and summarization.

Both BLEU and ROUGE compare generated text, called the candidate, with acceptable text, called the reference. They measure overlap in wording, rather than directly judging whether the candidate means the right thing.

BERTScore: reference-based semantic similarityHow contextual token matching produces precision, recall, and F1 scores that reward similar meaning without proving factual correctness or overall response quality.

BERTScore compares a generated response, called the candidate, with a reference answer. The reference is a human-written or otherwise trusted example of the desired answer. BERTScore measures semantic similarity to that reference, not quality in isolation.

BERTScore and N-Gram Overlap: meaning versus wordingWhy paraphrases and meaning-changing edits can receive different credit from BERTScore than from lexical-overlap metrics such as BLEU and ROUGE.

When evaluating generated text, a candidate is the model’s answer and a reference is an answer used for comparison. Two candidates can express the same idea using different words. Evaluation metrics differ in how much credit they give that change in wording.

LLM-as-a-Judge: rubric-based evaluationHow an evaluator LLM uses task context, evidence, and a rubric to judge response quality, and why those judgments need calibration and human oversight.

LLM-as-a-Judge means using a large language model to assess a response produced by a model, including another output from the same model. Instead of answering the original question, the evaluator judges how well the candidate answer meets explicit criteria. Its result might be a score, a pass/fail decision, or a preference between two answers.

End-to-End Testing: evaluating complete FM application journeysHow to test realistic user journeys through RAG, agents, and workflows against outcome-based acceptance criteria, accounting for variable model responses rather than relying on model benchmarks alone.

End-to-end testing checks an application as one connected system. A test begins where a user would begin, such as a chat interface or API, and follows the journey through to its result. It checks the response, any changed records, and any external actions—not just whether the model returned text.

Retrieval-Augmented Generation: evaluating evidence-grounded answersHow to evaluate retrieval and generated answers separately to identify missing evidence, unsupported claims, and misleading citations.

A fluent answer can still be wrong. In Retrieval-Augmented Generation (RAG), evaluate two things separately: whether retrieval supplied useful evidence, and whether generation used that evidence correctly. This separation helps explain what failed instead of merely marking an answer as bad.

Trajectory Evaluation: checking execution pathsHow to evaluate observable execution traces against required actions, dependencies, and behavioral rubrics to detect failures that final answers can conceal.

Trajectory evaluation checks the observable steps an application takes to reach an outcome. A trace can record an agent's tool calls, arguments, returned results, retries, and stopping decisions. For retrieval-augmented generation, or RAG, it can record retrieval and use of evidence; for workflows, it can record routing and stage transitions.

Task Completion Rate: measuring successful outcomesHow predefined success criteria, verified task outcomes, and a consistent denominator make task completion rate a measure of business goal achievement.

Task completion rate measures how often an application achieves a defined user goal. Calculate it as successfully completed tasks ÷ total eligible attempted tasks × 100. If an assistant completes 850 of 1,000 eligible tasks, its rate is 85%.

User Satisfaction: measuring the AI user experienceHow to measure users’ perceived experience with explicit feedback and CSAT, interpret sampling limitations, and assess business alignment alongside correctness, safety, and cost.

User satisfaction measures how users perceive an application, not just the quality of its underlying AI model. Clear language alone does not guarantee a good experience: slow responses, missing context, or confusing instructions can frustrate users. Begin by naming the intended users, their tasks, and the business objective, such as providing accurate support with less customer effort.

Task Completion Rate vs. User SatisfactionWhy completing a user's task and delivering a satisfying experience are distinct outcomes that can change independently.

Task completion rate asks whether users achieved their intended goals. It measures the share of eligible attempted tasks that were successfully completed. For an appointment assistant, this means the correct booking was actually made—not just that the assistant said it was.

DOMAIN 414% of the exam · 34 lessons
Guidelines for Responsible AI
Explain the development of AI systems that are responsible1 free24 lessons
Algorithmic Fairness: equitable treatment across groupsFreeHow context-appropriate fairness criteria expose harmful group disparities caused by bias that overall model accuracy can hide.

Algorithmic fairness concerns whether an automated system treats people and groups equitably in its intended context. It requires asking who benefits, who is harmed, and whether those differences are acceptable—not just whether predictions are usually correct.

Model Robustness: reliable behavior under input variationHow models maintain acceptable behavior under noisy, unfamiliar, or deliberately manipulated inputs, and why clean-test accuracy alone does not establish robustness.

Model robustness is the ability to maintain acceptable behavior when inputs or operating conditions differ from those used during development. Real inputs contain spelling mistakes, missing values, background noise, and unfamiliar patterns. Some inputs are deliberately manipulated to cause a failure.

Amazon SageMaker: Model CardsHow Model Cards document a model's purpose, training data origins, evaluation results, and limitations to support transparency and governance.

Amazon SageMaker Model Cards, a capability of SageMaker AI, provide a structured record of important information about a machine learning model. They bring together its purpose, training details, evaluation results, and limitations so people can understand the model without having to inspect its code.

Output Validation: enforcing downstream acceptance rulesHow application-side checks reject malformed responses, unsafe content, sensitive data, and unauthorized action proposals before downstream use.

Output validation checks a model response before another component relies on it. Fluent text and valid-looking JSON are still untrusted input: they can contain malformed data, prohibited content, private information, or an action the user cannot perform.

Output Validation: checking claims against evidenceHow checking generated claims and citations against reference material reveals unsupported answers, and why passing these checks does not guarantee factual accuracy.

Output validation checks a generated response before someone relies on it. For evidence checking, the important unit is a claim: a statement that can be compared with reference material. A fluent answer can contain both supported claims and invented details.

Environmental Sustainability: responsible model selectionHow to compare lifecycle energy, emissions, water, and hardware impacts when selecting models that meet application requirements, using consistent measures and documented trade-offs.

Responsible model selection means choosing an approach that meets the application's needs with justified environmental impact. First define acceptable quality, safety, fairness, privacy, and response time. Compare options that meet those requirements; lower resource use does not excuse failing them. Some tasks need only rules or a conventional statistical method, not a generative model.

Model Compression: resource and quality trade-offsHow quantization, pruning, and distillation reduce model resource needs, and when their deployment savings justify compression costs without compromising required quality, safety, or fairness.

Model compression reduces the resources a model needs while preserving enough quality for its intended use. Targets include smaller model files, less runtime memory, or fewer computations. These are different outcomes: a smaller file does not guarantee faster predictions. Inference means using a trained model to produce predictions or responses.

Intellectual Property Rights: permitted use in generative AIHow ownership, licenses, and provider terms affect permitted use of AI inputs and outputs, and why provenance and review matter for avoiding infringement.

Intellectual property rights protect certain creations and information. Copyright protects original expression, such as writing, images, and code; trademarks protect source-identifying names and symbols. Patents protect qualifying inventions, and trade secret law protects qualifying confidential information. For generative AI, an early question is whether you have permission to use the material you supply as input.

Algorithmic Bias: unfair outputs and their consequencesHow systematic unfairness in generative AI outputs can harm users, undermine trust, and create discrimination-related legal risks.

Algorithmic bias is systematic unfairness in an automated system’s behavior or outcomes. In generative AI, it can appear as repeated stereotyping, less useful answers for a language community, or different descriptions of equally qualified people based on their gender. The issue is an unfair pattern, not simply an answer being wrong.

Data Provenance: dataset provenance recordsHow records of a dataset's sources, collection context, permissions, and processing history support responsible dataset selection and transparent model documentation.

Data provenance is the evidence trail behind a dataset. It records where data came from, the circumstances of collection, the permissions that apply, and what changed before the data reached a model. Keep the record tied to an identifiable dataset version, so it describes particular data rather than a name whose contents can change.

Data Cataloging vs. Data ProvenanceHow organizing dataset descriptions for discovery differs from tracing evidence of where data originated and how it changed.

Data cataloging organizes descriptions of datasets so people can find and understand them. These descriptions are called metadata: information about the data, such as its name, location, columns, business meaning, and owner. A catalog helps answer, “Which dataset contains the information I need?” The data itself can stay in its original storage system.

Data Provenance vs. Source CitationHow attributing information to a source differs from documenting the evidence trail behind a dataset.

Source citation attributes information to a source. It answers, “Which source is being referenced?” A citation might identify a report, its publisher, and the release or section used, so a reader can locate and inspect the material.

Data Representativeness: coverage of the intended populationHow to assess whether a dataset’s sources and diversity adequately cover the people, subgroups, and operating conditions relevant to its intended use.

Data representativeness describes how well a dataset reflects the people and operating conditions relevant to its intended use. It is not a property a dataset has for every purpose. Records from urban stores may represent urban demand well but leave rural demand poorly covered, even when the dataset is large.

Class Imbalance: assessing label distributionsHow class frequencies reveal underrepresented outcomes and why unequal frequencies are not automatically a dataset defect.

A label records the outcome attached to an example, such as legitimate or fraudulent for a payment. Each outcome is a class. The label distribution describes how the examples are divided among those classes.

Class Imbalance and Data Representativeness: balance versus coverageWhy balancing target classes is different from representing the intended population and does not establish inclusivity or diversity.

A target class is a category a model learns to predict, such as billing or technical support. Class balance concerns how examples are distributed across those categories. Data representativeness concerns how well the data reflects the people and conditions the system is intended to serve.

AWS Data Exchange: selecting third-party data productsHow to use data product descriptions and provider documentation to assess provenance and population coverage when selecting curated, diverse data sources, without assuming marketplace availability guarantees balanced or inclusive data.

AWS Data Exchange helps you discover and subscribe to third-party data products. A product description is a starting point for assessing suitability, not proof that the data represents everyone your application will serve. Marketplace availability does not guarantee balanced or inclusive data.

AWS Glue DataBrew: profiling AI datasetsHow sample inspection and profile statistics help assess dataset completeness and balance, and why judging diversity and inclusivity also requires representative samples and source context.

AWS Glue DataBrew is a visual data preparation service. Before using data to train an AI model, you can examine individual records and summarize patterns across records. These views provide different kinds of evidence about data quality.

Overfitting and underfitting: distinguishing the errorsHow overfitting and underfitting differ in their relationship to bias, variance, and performance on training versus unseen data.

A model learns from training data, the examples used to fit it. Its real goal is to make useful predictions on unseen data, examples it did not learn from. This ability is called generalization.

Underfitting: systematic prediction errorsHow inadequate learned relationships produce high-bias errors on training and unseen data that can affect demographic groups differently.

Underfitting happens when a model fails to learn meaningful relationships in its data. Imagine predicting electricity demand with the same average value every hour. That rule cannot follow demand rising during working hours or changing with temperature.

Overfitting: failure to generalizeHow learning training-specific patterns creates high-variance predictions that can fail on unseen data, especially for sparsely represented demographic groups, and how to recognize and reduce that risk.

Generalization means making useful predictions on new examples, not just examples used for learning. Overfitting happens when a model learns training-specific details that do not reliably hold for the people or situations where it will be used.

Bias-Variance Tradeoff: balancing model fit and generalizationHow balancing statistical bias and variance reduces underfitting and overfitting without guaranteeing equal error rates across demographic groups.

A model should learn patterns that help it predict new examples, not just reproduce its training examples. Statistical bias is the systematic difference between a model’s average prediction across possible training datasets and the true relationship. High bias commonly causes underfitting: the model’s assumptions are too restrictive to capture important patterns.

Label Quality: checking labels for correctness and biasHow clear labeling criteria, independent human audits, and validation checks reveal label errors over time, including subgroup bias and unsupported factuality judgments.

A label is an answer attached to a data item, such as a category for a support ticket or a factuality rating for a generated response. Label quality means that the answer fits the intended task, follows clear rules, and has appropriate evidence. Calling a dataset ground truth does not make its labels correct.

Subgroup Analysis: revealing hidden performance disparitiesHow comparing task-relevant metrics across meaningful groups detects and monitors disparities, while label quality and sample sizes affect the reliability of those comparisons.

Subgroup analysis measures a system’s behavior separately for meaningful groups, such as languages, age ranges, device types, or accessibility needs. An aggregate metric combines everyone into one score, so a large group can hide poor performance for a smaller group. In the example below, 96% overall accuracy conceals 78% accuracy for Language B.

Amazon A2I: human review workflowsHow review triggers, reviewer instructions, and collected judgments enable human audits of model predictions and label quality without automatically detecting bias or guaranteeing truthfulness.

Amazon Augmented AI (Amazon A2I) helps you add human review to machine learning applications. A human review workflow defines the review task, the workforce that performs it, and where results are stored. A review of an individual item is called a human loop.

Recognize the importance of transparent and explainable models10 lessons
Model Transparency: visibility into a modelHow documentation of a model’s development, intended use, and limitations helps stakeholders evaluate it, even when its decision process is opaque.

Model transparency gives people visibility into a model and how it was developed and is used. Documentation can describe its training data, testing, intended purpose, known limitations, and owner. These details help stakeholders—such as developers, reviewers, and users—judge whether it is suitable for a task.

Post-Hoc Explainability: explaining trained model behaviorHow local and global explanations help people investigate an opaque model’s predictions, and why these explanations are approximations rather than proof of causality or correctness.

Post-hoc explainability applies explanation methods to an already-trained model to help people investigate its behavior. It is useful when the model is opaque: its internal calculations are difficult for people to follow directly. An explanation can identify patterns worth investigating without exposing a complete decision process.

Model Transparency vs. Post-Hoc ExplainabilityWhy explaining an opaque model’s predictions does not make its internal decision process transparent.

Model transparency concerns what people can know about a model and how it was developed and operated. This lesson focuses on transparency of its internal decision process: whether a person can directly follow how the model turns inputs into a prediction. A small decision tree can provide this visibility because its actual conditions and branches can be followed.

License Entitlements: AI model and data usage rightsHow model and data licenses govern rights to inspect, use, modify, and redistribute AI assets, and why those rights do not guarantee explainability.

License entitlements are the rights an agreement grants you. For AI assets, those rights can cover inspecting materials, using a model or dataset, modifying it, and redistributing copies. These are separate permissions: permission to run a model does not automatically include permission to publish its files.

Interpretability-Performance Tradeoff: evaluating model choicesHow to compare models using measured predictive quality and explanation usefulness, balancing transparency against safety risks and operational costs rather than assuming complexity improves results.

The interpretability-performance tradeoff is the choice between how well a model works and how easily people can understand its behavior. A small decision tree can expose readable decision rules. A large ensemble combines many models and is usually harder to inspect. But complexity does not guarantee better predictions: an understandable model can match or outperform a complex one.

Adversarial Robustness: transparency under attackHow to balance useful model explanations against information that helps attackers, measuring interpretability, predictive performance, and adversarial resistance separately.

Adversarial robustness means continuing to meet defined accuracy and safety requirements when someone deliberately tries to cause failure. A threat model describes what the attacker knows, what they can change, and which failures matter. An explanation available to an attacker becomes part of those conditions.

Selective Disclosure: audience-appropriate transparencyHow to choose what information each audience receives to support meaningful explanations and oversight without exposing sensitive data or enabling misuse.

Selective disclosure means choosing what information to reveal, to whom, at what level of detail, and for what purpose. In responsible AI, it supports meaningful explanations and oversight without unnecessarily exposing personal data, confidential information, or details that enable misuse.

Human-Centered Design: explanations people can useHow user needs shape transparent AI explanations that help people understand limitations, provide feedback, and challenge or correct decisions.

Human-centered design starts with people and their tasks, not with a model. Work with intended users, domain experts, and people affected by decisions to understand their goals and the consequences of errors. An explanation succeeds when people can understand a result and take an appropriate next step.

Feedback Loops: closing the loop on AI decisionsHow understandable explanations and accessible correction mechanisms turn user feedback into accountable review and evaluated improvements rather than merely collected ratings.

A feedback loop connects an AI result to evidence about what needs improving, responsible review, and a measured response. A rating button collects a signal; it does not complete that loop. Users first need enough information to recognize a problem and explain what is wrong.

Feedback Loops vs. Human-in-the-LoopHow a closed cycle of feedback and evaluated change differs from placing human judgment within an AI workflow, and why either can exist without the other.

A feedback loop connects what an AI system does to evidence used to improve its future behavior. The cycle includes observing results, considering a change, evaluating it, and checking what happens after an accepted change is used. Its central question is: Does evidence about results feed back into evaluated change?

DOMAIN 514% of the exam · 27 lessons
Security, Compliance, and Governance for AI Solutions
Explain methods to secure AI systems1 free21 lessons
Least privilege: scoped permissions for AI workloadsFreeHow to restrict AI workload and agent permissions to required actions and resources using roles, policies, and access conditions.

Least privilege means giving an AI workload only the access needed for its intended job. A document-reading assistant needs to read approved documents, not delete them or change who can access them. This limits the damage from mistakes, compromised code, or misleading instructions received by an agent.

Encryption: protecting cloud dataHow encryption protects AI data at rest and in transit, and why confidentiality depends on protecting keys and controlling decryption permissions.

Encryption converts readable data, called plaintext, into unreadable ciphertext. Decryption recovers the plaintext using the appropriate cryptographic key. The main benefit is confidentiality: obtaining encrypted data alone should not reveal its contents.

Network Isolation: private connectivity for AI workloadsHow network boundaries and AWS PrivateLink endpoints let AI workloads reach services privately while restricting unwanted communication.

Network isolation means deliberately restricting which resources can communicate. An AI application may need to send prompts to a model service while being prevented from contacting unrelated systems. Isolation preserves justified connections rather than necessarily disconnecting everything.

AWS PrivateLink: private access through interface endpointsHow interface endpoints let applications privately reach supported services such as Amazon Bedrock without public internet access, while IAM permissions and encryption remain separate safeguards.

AWS PrivateLink lets applications privately reach supported services without sending their requests over the public internet. An interface endpoint provides an entrance to a particular service through network interfaces with private IP addresses in your VPC, your isolated AWS network.

Amazon Macie: sensitive data discovery in Amazon S3How Macie identifies sensitive data in S3 and reports findings that help protect data used by AI systems, without enforcing access controls or filtering model responses.

Amazon Macie is a managed data security service that discovers sensitive data in Amazon S3. For an AI system, that could mean finding personal or financial information in documents intended for training or retrieval.

Data Cataloging: documenting dataset originsHow catalog entries connect discoverable datasets and their versions to source records, ownership, and usage restrictions without proving data quality or granting access.

Data cataloging organizes descriptions of datasets. A catalog entry helps people discover a dataset, understand what it contains, and find its location. The actual data can remain in a database, object storage, or another source system.

Source Citation: linking AI artifacts to evidenceHow stable, versioned source references connect AI answers, datasets, and model documentation to inspectable evidence, with lineage records, catalogs, and SageMaker Model Cards supplying supporting context.

Source citation connects an AI artifact to evidence that an authorized person can inspect. An artifact can be an AI answer, a dataset, or model documentation. For an answer, the citation should identify the document and passage supporting a claim. For a dataset, source references can identify the original collection and the version used.

Data Quality: assessing fitness for useHow to turn purpose-specific quality requirements into measurable checks, including AI dataset checks, and use their results to accept, quarantine, or investigate data.

Data quality means fitness for a specific purpose. A daily inventory snapshot may be useful for demand forecasting but too old for deciding whether an order can ship now. Begin with the consuming application's needs and the consequences of wrong, missing, duplicated, or late data—not with a universal definition of “clean.”

Access Control: least-privilege access to dataHow permissions restrict people and workloads to the datasets and read, modify, or administrative operations their tasks require.

Access control determines whether a requester may perform an action on a resource. The requester can be a person or a workload, such as an application, data-processing job, or model-training job. The resource might be a dataset, table, or stored file.

Data Integrity: protecting AI datasets from improper changesHow validation, trusted checksums, controlled writes, and version history keep changes to AI datasets authorized, valid, and traceable.

Data integrity means keeping changes to data authorized, valid, and traceable. AI datasets need to change as new examples arrive or labels are corrected. The goal is not to prevent every change, but to prevent improper changes, including accidental damage and deliberate manipulation.

Data Integrity and Data Quality: preserved versus fit for useWhy evidence that data has not changed improperly does not establish that it is accurate, representative, or suitable for an AI workload.

Data integrity concerns whether data keeps its intended meaning and state without improper changes. It covers accidental damage and unauthorized manipulation. Integrity does not mean that data must never change: an approved, valid, traceable correction can preserve integrity.

Privacy-Enhancing Technologies: limiting disclosure in AI data processingHow to choose privacy-enhancing techniques that limit sensitive information revealed during AI data processing, accounting for re-identification risks, trust assumptions, and utility trade-offs.

Privacy-enhancing technologies (PETs) help process data while limiting sensitive information revealed to others. Privacy matters even when access is authorized: an analyst or model user may learn more than they need. Start by identifying the sensitive information, the intended computation, and who could see inputs, intermediate results, or outputs.

Amazon Inspector: automated vulnerability scanningHow Inspector continuously scans supported AWS workloads for vulnerabilities and produces severity-ranked findings with remediation guidance to help secure applications, including AI systems.

Amazon Inspector automatically assesses supported AWS workloads for security vulnerabilities. A vulnerability is a weakness, such as a flaw in a software package, that an attacker could exploit. Inspector helps you find these weaknesses before they cause harm; a finding does not mean an attack has occurred.

Encryption at Rest: protecting stored dataHow encryption and controlled access to keys protect stored AI data from disclosure, and why authorized application access can still expose it.

Encryption at rest protects data while it is stored. For an AI system, that can include training files, saved prompts, generated reports, model artifacts, and backups. Sensitive information can exist in any of these copies, not just the main dataset.

Encryption in Transit: protecting connections with TLSHow TLS protects data moving to and between AI application components, why certificate validation matters, and where protection ends when a connection terminates.

Transport Layer Security (TLS) protects data moving between two endpoints. It provides confidentiality, so someone watching the network cannot read the protected content, and integrity, so changes to that content are detected. For an AI application, that content can include prompts, retrieved text, credentials, and model responses.

Encryption at Rest vs. Encryption in TransitHow storage protection and network protection cover different stages of the data lifecycle, and why neither substitutes for the other.

The same data can move between several states. An AI application's report might travel to a storage service, remain saved there, and later travel back to a reader. Encryption at rest and encryption in transit protect different stages of that journey.

Data Loss Prevention: enforcing sensitive data boundaries in AI applicationsHow content inspection and disclosure policies prevent sensitive data from escaping through AI inputs, outputs, tools, and logs, and why encryption alone cannot prevent these disclosures.

Data Loss Prevention (DLP) protects sensitive information from leaving its permitted boundaries. A boundary might separate tenants, organizations, approved recipients, or geographic regions. A disclosure policy states which data is protected, who may receive it, which destinations are allowed, and what to do when a transfer violates the rules.

AWS CloudTrail: account activity audit recordsHow CloudTrail records who performed AWS actions, when and from where, and makes those records available as evidence for governance and compliance.

AWS CloudTrail records AWS account activity as events. These records help answer who did what, when, and from where. Activity can originate from the AWS Management Console, command line tools, applications, or AWS services.

AWS CloudTrail vs AWS Config: activity records and resource configurationWhen governance questions need CloudTrail's record of AWS activity rather than AWS Config's configuration history and rule evaluations.

AWS CloudTrail and AWS Config both support governance: checking that AWS resources and their use follow an organization's expectations. They provide different evidence, so begin with the question you need to answer.

Audit Logging: the audit record lifecycleHow audit records are preserved as evidence through storage, access, retention, and deletion policies that meet governance and residency requirements.

An audit record describes security-relevant activity: who acted, what they did, which resource was involved, when it happened, and the outcome. Preserving these records lets reviewers reconstruct events and check accountability. A record is useful as evidence only if it remains trustworthy and available when needed.

Confidence Scoring: scores and decision thresholdsHow to interpret confidence scores and use thresholds to flag potentially unsupported AI responses for review without treating scores as proof of factual correctness.

A confidence score expresses how strongly a system supports a prediction or response according to a particular method. It can help identify answers that need attention, especially when fluent language hides unsupported claims. Its meaning depends on how it was produced: always check the score's definition, scale, and direction.

Recognize governance and compliance regulations for AI systems6 lessons
AWS Config: resource configuration complianceHow AWS Config evaluates recorded resource configurations against rules and provides configuration history as evidence for compliance reviews.

AWS Config records the configurations of supported AWS resources so you can inspect their settings and changes over time. You enable recording and choose which supported resource types to record. Recording provides the evidence that configuration checks can use.

AWS Trusted Advisor: best-practice checksHow Trusted Advisor checks identify infrastructure risks and recommend improvements that support governance without certifying regulatory compliance.

AWS Trusted Advisor evaluates aspects of your AWS environment against AWS best practices. A check looks for a particular condition, such as a security group that allows unrestricted access to certain ports. Its results identify potential risks or opportunities and recommend improvements.

AWS Trusted Advisor vs AWS Config: recommendations and configuration complianceHow Trusted Advisor's AWS best-practice recommendations differ from AWS Config's configuration history and rule-based compliance evaluations.

AWS Trusted Advisor asks, “Where could this AWS environment improve?” It uses AWS best-practice checks to identify risks and recommend improvements, such as reducing unnecessary costs or addressing a security concern. Its main purpose is guidance on better use of AWS resources.

Data Retention: retention policiesHow retention policies define which data to keep, when retention starts and ends, and how legal holds affect disposal while balancing business, compliance, privacy, and cost.

A retention policy defines which data to keep, why it is needed, how long to keep it, and what happens afterward. Its scope identifies the data it covers, such as financial records or support tickets. It should also name a data owner responsible for the policy and its exceptions.

Management and governance: AI governance protocolsHow AI governance frameworks translate policies into assigned responsibilities, risk-scoped reviews, transparency standards, and team training requirements.

AI governance sets the rules for how an organization develops and uses AI. A governance framework organizes those rules into responsibilities, reviews, evidence requirements, and training. A policy states an expectation, such as protecting confidential information; a protocol explains who must do what to meet it.

AI Transparency: governance controlsHow governance policies translate transparency requirements into audience-appropriate disclosures and lifecycle evidence that assigned owners maintain and reviewers assess.

AI transparency means giving people relevant, understandable information about an AI system throughout its lifecycle. It covers AI use, intended purpose, limitations, development records, and accountability. It is broader than explainability, which concerns understanding how model outputs arise.

Other courses