AWS Certified Machine Learning Engineer - Associate (MLA-C02) Course

Amazon S3 stores objects in buckets. An application identifies an object by its bucket and key, then uses API requests such as GetObject and PutObject, usually through an SDK. This lesson uses S3 general purpose buckets: a key such as datasets/train.csv is an object name, not a file inside a real directory hierarchy.

A dataset reader might open /data/train.csv with ordinary file operations on either service. The important difference is what the operating system accesses underneath that path: an EBS block device or an EFS shared file system.

An S3 dataset often consists of many objects, such as CSV files, images, or Parquet files. An application retrieves each object by specifying its bucket and exact object key. In s3://training-data/events/day=2026-01-01/part-000.parquet, the bucket is training-data and the key is events/day=2026-01-01/part-000.parquet.

An Amazon EBS volume gives an EC2 instance a block device, like an additional disk. To read files already stored on that disk, you first attach the volume in AWS, then mount its existing filesystem inside the operating system. This lesson uses Linux.

Amazon EFS provides a shared file system that Linux clients access using NFS. Mounting connects that remote file system to a local directory, such as /mnt/datasets. Applications then use paths beneath that directory rather than making storage API calls.

Amazon RDS manages relational database engines. A relational schema defines tables, columns, data types, and constraints. For example, an orders table can hold a customer identifier that relates each order to a row in customers. A pipeline can use these relationships to combine records rather than expect every order to contain the complete customer profile.

An ML extraction job reads RDS tables through the database engine's SQL connection, using its endpoint, port, database name, and an engine-compatible driver such as a JDBC driver. Permission to call RDS management APIs does not grant permission to read table rows.

Choose a DynamoDB read operation from the keys you know. GetItem retrieves one item using its complete primary key: the partition key and, when present, the sort key. Query retrieves items with one partition-key value and can narrow them with a sort-key condition. For example, query one customer's orders within a date range when customer ID is the partition key and order date is the sort key.

Amazon EFS provides shared file storage over NFS for Linux-based workloads. Multiple clients can work with the same directories and files using ordinary file operations and POSIX permissions. Its storage capacity grows and shrinks as files are added and removed, without you provisioning a storage size.

Amazon FSx is a family of managed file systems, not one interchangeable storage service. Start with the protocol, client operating system, and file-system features your data loaders require. A fast file system is not useful if the application cannot mount it correctly.

An embedding is a list of numbers that represents content, such as a document passage. An embedding model produces these numbers so that related content can be found by comparing vectors. Amazon S3 Vectors stores embeddings and supports similarity queries; it does not generate the embeddings for you.

Data residency describes where data is stored geographically. Start with the requirement owner: identify the covered data classes, the permitted geography, and whether the rule also covers processing, logs, backups, and derived data. Do not assume that every privacy rule requires local storage or that encryption permits storage elsewhere.

Data retention determines how long data must be kept and when disposal is required or permitted. Storage tiering determines where that data is stored to balance cost and access performance. A change to one does not automatically change the other.

S3 Object Lock provides write-once-read-many protection for individual object versions. Use it when records, training datasets, or other evidence must remain unchanged and cannot be permanently deleted during a required period. Object Lock requires S3 Versioning. Once Object Lock is enabled on a bucket, you cannot disable it or suspend versioning.

Storage tiering matches data to a storage class using both how often it is read and how quickly it must be available. Separate data into useful groups: active training datasets, occasionally reused model artifacts, and rarely retrieved historical records can have different needs. Measure access frequency and the amount retrieved; an occasional full-dataset read can matter more than many small reads.

An S3 storage class applies to an object, so one general purpose bucket can hold objects in different classes. Choose using access frequency, acceptable retrieval delay, and resilience—not storage price alone. S3 Standard suits frequent access: it provides millisecond reads without a per-GB retrieval charge, although request and applicable data-transfer charges still apply.

Storage price-performance compares total cost with the useful work storage enables. The lowest price per stored gigabyte is not necessarily the lowest cost per completed data-processing batch. First apply eligibility requirements; only then compare cost efficiency.

Throughput is completed work divided by elapsed time. For a data pipeline, choose a clear completion boundary, such as records successfully committed to destination storage. Count useful completions, not submission attempts or retries.

Ingestion throughput is the amount of data successfully ingested per unit of time. Horizontal scaling increases processing capacity by adding workers, rather than making one worker larger. The extra workers help only when they can process different pieces of work at the same time.

Horizontal scaling changes the number of workers serving a workload. Adding workers is scale-out; removing them is scale-in. A worker might be an EC2 instance or a container running GPU inference code. The term describes what changes, not why it changes.

Capacity planning starts with how much data arrives, how long it must remain available, and how much processing delay is acceptable. Measure normal arrivals, short peaks, and peak duration. A daily average can hide a burst that overwhelms ingestion.

A CloudWatch metric is a time series of measurements. Within an account and Region, its identity includes its namespace, metric name, and complete set of dimensions. A namespace identifies the source, such as AWS/EC2; dimensions are name-value pairs that identify the resource or population being measured.

CloudWatch Logs centralizes application and system log events so an investigation does not depend on reaching the original host. For an application on Amazon EC2, configure the CloudWatch agent to collect the relevant log files and send them to CloudWatch Logs. EC2 monitoring alone does not collect arbitrary application log files.

A CloudWatch metric alarm evaluates a metric statistic against a condition. A static threshold is a fixed boundary, such as average inference latency greater than 500 milliseconds. Choose the comparison direction deliberately: high latency is bad, while low successful-processing throughput may be bad.

Amazon Kinesis Data Streams divides a stream into shards. Each shard has write limits for both bytes and records, plus read throughput and request limits. Either write limit can become the bottleneck: many small records can exhaust record capacity, while fewer large records can exhaust byte capacity.

An event records something that happened, such as a payment completing or a sensor reporting a temperature. A producer publishes events continuously to a retained log instead of sending separate requests to every application that needs them.

Amazon Data Firehose, formerly Amazon Kinesis Data Firehose, manages delivery of streaming records to a configured destination. You create a delivery stream, select its source and destination, and grant the permissions it needs. Firehose operates the delivery infrastructure.

Amazon Kinesis Data Firehose, now named Amazon Data Firehose, is a managed delivery service. Choose it when the main job is to move incoming records to a supported destination, such as Amazon S3, without writing a delivery application. You configure the destination; Firehose handles buffering and delivery retries.

Amazon MSK runs managed Apache Kafka clusters. A producer publishes records to a named topic, and consumers read those records using Kafka clients. Your applications still choose the topics, record keys, and processing behavior.

Amazon MSK and Amazon Kinesis Data Streams both ingest and retain streaming records so applications can read them independently and replay retained data. Either can supply events to analytics or machine learning pipelines. Their biggest difference is the interface applications use, not whether they support real-time ingestion.

A source connector is the application code that reads an external stream and introduces its records into Flink. Use a Kinesis connector for Kinesis Data Streams and a Kafka connector for Kafka topics, including topics hosted on Amazon MSK. Include connector dependencies compatible with the application's Flink runtime.

Amazon Kinesis Data Streams accepts records from producers and durably retains them for a configured retention period. Consumers read those records to do useful work. Reading a record does not remove it, so separate consumers can process the same input independently.

Amazon Kinesis Video Streams ingests, stores, and serves timestamped media from devices. A producer sends audio or video, while a separate consumer retrieves it for processing. The service manages the media stream; it does not run your ML model.

Comma-Separated Values (CSV) represents a table as a sequence of records, each containing fields in an agreed order. It is a flat format: it has no native structure for nested objects or lists. An optional header names the columns, but does not declare their types.

JSON is a text format for exchanging and storing data. An object contains name-value pairs with double-quoted names; an array contains an ordered sequence. For example, {"customer_id":"c42","purchase_total":29.95,"items":["book","pen"]} combines strings, a number, and a nested array. Values can also be objects, booleans, or null. Array order matters; object member order should not carry business meaning.

Columnar storage groups values by field rather than keeping every record's fields together. In CSV, an order's identifier, region, amount, and description appear together on a row. JSON also commonly carries fields together in individual records. An analytical query that totals amounts by region needs only a few of those fields, but a text reader generally must scan and parse the records.

Apache Parquet is a file format, not a database, query engine, or storage service. It stores data and schema information inside files that different processing tools can read. Its typed schema distinguishes values such as integers, strings, decimals, and timestamps, rather than leaving every value as untyped text.

ORC is a schema-aware, columnar file format for repeated analytical reads. Its schema records types such as integers, decimals, timestamps, strings, and nested structures. A compatible reader interprets these types without inferring each value from text.

Apache Parquet and ORC are open, typed columnar file formats for analytical work. Both support reading selected columns, compression, and metadata-based skipping in compatible readers. Neither is a query engine, storage service, or table transaction system, so these shared capabilities do not establish a universal winner.

Amazon Athena runs SQL against files in Amazon S3 without loading them into a database first. For the external S3 tables in this lesson, a table definition in the AWS Glue Data Catalog describes columns, data types, the S3 location, and how to interpret the files.

A join combines related rows from two datasets using a matching condition. For example, joining orders to customers on customer_id adds customer attributes to each matching order. Unlike a union, which appends rows, a join matches records and combines their columns.

Data deduplication removes redundant records, not simply repeated values. Start by defining the dataset's grain: what one row represents. For a current order-line snapshot, the grain might be one row per (order_id, line_no). A customer ID alone is too broad because one customer can place several orders, each with several lines.

An AWS Glue Spark job reads datasets, transforms distributed records, and writes the result. Within the job, you can use Glue DynamicFrames, Spark DataFrames, or both. They are representations of data inside the job—not separate processing services.

Amazon EMR runs Apache Spark jobs that process data across multiple workers. A Spark DataFrame represents distributed rows with named, typed columns. You describe operations on those columns rather than writing a loop over every record.

A join needs matching keys to meet. A shuffle join can repartition both inputs by the join key, moving rows between workers. A broadcast join instead replicates one input to the participating executors, the Spark processes that run tasks. Each executor can then join its existing partitions of the other input locally.

A vector index organizes stored embeddings for similarity search. Configure it for the output of your embedding model: the vector’s dimension, its supported data type, and the similarity measure recommended for that model. For example, an index expecting a different number of values cannot accept the model’s vectors.

Approximate nearest neighbor search reduces search work by accepting that it may miss true nearest neighbors. Start with an exact-search benchmark: for representative queries, save the exact top-k vector IDs from a fixed dataset. Use the same embedding model, preprocessing, distance measure, and metadata filters for exact and approximate searches. Matching vector dimensions alone does not make different embedding models compatible.

Amazon RDS for PostgreSQL can store and search embeddings using the pgvector extension. An embedding is a list of numbers that represents content for similarity comparisons. An embedding model generates these numbers outside the database; pgvector stores them and calculates distances between them.

A vector index stores embeddings alongside document text, identifiers, and metadata. Map the embedding field as knn_vector, with its dimension equal to the number of values your embedding model produces. A model that returns 768 values needs dimension: 768. Stored and query vectors must use compatible embedding models and preprocessing; matching dimensions alone does not make different models compatible.

In Retrieval Augmented Generation (RAG), a retriever finds relevant content and supplies it as context to a language model. A vector store holds embeddings, numerical representations used to find content with similar meaning. Both Aurora PostgreSQL and Amazon OpenSearch Service support vector retrieval. The choice depends on what retrieval must do alongside similarity search.

A schema is the contract that each ingested record must follow. It defines field names, meanings, data types, and rules such as whether a value is required. Define this contract before accepting a tabular dataset, rather than assuming that the first few rows describe every possible value.

A semi-structured record can contain nested objects, arrays, and optional fields. For example, {"id":"clip-7","metadata":{"language":"en"},"labels":["speech"]} keeps metadata together and represents labels as a list. Preserve those relationships during ingestion rather than turning every field into a string.

Preserve text, images, and audio as original files in Amazon S3. An S3 object stores the file's bytes in a bucket under an object key. Keys such as sources/a17/call.wav organize objects; their prefixes are not filesystem directories.

Structured data follows an explicit, consistent schema: named fields with defined meanings and data types. In a customer table, each row represents a customer, while fields such as customer_id, registration_date, and balance describe that customer. Applications can compare balances or filter dates using those definitions.

Semi-structured data uses named fields, tags, or nested elements to organize records without requiring a uniform relational table. For example, JSON purchase events can contain identifiers, nested customer details, and arrays of items. One event can have a coupon field while another omits it. Flexible fields still have meanings and expected types.

A metadata schema is a contract for the attributes that describe a resource. It defines each field's meaning, type, requiredness, and allowed values. Text, images, and audio can share this contract even though their content has different formats.

AWS DataSync copies files and objects between supported storage locations. It moves dataset contents such as text, images, and audio without parsing or transforming them. A location describes a storage endpoint; a task connects a source location to a destination location and defines how to copy.

Amazon SageMaker Feature Store stores named features, such as a customer's purchase count. A feature group defines the features that belong together, their data types, and the stores used to hold their records. Before ingestion, define the schema using String, Integral, or Fractional feature types. Incoming feature names and values must match those definitions.

A Data Wrangler flow records the transformations that turn imported data into prepared data. Select the node you want to clean and add transformations from there. Each transformation uses the output of the steps before it, so changing an earlier step can change later results.

AWS Glue DataBrew separates interactive preparation from batch execution. A dataset identifies an input source and its format or connection settings. A project opens a sample of that dataset in a visual workspace. As you apply transformations, DataBrew records them as ordered steps in a recipe.

Amazon SageMaker Data Wrangler and AWS Glue DataBrew both provide visual data preparation. Both can help handle missing values, remove duplicate rows, and transform columns. A requirement such as “clean this dataset without writing extensive code” does not, by itself, distinguish them.

Amazon EMR and AWS Glue can both run Apache Spark transformations that combine sources, clean records, and write prepared datasets. A join or aggregation alone rarely decides between them. The important difference is how much control the job needs over its runtime: the Spark version, libraries, configuration, and surrounding software.

AWS Glue DataBrew fits visual, batch-oriented data preparation. An analyst or engineer can inspect data, preview cleaning operations, and save those operations as a reusable recipe. Choose it when its built-in transformations cover the work and the team benefits from seeing how changes affect the data.

A point-in-time join reconstructs the features a model could have used for a historical prediction. Each training example needs an entity identifier, a unique example identifier, and a prediction cutoff. The cutoff is the time the prediction was made, not the later time when its outcome became known.

Training-serving skew occurs when training inputs do not faithfully represent the inputs available during inference. Matching column names and data types is not enough: a purchase count over a calendar day differs from a count over the preceding 24 hours.

AWS Lambda can transform streaming records in batches. In this pipeline, Amazon Data Firehose buffers incoming records and invokes Lambda synchronously with several records in one event. Your handler must process the records in that event, rather than assume each invocation contains just one. Buffering settings balance waiting time against work per invocation.

A streaming record can reach transformation code more than once, including when processing succeeds but its success is not recorded before a failure. Idempotency means that repeating the same logical event does not produce another intended effect. The code may execute again; the output must not become another independent result.

Event time is when an event occurred. Ingestion time is when an ingestion system received it, and processing time is when the application handled it. A purchase made at 10:02 but processed at 10:08 belongs to the 10:00–10:05 event-time window, not the 10:05–10:10 window.

A windowed feature summarizes events for one entity over a bounded period: purchases per account, requests per device, or average temperature per sensor. Define the entity key, timestamp field, window, and aggregate together. Use event time to describe when activity happened; processing time describes when the system handled it. A purchase made at 10:02 but processed at 10:07 belongs to the earlier interval under event-time windowing.

Numeric scaling changes the size of numeric inputs without changing what they measure. A distance-based model can otherwise give income more influence than age simply because income has larger numbers. Scaling is often useful for distance-based methods, regularized linear models, and neural networks; tree-based models generally do not require it.

Min-max scaling transforms each numeric feature independently using reference minimum and maximum values. For a target range from a to b, the rule is x_scaled = a + ((x - x_min) / (x_max - x_min)) * (b - a). The reference minimum maps to a, and the reference maximum maps to b.

Feature engineering changes how an input represents a problem. For example, subscription length can remain an exact number of days or become a membership band. Binning replaces numeric values with the ranges they belong to. A bin defined as 10 <= x < 20 gives 12 and 18 the same representation. This can expose threshold effects when broad ranges matter more than small differences.

Binning replaces a numerical value with membership in an interval. Account ages of 8 and 20 days might both become “7 to below 30 days.” This represents range-based patterns but discards differences within each interval. Unlike scaling, binning does not preserve continuous variation.

Tokenization splits text into units called tokens and maps each unit to an integer token ID in a vocabulary. The result is a sequence of IDs that a model can process. A token can be a word, part of a word, punctuation, or another text unit; it is not necessarily a whole word.

An embedding model converts a supported input into a vector of numbers. Choose the model by its supported input types, languages, and intended use—not simply because it returns a vector. In Amazon Bedrock, you select an embedding model through its model ID; a text-generation model is not a substitute.

An embedding represents text or an image as a numerical vector. An embedding model produces that representation; cosine similarity compares the directions of two representations. Related directions can indicate related content when the model was trained for the intended task.

Dense retrieval compares a query embedding with stored passage embeddings. It is useful when questions and documents express similar meanings with different words: “recover account access” can match “reset a forgotten password.” Similarity is a ranking signal, not proof that a passage answers the question.

An embedding model is useful when it retrieves content people need. Build a representative query set from real search needs, including common requests, domain terms, supported languages, and difficult wording. Have reviewers identify relevant passages in a fixed document collection. Keep a held-out evaluation set separate from queries used while tuning.

A tokenizer converts text into a sequence of numerical token IDs. Two parts work together: segmentation rules determine how to split the text, and a vocabulary maps the resulting pieces to IDs. An ID identifies a vocabulary entry; its numerical value does not express the meaning of the text.

Lemmatization maps an inflected word to its dictionary form, called a lemma. For example, cars becomes car, and the past-tense verb went can become go. It uses language-specific vocabulary, grammatical rules, or trained models rather than simply removing letters.

Label-preserving augmentation creates new ways to express an example without changing the meaning that determines its label. For a billing-support classifier, “Invoice INV-42 was charged twice. Do not cancel my plan” can carry the label duplicate_charge. “I was charged twice for INV-42; please keep my plan active” expresses the same request differently.

Back-translation creates candidate training examples by translating text into an intermediate language and then translating that result back into the original language. Amazon Translate performs the translations; your application connects the two calls. The round trip can change wording while keeping the intended meaning.

Amazon Textract extracts printed and handwritten text from document images and scans without requiring you to train an OCR model. Optical character recognition (OCR) turns the marks on a page into machine-readable text. Clear, readable input matters: blur, poor contrast, and difficult handwriting can reduce accuracy.

A RAG source corpus is the collection of documents approved to supply evidence for answers. Extracted text is not automatically ready for that job: broken words, repeated page furniture, and conflicting versions can all become retrieved passages. Prepare the corpus before chunking, rather than expecting smaller chunks to repair bad sources.

Document chunking divides source content into passages that can be embedded, indexed, and retrieved for RAG. It is not summarization: preserve the source evidence rather than replace it with a shorter interpretation. Start with clean, correctly ordered text; scanned pages may need OCR and layout extraction before meaningful boundaries are available.

Metadata extraction turns document information into structured fields, such as document_id, revision, or effective_date. In retrieval augmented generation (RAG), these fields connect retrieved chunks to their source and support filtering, citations, and troubleshooting. An embedding captures semantic similarity; it does not replace explicit dates, versions, or identifiers.

Data masking changes selected sensitive values while keeping useful context. For ML preparation, use static masking: create a modified dataset whose stored contents are suitable for downstream use. Hiding a value in a notebook display does not change the underlying dataset.

Data anonymization aims to prevent information from being associated with an identifiable person while preserving value for an intended use. Start by separating two identification routes: direct identification and linkage through other attributes.

Data masking hides or replaces selected values so people can work with data without seeing information they do not need. For example, a support view can replace a customer's email address with asterisks while leaving transaction details visible. This limits disclosure of that field; it says nothing by itself about whether the customer can still be identified.

Pseudonymization replaces a direct identifier with a substitute while keeping the information needed to connect it to an identity separately protected. If customer C104 becomes token t7, every transaction for that customer can carry t7 instead. Analysts can join purchases and calculate customer totals without receiving C104.

Pseudonymization replaces an identifier with a substitute while preserving an association with a person. Replacing a customer's email address with customer_7Q can let analysts join that customer's purchases without seeing the email. A separately protected lookup table can connect the token back to the original identity.

Amazon Macie answers a discovery question: which data in Amazon S3 contains sensitive information? It analyzes supported S3 objects and produces findings that help you locate and investigate sensitive data, including personally identifiable information (PII). A finding does not mask, redact, or rewrite the object.

Continued pre-training and supervised fine-tuning both adapt a pretrained model through additional training. The useful distinction is what the model is trained to predict and where the correct targets come from. Here, we compare them for generative language models.

A domain corpus is the collection of text used to continue training a model. For industrial maintenance, it might contain equipment manuals, inspection reports, and technical publications. Start with documents you have permission to use for training. Preserve an access-controlled, immutable source snapshot rather than repeatedly downloading whatever happens to be current.

Fine-tuning adapts a pretrained model through additional training on a targeted dataset. Training updates existing model parameters or trains additional adaptation parameters. The result is a learned adaptation that persists across requests, rather than information supplied only in a prompt.

Task-specific fine-tuning learns from examples of an input and the output the model should produce. For a support classifier, the input might be a customer message and the target output an approved category. For a summarizer, the target is a reviewed summary—not simply whatever an agent originally wrote.

Model distillation identifies the source and purpose of a learning signal: a teacher model supplies signals so a student can learn selected teacher behavior. The student is typically smaller, with the aim of making that behavior less expensive to serve. It learns from the teacher rather than simply receiving a copy of the teacher’s parameters.

For response-based model distillation, a teacher generates answers that become learning targets for a student. Each training example connects an application prompt to a desired response. The preparation work starts with representative prompts, not with generating as many answers as possible.

A dataset profile summarizes the records examined. Start by identifying the row count, columns, filters, and time window. Completeness measures whether expected values are present. If 950 of 1,000 rows contain a non-null customer ID, non-null completeness is 95%. Check how the profiler treats empty strings, whitespace, and placeholders such as unknown: these can count as populated without providing useful information.

Schema validation checks each record against a declared contract. Specify allowed fields, required fields, types, nullability, and value constraints. Choose a schema format and validator that support the consuming system; different formats do not enforce identical rules.

Data profiling discovers observed characteristics of data. It measures what is present, such as observed types, missing values, distinct values, and numeric ranges. Schema validation compares data with declared structural expectations, such as field names, types, and nullability. Profiling produces observations; schema validation produces results against a contract.

A useful data requirement names what a model needs and makes acceptance measurable. Replace “labels should be complete” with “every row in this training batch must have a non-null label.” Define the scope, the metric, and the acceptance threshold. Also name the owner who approves the requirement and the action taken on failure.

Data profiling describes what is observed in a dataset. A profile might report that customer identifiers are populated in 98% of records, that a column contains three country-code formats, or that the newest record is from yesterday. These findings help you understand the data; they do not, by themselves, decide whether it is acceptable.

Schema validation checks data against a declared contract: required fields, types, allowed values, and other explicitly configured constraints. Data quality asks whether the data is fit for its intended use. Structural conformance is one part of that assessment, not a substitute for it.

An AWS Glue DataBrew recipe job applies transformation steps and produces prepared data. A successful run tells you that processing completed; it does not establish that the output meets business requirements. For example, converting an amount column to a numeric type can succeed even when some amounts are negative.

AWS Glue Data Quality checks data against expectations you define. Those expectations form a ruleset, written in Data Quality Definition Language (DQDL). Glue evaluates the rules against the dataset supplied to the evaluation, so choose a point in the ETL job where the data should meet those expectations.

An annotation schema defines the target information attached to each example: its format, allowed values, and how many annotations are permitted. Choose it from the model’s prediction task. A category answers “what kind?”, a bounding box answers “where is each object?”, and a segmentation mask identifies which pixels belong to a region. These annotations are not interchangeable.

In data annotation, a label describes an example for a learning task: a document category, an object boundary, or a sentiment class. Human-in-the-Loop annotation uses people to check candidate labels before those labels become training targets. A model or another annotator can propose a label; the reviewer must compare it with the original example and the annotation instructions, not simply approve the suggestion.

Label quality means that an annotation fits the intended task and follows an explicit labeling policy. Calling a dataset “ground truth” does not establish that its labels are correct. Before reviewing annotations, define the allowed classes, class boundaries, required evidence, exceptions, and treatment of uncertainty.

An Amazon SageMaker Ground Truth labeling job turns unlabeled examples into annotations for training. Start with data in Amazon S3 and an input manifest: a JSON Lines file whose records identify the examples. For files such as images, a record can use source-ref to point to an S3 object.

Amazon Augmented AI (Amazon A2I) adds human review to machine learning workflows. Choose it when a model has produced an output and people need to check, correct, or judge that output as part of the application’s workflow.

Representation bias arises when training data inadequately covers people or conditions the model must serve. Start with the intended population, then compare each relevant group's record count and share with that population. Check combinations, such as language and region, because broad groups can hide missing coverage. Equal group sizes are not automatically the right goal.

Data shuffling changes record order without intentionally changing values, labels, or overall frequencies. If files list all approved applications before rejected ones, taking the first portion for training can create an unrepresentative split. Feeding that order into training can also produce long runs of similar examples in successive minibatches.

A useful split imitates the predictions you will make after deployment. Define the prediction population: new customers, new patients, future transactions, or another specific population. Use training data to learn model parameters, validation data to choose models and settings, and an untouched test set to evaluate the final choice.

A train-validation-test split assigns data three different jobs. The training set supplies examples from which a model learns parameters, such as weights. The validation set guides development decisions, such as choosing hyperparameters, comparing algorithms, or selecting a classification threshold.

Data augmentation creates additional training variations while preserving the correct label. Target it where the model lacks useful coverage, rather than making every class equally large. Start by counting training examples by class, relevant group, and important intersections, such as defect type within a camera condition.

Representativeness means matching a defined deployment population, not simply collecting many examples. Specify the users, operating conditions, time period, and prediction unit the system will serve. For a system using numeric records, text, and images, the unit might be one support case with transaction values, a message, and an attached photo.

Start a distribution audit by defining the unit you count, the relevant groups, and a justified reference distribution. Count people, documents, or images consistently; repeated records can make coverage look better than it is. Compute each group share as its count divided by the total. Report counts alongside percentages, especially for small groups.

Data representativeness asks whether a dataset reflects a defined deployment population and operating environment. Fairness asks whether the system treats people equitably under explicitly chosen criteria. One concerns what the data reflects; the other concerns how people are treated.

Subgroup analysis compares data coverage and model behavior within meaningful groups, rather than trusting one overall score. Start with the people affected, the decision being made, and the harm an error could cause. Define groups and comparisons before evaluation where possible. A useful group reflects a real use condition, not just an available column.

Class imbalance means that some labels have fewer examples than others. Equal class counts are not always the goal: failures and fraud really can be rare. Start by counting labels and training a baseline. If only 1% of examples are positive, predicting negative every time gives 99% accuracy but finds none of the positives.

Class resampling changes the class mix presented during training. It can give a rare class more influence, but a balanced training set is not the goal by itself. For example, when almost every transaction is legitimate, high accuracy can hide a model that misses most fraud.

SMOTE, the Synthetic Minority Over-sampling Technique, adds minority-class examples to classification training data. Unlike random oversampling, which repeats existing records, SMOTE generates feature vectors between existing minority examples. Unlike undersampling, it does not require removing majority-class records.

Class-aware sampling controls how often training selects examples from each class. It can present rare-intent documents or uncommon-defect images more often while keeping every example paired with its original label. Sampling with replacement allows an example to be selected again without storing extra copies.

A validation gate decides whether a prompt–response example is eligible for an approved training dataset. It checks the example against trusted evidence and approved rules, rather than accepting it just because it arrived successfully or contains readable text.

A fine-tuning example teaches a behavior: given this prompt and context, produce this approved response. A quality gate is a rule that an example must pass before admission to the training dataset. The first gate checks structural validity against the selected model and training platform: readable records, required fields, supported roles, appropriate field types, and applicable length limits. A file that parses successfully can still contain invalid examples.

Anomaly detection identifies observations that differ from an expected pattern. An unusual value is a reason to investigate, not proof of an error. A large order may be normal for a wholesale customer but unusual for a retail customer. A long processing time may be expected for a large batch but unusual for a small one. Your baseline should describe comparable observations, not simply everything you have collected.

Missing value imputation replaces absent values so records remain usable. First identify how absence is recorded: NULL, NaN, empty strings, or sentinels such as -999. Normalize these only after checking the field definition. A recorded weight of zero is not automatically a missing weight, and an unanswered question is not automatically a negative answer.

A benchmark dataset is a common set of test cases used to compare models. Start by identifying the application's tasks, users, languages, input lengths, and required behavior. A support assistant, for example, needs cases covering routine questions, ambiguous requests, and situations requiring escalation—not just general knowledge questions.

A model's context window is the information it can work with during one inference request. For text, capacity is measured in tokens, not words or characters. Count system instructions, conversation history, examples, retrieved passages, tool definitions, and tool results—not just the latest question. Stored conversation history is visible to the model only when the application supplies it.

A representative comparison test asks how candidate models handle the work your application will actually receive. Start with the intended tasks, users, and operating conditions. Select cases that reflect that work, rather than choosing examples because a candidate answers them well. A public benchmark can help narrow the candidates, but it does not establish suitability for your application.

A task-aligned metric measures a result that matters for the intended use. Start with the decision the model supports and the consequences of getting it wrong. For classification, accuracy is the fraction of predictions that are correct, but it can hide poor detection of a rare class. If only 1% of cases need escalation, predicting “no escalation” for every case gives 99% accuracy while missing every required escalation.

A scoring rubric tells an assessor what to judge and what each score means. For generated answers, useful criteria include correctness, relevance, completeness, groundedness, and instruction following. Score these separately: a clear, relevant answer can still contain false claims. Give the assessor the user request, the response, and any evidence needed to judge it.

Fine-tuning creates a persistent model adaptation from targeted examples. Use it to improve a repeatable behavior, such as assigning support categories or producing summaries in an established format. Define success before training: for example, correct category selection and valid output structure, with acceptable safety, latency, and cost.

Parameter-efficient fine-tuning (PEFT) adapts a pretrained model by training a limited set of parameters. Start with a prompted baseline: clear instructions and examples may already meet the need. Consider PEFT when representative examples are available and you need more consistent task behavior, terminology, style, or output structure. Use retrieval for changing facts or document-grounded answers; retrieval and PEFT can work together.

Full fine-tuning and parameter-efficient fine-tuning (PEFT) both adapt a pretrained model. The difference is which parameters training can change. Full fine-tuning normally makes all original model weights trainable. PEFT freezes most or all original weights and trains a small subset of parameters or additional components. A frozen weight still participates in computation; training simply does not update it.

Instruction tuning teaches a pretrained model how to carry out requests using supervised demonstrations. Each example contains an instruction, optional input or context, and a desired response. Pretraining supplies broad language capabilities; instruction tuning builds on them to teach task behavior, such as extracting fields, summarizing reports, or answering in an application's preferred style.

Supervised learning learns from examples that pair inputs with known outputs. The inputs are features, such as purchase frequency or account age. The known output is a target, also called a label: for example, whether a customer later canceled a subscription. During training, the model adjusts to reduce disagreement between its predictions and those targets.

In supervised learning, each training example pairs input features with a known target, also called a label. An algorithm fits a model to predict that target from the features. For example, account activity can be the input and a customer's later cancellation outcome the target. At prediction time, the model receives features; the outcome is not yet known.

Choose a structure-discovery method by asking what someone needs to do with its output. Clustering discovers groups of similar observations, such as customer segments based on purchasing behavior. Its output is group membership, not predefined business categories: analysts must inspect each group before naming it or deciding how to act on it.

An interpretable baseline, such as a small decision tree or sparse linear model, gives you a reference for both prediction quality and understandable behavior. Compare it with complex candidates rather than assuming an ensemble or neural network will improve results. Even a linear model can be difficult to understand when its inputs are heavily transformed or correlated.

Inference latency is the time needed to obtain a model's output. Before comparing models, define where the timer starts and stops. Model execution latency covers model processing; end-to-end latency measures the application's full wait, including network transport, authentication, retrieval, queueing, and response delivery. A fast model can still sit inside a slow application.

AWS Marketplace is a catalog for finding and acquiring third-party products. For ML, start with a short requirements sheet: the task, input and output formats, domain and languages, interpretability needs, and quality and latency targets. Separate must-haves from preferences so a popular listing does not displace a business requirement.

Total Cost of Ownership (TCO) estimates the complete cost of a solution over an agreed evaluation period. For ML, this includes preparing data, developing or adapting a model, deploying it, operating it, updating it, and retiring or replacing it—not just training or inference charges.

Start with a credible baseline: the current workflow, deterministic rules, or a simple statistical model. Define the business outcome, minimum acceptable quality, response-time limit, and evaluation period. AI must justify its additional effort against an approach that could actually meet the need. For exact contractual calculations, authoritative rules are usually the right baseline, not generated answers.

Total Cost of Ownership (TCO) estimates what a solution will cost over its complete lifecycle within an agreed evaluation period. Cost-benefit analysis asks whether the expected benefits justify the investment. One measures the cost commitment; the other supports a decision about value.

Transfer learning reuses knowledge learned by a pretrained model. Its internal representations describe useful patterns, such as image shapes or language structure. Reusing those patterns can reduce the target data and training compute needed compared with learning comparable capabilities from scratch.

Model hosting means running a trained model so an application can send input and receive predictions. Both Amazon EC2 and Amazon SageMaker can support this. The main difference is where your team's operational responsibility begins. Here, SageMaker hosting means the managed inference capabilities of Amazon SageMaker AI.

Batch inference applies a trained model to a collected dataset when predictions do not need to return immediately to a user. Both AWS Batch and Amazon SageMaker batch transform can run this work without a persistent inference endpoint. The main choice is not whether the work is offline, but who controls the prediction workflow.

A model trained outside AWS does not have to be retrained on AWS to be hosted there. Amazon Bedrock Custom Model Import and Amazon SageMaker AI both provide routes to inference, but they accept different kinds of serving workloads.

Amazon Bedrock AgentCore provides managed infrastructure for agent applications. Its capabilities address execution, memory, identity, tool connectivity, and operational visibility. It is framework-independent: your application can use its chosen agent framework rather than adopting a reasoning framework supplied by AgentCore.

In a retrieval-augmented generation workflow, or RAG, vector search finds passages that could help answer a question. Documents are usually divided into chunks, with an embedding stored for each chunk. At query time, a compatible embedding represents the question. Search compares that query vector with stored vectors and returns a limited candidate set.

In RAG, similarity search retrieves chunks whose embeddings are close to the query embedding. Use a compatible query embedding model and the similarity measure recommended for that model. A high similarity score signals closeness in that embedding space, not factual correctness or a probability of relevance.

Retrieval Augmented Generation (RAG) retrieves relevant source material and includes it in a model's prompt. An embedding is a numeric representation of content used to compare meaning. Aurora PostgreSQL can store embeddings alongside the text and relational metadata that an application needs.

Model complexity is a model’s flexibility to learn patterns: for example, a deeper decision tree can represent more detailed rules. Extra flexibility can improve predictions, but fitting the training examples better is not enough. Generalization means predicting well on examples the model did not learn from.

A training curve shows how a metric changes while a model learns. For loss, lower is better. Training loss measures fit to the examples used to update the model. Validation loss measures performance on held-out examples that do not update its parameters; use it to guide tuning and checkpoint selection, not the test set.

Automated hyperparameter tuning runs training trials with different configurations and compares their validation results. In Amazon SageMaker AI automatic model tuning (AMT), you supply the training setup, search space, objective, search strategy, and resource limits. Each trial learns model parameters; the tuner chooses hyperparameter values.

Model compression reduces the resources a model needs while preserving enough predictive quality for its use. Separate three targets: model file size, runtime memory, and computation. A smaller file does not guarantee faster inference, and archive compression alone does not reduce inference work.

AWS Inferentia accelerates inference: running a trained model to produce predictions. Amazon EC2 Inf1 instances use first-generation Inferentia chips; Inf2 instances use Inferentia2. Inf2 provides more accelerator memory per chip and is designed for more demanding workloads, including large language and diffusion models. That does not make it the cheapest choice for every model.

Amazon Comprehend analyzes existing text with pre-trained natural language processing models. Your application selects an operation for the insight it needs; you do not train a model or deploy a custom endpoint for these operations. A single text can be sent to several operations, but one operation does not automatically return every kind of insight.

Amazon Textract extracts text and document structure from document images, including scanned documents. It can recognize printed and handwritten text and extract relationships such as form keys and values or table cells. It answers questions such as “What words are on this page?” and “Which value belongs to this field?”

Amazon Rekognition provides pretrained visual analysis through APIs, without requiring you to train or host a model. Start with the question your application must answer: what is visible, whether content needs review, where faces are, or whether faces match.

Both services can recognize text in images, but they target different jobs. Amazon Rekognition fits scene text: words appearing within a wider picture, such as a shop sign, a product name on packaging, or lettering on a vehicle.

Amazon Transcribe converts speech into text using automatic speech recognition. It can produce punctuation and word timestamps, making recordings useful for searching, reviewing conversations, and creating captions. Recognition is not perfect: noise, overlapping speech, and unfamiliar terms can affect the result.

Amazon Transcribe converts speech into written words. This is speech recognition: a recorded interview or spoken customer message becomes a transcript that people and applications can read.

An Amazon SageMaker AI managed training job runs training code in a container on compute resources provisioned for that job. You configure the workload; SageMaker provisions the resources, runs training, saves the outputs, and releases the training resources afterward.

TensorFlow on AWS uses ordinary TensorFlow and Keras code inside an AWS execution environment. AWS Deep Learning AMIs provide an instance-based environment on Amazon EC2; AWS Deep Learning Containers provide container images. Amazon SageMaker AI can manage training jobs. Choose a TensorFlow version compatible with your code and, for GPU training, its accelerator libraries.

PyTorch supplies the training code; AWS supplies the compute and storage. Run your script on Amazon EC2 using a PyTorch Deep Learning AMI, or submit it as an Amazon SageMaker AI training job with a PyTorch container. Here, use one training process on a CPU or a single GPU. Make training files in Amazon S3 available locally before reading them, and give the compute an IAM role with the required S3 permissions.

In SageMaker AI automatic model tuning, select Strategy="Random" to sample hyperparameter configurations independently of earlier trial results. Each configuration supplies settings to a training job; training learns the model parameters. The search returns the best configuration observed, not a guaranteed global optimum.

Bayesian optimization guides expensive evaluations toward promising configurations. In hyperparameter tuning, each trial trains a model with one configuration and measures its validation performance. Hyperparameters, such as learning rate and tree depth, control training; they are not the weights learned during training.

A trial trains and evaluates a model using one hyperparameter configuration, such as a learning rate and tree depth. Both Bayesian optimization and random search compare trial results using a validation objective. The important difference is whether those results influence the selection of later configurations.

Early stopping ends training before its maximum number of epochs, steps, or boosting rounds when a validation metric stops improving meaningfully. It reduces work that provides little benefit and can limit overfitting. Keep a maximum training budget as a separate limit: early stopping does not guarantee that a plateau will occur.

Mixed precision training uses lower precision for suitable operations and FP32 where extra numerical protection is needed. Matrix multiplications and convolutions often benefit from FP16 or BF16 arithmetic. Smaller activations can also reduce memory use, but total training memory does not necessarily halve: parameters, optimizer state, and other buffers may remain in FP32.

Synchronous data parallelism gives every worker a complete model replica. Workers process different samples, compute gradients, and combine those gradients before applying matching optimizer updates. An all-reduce combines contributions and returns the result to every worker; these are not independent training jobs.

Training error measures mistakes on examples used to fit the model. Validation error measures mistakes on held-out examples and helps you choose between model configurations. Here, error means a task-appropriate measure where lower is better.

Overfitting happens when a model learns training-specific details rather than relationships that hold for new examples. A lower training error is not always progress: the model may be fitting noise or accidental correlations. Controls against overfitting aim to preserve useful relationships while reducing this training-specific fit.

Underfitting means a model has not captured meaningful relationships in the data. Once you have checked data quality and preprocessing, look for limits in its inputs, capacity, regularization, or training time. The goal is to remove a specific limit, not simply make every setting larger.

Catastrophic forgetting is a substantial loss of previously learned capabilities after additional training. A neural network's parameters, the learned values that shape its predictions, often support several capabilities at once. Fine-tuning changes those shared values to improve the new training objective. Those changes can disrupt patterns that earlier tasks depend on.

Catastrophic forgetting is a substantial loss of previously learned capabilities after additional training. For example, fine-tuning a model on support conversations can make it worse at summarization because updates change shared parameters that supported that earlier capability.

An ensemble combines predictions from multiple models into one result. Its advantage comes from complementary errors: one model gets some examples right that another gets wrong. Different algorithms, training samples, or feature sets can create this diversity. Compare models on the same validation examples and inspect where they fail together. Different model names alone do not prove useful diversity, and disagreement alone does not prove improvement.

A selective cascade starts with an inexpensive model and invokes a more capable model only for selected requests. The acceptance check decides whether the first result is good enough to return or needs escalation. An incorrect result accepted here never reaches the later model.

Ensemble learning combines contributions from multiple models into a prediction or decision. For example, two models can estimate transaction risk and a combiner can average their compatible probability estimates. The defining question is: How do multiple model contributions determine the result?

Knowledge distillation trains a student model to approximate a teacher’s behavior, rather than copying its parameters. The teacher is usually more capable; the student is often smaller and specialized for a workload. Lower inference cost is a goal, not a guaranteed result.

Knowledge distillation trains a student model to approximate a teacher’s behavior. The teacher supplies training signals, such as output probabilities or generated answers. Training changes the student’s parameters so it can produce useful answers itself; it does not simply copy the teacher’s parameters. A smaller student is common, but not required.

Per-device batch size is the number of training examples one device processes in a single forward-and-backward pass. That pass calculates a loss and gradients: values the optimizer uses to adjust model parameters. Effective batch size is the total number of examples contributing to one optimizer update.

An epoch is one complete pass through the training dataset. Repeating those passes gives the model more opportunities to learn. The epoch limit sets the maximum number of passes a training run may complete; it does not identify the best model in advance.

Batch size controls how many training examples are processed together. Epochs count passes through the training dataset. Dividing a dataset into smaller groups does not create extra passes: it creates more groups within each pass.

Reranking adds a relevance assessment after retrieval and before evidence selection. Configure two separate limits: the candidate count, which bounds how many retrieved passages are scored, and the output count, which bounds how many highest-ranked passages are selected. For example, you might score 40 candidates and select 5 passages. These are illustrative choices, not recommended defaults. Reranking cannot select evidence that was not supplied as a candidate.

Determinism means that identical inputs, initial state, and execution conditions produce identical results. In an ML experiment, the effective inputs include the training records, labels, preprocessing, and train/validation/test assignments—not just the command you run.

A version-pinned release names the exact model or agent revision to deploy instead of asking for whatever is newest. Treat released content as immutable: preserve it, and create another revision for a change. Development can then continue without silently changing production.

A reusable prompt is an application artifact: changing its instructions can change application behavior. In Amazon Bedrock Prompt Management, you develop that artifact in an editable draft. You can revise the instructions, template variables, and supported model configuration while you work.

An experiment groups related work; a run records one execution, such as training or evaluation. Give each run a unique run ID. In MLflow on Amazon SageMaker AI, training code can log parameters, metrics, and artifacts to a tracking server. That record remains available after the training compute stops.

Version control preserves identifiable revisions of assets such as source code, configuration files, and prompts. It answers questions such as “What changed?” and “Which source revision did we use?” A Git commit identifies a recorded source snapshot; it does not prove that the code ran or that its output passed evaluation.

Data drift means that the distribution of incoming data differs from a reference distribution, such as training data or an accepted production period. It answers: Has the data reaching the model changed? Examples include different income ranges, new product categories, or a larger share of customers from one region. Inputs can follow the correct schema and still show data drift.

Baseline-based monitoring compares recent production observations with a documented reference. The reference dataset represents accepted behavior: it might be validated training data or a production period with known, acceptable performance. Training data is not automatically a suitable reference if production serves a different population.

Population Stability Index (PSI) compares a reference distribution with a current distribution, usually for one input feature or prediction score at a time. Choose a representative reference, such as a validation dataset or a stable production period. PSI measures population change; labeled performance evaluation measures how well predictions match actual outcomes.

Custom quality metrics are numbers that your application or evaluation workflow produces and publishes to Amazon CloudWatch. CloudWatch stores and monitors those numbers; publishing a metric does not ask CloudWatch to judge retrieved documents or model answers.

A drift signal is a reason to investigate, not permission to replace the production model. Check what changed, which predictions are affected, and whether available outcomes show a meaningful performance problem. An automated workflow can collect this evidence without automatically authorizing retraining.

Shadow testing runs a candidate on copies of real requests while the production variant continues supplying every user response. The candidate's outputs become evaluation evidence, not serving decisions.

Blue-green deployment keeps a current environment and a replacement environment separate. Blue serves the current version while green holds the replacement. Promotion redirects production traffic to green, making the replacement responsible for user responses.

A candidate is the new version being evaluated; the stable version is the established version. Both canary deployment and shadow testing run a candidate against production inputs. Their defining difference is whether the candidate's output determines a user's response.

Shadow testing runs a candidate on copies of live requests while the production version continues answering users. The candidate response is retained for evaluation, not served. A request can therefore produce two outputs for comparison, but only the production output determines the user's experience.

LIME, short for Local Interpretable Model-Agnostic Explanations, explains an individual prediction using a separate, simpler model called a local surrogate. It treats the trained model as a black box: it needs to submit inputs and retrieve outputs, not inspect weights or gradients. The surrogate approximates the model near one input; it does not replace the original model.

SHAP assigns credit to input features for a model output, based on their marginal contributions across combinations of available features. For one record, the explanation follows baseline + sum of feature contributions = explained output, subject to approximation error. The baseline is typically the expected model output over selected background data, not zero or the prediction for an all-zero input.

LIME and SHAP explain model outputs after a model has been trained. Both can explain an individual prediction, so “local explanation” does not distinguish them. The important difference is how each method constructs its explanation.

A local explanation describes one prediction, not the model's behavior everywhere. Before reading feature contributions, identify the output being explained: which class, and whether the values use probability, log-odds, or another score. A positive contribution raises that output; it does not necessarily mean a desirable outcome.

The learning rate controls how strongly an optimizer changes model parameters using gradient information. A schedule changes that rate during training. Larger rates can support fast early progress, while smaller rates can help later refinement. Even adaptive optimizers such as Adam can benefit from a scheduled base learning rate.

Exploding gradients are excessively large gradients produced during backpropagation. They can cause disruptive parameter updates, loss spikes, oscillation, or numerical overflow. Gradient clipping limits these gradients before the optimizer uses them; it does not change the loss calculation itself.

Unit testing asks whether a small piece of code follows its contract for known inputs. For example, a test can check that a feature function calculates the expected value or that a response parser handles a fixed model response correctly. Passing establishes those tested behaviors, not the quality of the model's predictions.

Model evaluation measures how well a model or AI application performs on a task. It produces evidence for decisions such as choosing a model, approving a release, or investigating a quality decline. It can assess individual answers, but those assessments contribute to understanding quality under the evaluated conditions.

Cross-validation evaluates a model configuration across several training and validation partitions of development data. A configuration includes choices such as model type, hyperparameters, and preprocessing. In standard k-fold cross-validation, divide the data into k nonoverlapping folds, train on k minus one folds, and validate on the remaining fold. Repeat until each fold has served as validation data once.

A regression suite is a saved collection of evaluation cases and scoring rules that you run again after changes. It checks whether a new model, prompt, or application configuration makes previously acceptable behavior worse. Treat a prompt edit as an evaluation-triggering change, even when the model stays the same.

Human-in-the-loop evaluation uses people to assess model outputs against explicit criteria. It is useful for open-ended tasks where several answers can be acceptable. A repeatable evaluation starts with the application's requirements: for a support assistant, an answer might need to follow the refund policy, address the question, and avoid invented exceptions.

Human-in-the-loop is the broad idea of deliberately involving people in an AI workflow. People may approve a proposed action, correct an output, request more information, or direct what happens next. Human-in-the-loop evaluation is a specific use of that involvement: people assess model outputs against explicit quality criteria.

Inter-rater agreement measures how closely independent human evaluators judge the same material. It describes an evaluation process: the rubric, sampled items, and evaluator population all matter. Agreement is not the same as correctness; reviewers can agree because they share a misconception.

Amazon Augmented AI (Amazon A2I) manages human review of model outputs. A flow definition is the reusable workflow configuration: it connects a worker task template, a work team, and an output location, with activation conditions for supported built-in integrations. A human loop is one execution of that workflow for an item being reviewed.

N-gram overlap compares a generated candidate with one or more reference answers by counting matching token sequences. An n-gram contains n consecutive tokens: a unigram has one, a bigram two, and a trigram three. It measures shared wording, not correctness or meaning directly.

BLEU, short for Bilingual Evaluation Understudy, compares generated text, called the candidate, with one or more reference texts. It measures matching sequences of tokens. An n-gram is a consecutive sequence: a unigram contains one token, a bigram contains two, and so on. Tokenization determines the tokens.

ROUGE compares generated text, the candidate, with an acceptable reference. It is useful for reference-based tasks such as summarization. ROUGE-1 counts individual-token matches. For each distinct token, take the smaller of its candidate and reference counts, then add those matches. This clipping prevents repetition from earning unlimited credit.

Text-overlap metrics compare a generated candidate with an approved reference. For translation, the reference is a translation in the target language. For summarization, it is a reference summary—not the entire source document. Precision asks how much of the candidate matches the reference; recall asks how much of the reference the candidate covers.

Semantic similarity estimates how closely two texts convey similar meanings. To evaluate an answer, compare the candidate with a suitable reference answer. An embedding model converts each text into a numerical vector. Use the same compatible model and configuration for both texts so their vectors share a meaningful coordinate system.

A generated candidate can be compared with a reference in two ways. N-gram overlap asks which token sequences match. Semantic similarity asks how closely the texts express the same meaning. Matching wording and matching meaning are related, but they are not the same objective.

BERTScore compares generated text, called the candidate, with a reference answer. A pretrained contextual encoder represents each token as a vector that reflects its surrounding text. Tokens can be parts of words, not just whole words. The scoring encoder is separate from the model that generated the answer.

BLEU and BERTScore compare generated text, called the candidate, with trusted reference text. They answer different questions: how much wording overlaps, and how similar tokens are in context. Neither directly measures whether an answer is correct.

Fairness evaluation asks whether model outputs treat people equitably under an explicitly chosen criterion. Start with the decision, the affected groups, and the harm you want to detect. A difference between groups is evidence to investigate, not automatic proof of unfair treatment. Overall accuracy cannot answer every fairness question.

LLM-as-a-Judge uses a language model to assess a candidate answer against explicit criteria. Its judgment depends on the evaluation prompt, not on a universal definition of quality. Supply the original task, the candidate answer, and the rubric. Add a reference answer or supporting documents when the criteria require them.

Output validation checks a generated response before another component relies on it. Treat the response as untrusted input, even when it is fluent or follows the requested format. Define an acceptance contract for its destination: required structure, permitted content, sensitive-data restrictions, and evidence requirements. A prompt asking for correct JSON is not enforcement.

Precision@k measures the fraction of the first k retrieved results that are relevant to a query: Precision@k = relevant results in the first k / k. If three of the first five results are relevant, Precision@5 is 3/5 = 0.60. Count only results inside the cutoff; a relevant result at rank six does not contribute to Precision@5.

Recall@k measures how much of a query’s relevant evidence appears in its first k retrieval results. In retrieval-augmented generation, or RAG, missing evidence can leave the model unable to answer completely, even when its response sounds fluent. Measuring coverage starts with representative queries and explicit relevance judgments.

In retrieval-augmented generation (RAG), retrieved passages supply evidence for an answer. Precision@k asks, “How much of the top-k context is relevant?” Recall@k asks, “How much of the relevant evidence did the top-k results recover?” Both count the relevant items retrieved, but they divide that count by different totals.

Model invocation telemetry describes how production requests behave: how long inference takes, how many tokens it consumes, and whether requests are throttled or fail. This lesson uses Amazon Bedrock inference as the source and Amazon CloudWatch as the place to examine that evidence.

CloudWatch metrics describe measured behavior over time: request counts, error rates, latency, or resource use. Use them to establish when a problem started, how widespread it is, and whether performance has recovered. A latency increase shows a symptom, but does not identify which dependency delayed an individual request.

A performance dashboard needs a metrics backend: a service that stores measurements over time and answers queries about them. Choose that backend from how metrics are collected and queried, not simply from where the application runs.

Amazon CloudWatch collects and stores operational telemetry and provides dashboards and alarms. Many AWS services publish metrics to it automatically; applications can also publish custom metrics. Choose CloudWatch dashboards when the view mainly uses CloudWatch data and belongs alongside your AWS monitoring workflow.

An EC2 instance type determines the hardware available to your workload. Its family emphasizes a resource mix; its size determines the amount of those resources. Start with the workload's bottleneck rather than choosing the largest instance. CPU-based preprocessing, an in-memory feature transformation, and deep learning training can need different configurations.

An EC2 instance type is a hardware profile: CPU architecture, virtual CPUs, memory, networking, storage connectivity, and sometimes accelerators. The family identifies a resource emphasis; the size scales resources within that family. In m7i.large, m is the general-purpose family, 7 is the generation, i identifies an Intel processor variant, and large is the size. Equal size labels across families do not mean equal resources.

Amazon SageMaker AI can host a model trained outside AWS without running a SageMaker training job. Hosting needs two things: the model artifacts, such as learned weights and preprocessing files, and a compatible inference container that loads those files and serves predictions.

An Amazon ECS task definition is a versioned blueprint for running containers. Its container definitions specify images, commands, resource settings, and other runtime configuration. A model-serving application can put a prediction API and its model server in the same task definition.

Amazon EKS provides a managed Kubernetes control plane. You describe your model-serving application with Kubernetes objects; EKS does not choose how your models are packaged or exposed.

A container orchestrator schedules containers, keeps the requested workloads running, and coordinates replacements when they fail. Amazon ECS provides AWS-native orchestration through ECS APIs and configuration. Amazon EKS provides managed Kubernetes, using Kubernetes APIs and configuration. Both can run containerized model-serving applications; needing multiple containers does not by itself decide between them.

Real-time inference returns a prediction within the original application request. Its usefulness depends on the application's end-to-end latency budget: the time allowed from sending input to receiving the response. Real-time does not mean instantaneous or guarantee a deadline.

Batch inference applies a trained model to a collected dataset and saves predictions for later use. A job can start on a schedule or when its dataset is ready. Its useful performance measure is throughput: records processed per unit of time, rather than how quickly one prediction returns.

Asynchronous inference separates request acceptance from model execution and result delivery. The application receives an acknowledgment without keeping its original connection open until the prediction finishes. Acceptance does not mean that inference has succeeded.

Batch inference applies a trained model to a collected dataset and saves predictions for later use. Its deadline is about when the results must be ready for a downstream activity. A campaign can use batch scores if the job finishes and publishes them before the campaign starts. Batch jobs can start on a schedule or when a dataset is ready.

An inference response contract defines what the caller receives from submitting a prediction request. With real-time inference, a successful invocation returns the prediction within the original request-response exchange. The caller needs that response to obtain the prediction.

Asynchronous inference accepts independently submitted prediction requests over time and queues them for processing. Each request has its own result to deliver later. Batch inference processes a defined, finite dataset as a job, with outputs intended for later use. Both can run in the background; background execution alone does not distinguish them.

Offline inference processes a collected dataset and saves predictions for later use. With AWS Batch, you submit a finite containerized job: it starts when resources are available, makes predictions, and exits. There is no persistent serving endpoint waiting for prediction requests.

Inference means using a trained model to produce an output. With a managed model API, you select an available model and send requests to a provider-operated service. Amazon Bedrock offers this approach: you do not provision the model’s inference servers or obtain its weights simply by calling the API.

Hosting a foundation model means running its inference software on infrastructure you select. Before choosing that infrastructure, inspect the released artifacts. A checkpoint contains learned parameters, or weights; it is not necessarily a complete application. A usable release also needs compatible architecture definitions, configuration, preprocessing assets such as a tokenizer, and inference software.

Custom Model Import brings a supported model trained or fine-tuned outside Amazon Bedrock into Bedrock for managed inference. It is not a general-purpose loader for arbitrary model code. Check the current Custom Model Import documentation for the target AWS Region, supported architecture, model configuration, and artifact requirements before uploading.

An image packages an application's files, dependencies, and startup settings. A container is a running instance of that image. You build an ML workload image from a base image, which supplies an initial filesystem and often a Python runtime or ML framework.

Amazon ECR stores container images; it does not run them. A private registry belongs to an AWS account in a Region and contains repositories. For this example, create a repository named model-server and set REGISTRY to 123456789012.dkr.ecr.us-east-1.amazonaws.com and REGION to us-east-1. With a local image already built, give it its destination name: docker tag model-server:local $REGISTRY/model-server:v1.

A tool definition tells a supported model which external operation it can request. It supplies a name, a description of when to use the tool, and an input schema describing argument names, types, and required fields. Application code separately implements the operation.

Model Context Protocol (MCP) standardizes communication between an AI application and external capabilities. The host is the agent application. It manages an MCP client for each server connection; the server exposes capabilities backed by external systems. One host can connect to several servers through separate clients.

Tool calling lets a model propose an operation by producing a tool name and structured arguments. The application supplies the available tool definitions and handles the model's request. A request such as lookup_order with an order identifier does not itself perform a lookup.

Amazon Bedrock AgentCore Runtime runs your agent application in managed, isolated execution environments. Your code still defines the agent’s instructions, model calls, and tool use. This lesson follows a container deployment: package the application and its dependencies in an ARM64-compatible container image and store it in Amazon ECR.

A deployed agent needs a task contract, not just a broad instruction to be helpful. For an order-status agent, define the goal as retrieving an authenticated customer's order and reporting its recorded status and estimated delivery. Define success as having a valid tool result that supports that answer. Changing an address, cancelling an order, and issuing a refund are outside this task.

Tool calling lets a model produce a structured request naming an operation and its inputs. For example, it might request get_order_status(order_id="123"). That output describes work to be done; it does not show that the order system was contacted. The application or runtime must execute the permitted operation before there is an external result.

Right-sizing asks whether the resources assigned to work are an appropriate fit. For an ML inference worker, that includes enough processing power, memory for the model, and network capacity to meet response-time requirements. The right choice is not necessarily the smallest resource: insufficient capacity can cause slow responses or failures.

Quantization represents model values with fewer bits. FP16 and BF16 use 16-bit floating-point representations; INT8 uses 8-bit integers, while packed 4-bit formats store weights more compactly. Integer quantization commonly uses scales, and sometimes zero points, to map values between representations. Rounding and clipping introduce error: quantization is not lossless compression.

Model parallelism partitions one model across multiple accelerators. The devices hold different portions of the model and cooperate on its execution. Together, they form one serving replica: a complete allocation capable of serving requests.

Right-sizing means choosing resource capacity that meets workload requirements without paying for unnecessary capacity. The goal is not the smallest resource or the highest utilization. First define the required response time, throughput, error rate, or job completion deadline, along with the failures the workload must tolerate.

An EC2 purchasing option changes how you pay for compute. On-Demand Instances charge for usage without a long-term commitment. They suit experiments, short-lived workloads, and unpredictable demand. Their compute rate is the reference against which purchasing discounts are compared.

Workload predictability is confidence in future compute needs, not simply flat traffic. To find dependable demand, examine utilization history across normal operating cycles, including quiet periods and recurring peaks. Compare resource use with request volume, job completion times, and latency so that low utilization is not mistaken for spare capacity when another resource is limiting performance.

Demand-aligned ML capacity supplies enough inference capacity when it is needed without paying for unnecessary idle capacity. Compare two models: on-demand inference, where usage drives charges, and provisioned capacity, where you allocate capacity that remains available during the provisioned period.

Infrastructure as code describes resources and their configuration in version-controlled files. With AWS CloudFormation, a template describes the desired infrastructure, and a stack is a deployed instance of that template. AWS CDK lets you author infrastructure in supported programming languages and synthesize CloudFormation templates.

A CloudFormation template describes desired infrastructure in YAML or JSON. Its Resources section can declare an AWS::EC2::Instance with properties such as its image ID, instance type, subnet ID, and security groups. A stack is a deployed instance of that template, managed through creation, updates, and deletion.

A resource lifecycle event reports a change that has happened, such as a stack finishing creation. Instead of making that stack call every downstream workflow, publish the fact and let independent consumers react. The stack does not need to know which consumers exist.

Amazon EventBridge answers, “Who should react to this event?” An event describes something that happened, such as a dataset becoming ready. EventBridge matches events against rules and routes matching events to targets. Producers do not need to know the consumers' workflow logic.

An event bus receives events; its rules select which events reach targets. AWS service events arrive on the default event bus. Custom applications publish with PutEvents to a default or custom bus, supplying Source, DetailType, and JSON Detail. In the received event, these appear as source, detail-type, and detail.

An AWS Step Functions state machine defines tasks and the transitions between them. An execution runs that definition with its own input and progress. A Task state can call a supported AWS service API directly through an integration, without a Lambda function that only forwards the request.

AWS Deep Learning Containers (DLCs) are prebuilt container images containing a deep learning framework and supporting libraries. Use one as the base image for your custom image rather than assembling the framework stack yourself. Start with the official DLC available-images documentation to find a supported image URI for your AWS Region and intended runtime.

Amazon ECR image scanning checks software packages in stored container images for known vulnerabilities. For images in private repositories, ECR offers basic scanning and enhanced scanning. The choice changes both what software is checked and how scanning can continue after an image is pushed.

A container image packages application files and dependencies. A running container uses that image plus its own writable layer. Installing a fixed package inside one running container changes that container, not the image stored in your registry. Other replicas remain unchanged, and a replacement container started from the original image still contains the vulnerable package.

Immutable infrastructure means replacing deployed containers when their software changes, rather than installing patches inside running containers. An interactive package update changes only that container, not the image used for its replacement. Instead, maintain the Dockerfile, application source, and dependency lock files in version control.

Network segmentation divides workloads into groups, such as training systems, inference services, and development systems. These groups make different security requirements easier to express and manage. But putting workloads in different network segments does not, by itself, establish which communication is blocked.

For an ML workload, write down the approved communication first: who can submit inference requests, where model files come from, and which destinations the workload must contact. A VPC provides a private network boundary, and subnets place network interfaces within that network.

A SageMaker model's VPC configuration identifies subnets and security groups. SageMaker creates network interfaces in those subnets so the model container can reach resources through your VPC. You control the addressing, routes, and traffic rules for those interfaces; you are not moving the entire SageMaker service into your VPC.

AWS PrivateLink lets an application in your VPC reach SageMaker AI through interface VPC endpoints. Each endpoint creates network interfaces with private IP addresses in your selected subnets. These calls do not require public IP addresses, a NAT gateway, or an internet gateway.

The AWS CLI turns commands into service API requests. For SageMaker model hosting, a script creates a model, an endpoint configuration, and an endpoint. Before running it, configure credentials and a consistent Region. The caller needs SageMaker permissions and permission to pass the model’s execution role; that role lets SageMaker access the model’s assets.

Boto3, the AWS SDK for Python, exposes service operations through clients. For model hosting, sm = boto3.client('sagemaker', region_name=region) creates a management client. Your code calls operations such as create_model, create_endpoint_config, and create_endpoint, supplying their request fields explicitly. Choose this interface when you need direct control over service requests and orchestration.

Boto3 exposes AWS API operations directly. With sm = boto3.client('sagemaker'), your Python program supplies request parameters and calls operations such as sm.create_model(...). Boto3 handles request serialization, signing with your AWS credentials, and sending requests to the selected Region.

Resource utilization measures how much of a resource's available capacity is being used. For ML workloads, examine CPU, GPU, memory, storage, and network separately. Low CPU utilization does not mean an inference endpoint has spare capacity if its GPU is saturated.

Target tracking adjusts replica count to keep a load metric near a chosen target. When load per replica rises, it adds capacity; when load stays low, it can remove capacity. It manages a target rather than requiring you to choose every scaling increment.

A passage-retrieval pipeline has two paths. Ingestion prepares searchable passages before a request arrives. Query-time retrieval embeds a question and searches those stored passages; it does not re-embed the document collection for every request.

Hybrid search retrieves candidates through lexical and vector searches, then combines their rankings. Both searches must refer to the same searchable units. When you split a guide into passages, index each passage's text, embedding, metadata, and stable chunk ID. A document ID alone cannot distinguish several passages from that document.

Consider the query, “How do I fix error E1047 after rotating database credentials?” It contains an exact identifier, which distinguishes one error from others, and natural-language intent, which describes the help the person needs.

Conversation memory is state retained between turns so an application can supply useful history to its next model request. In a typical inference workflow, the model uses the context supplied for that invocation; it does not automatically recall earlier requests. Supplying history does not train the model or change its weights.

A durable checkpoint records enough execution state for another worker to continue an agent workflow after a failure. It must survive the failure being handled: state kept only in a worker's memory or temporary files does not survive losing that worker.

Conversation memory preserves information that helps an assistant understand later turns. It can retain earlier messages, a summary, or facts such as the user's preferred language. It answers, “What context does the model need to understand this request?”

An agent's session state is the working context it needs between requests: conversation summaries, a current task, or completed tool steps. Keeping that context only in one worker's memory ties the session to that worker. Amazon ElastiCache for Valkey or Redis OSS lets different workers use the same in-memory state.

Agent state is information an agent carries between actions: recent conversation context, intermediate tool results, or a checkpoint describing completed work. Choose its store by asking how quickly it must be read and what happens if it disappears. Frequently accessed state is not necessarily disposable state.

A GPU scaling policy needs a signal that shows whether workers can keep up. Demand signals describe arriving or waiting work, such as queued requests per ready worker or the age of the oldest waiting job. Resource signals describe the workers: GPU utilization measures processing activity, while GPU memory utilization measures occupied device memory.

Amazon EC2 Auto Scaling manages fleets of EC2 instances through Auto Scaling groups. It launches and terminates instances to maintain capacity and replaces instances that fail health checks. Choose it directly when your work is about operating an EC2 fleet, including its launch settings, instance lifecycle, and scaling policies.

AgentCore Observability integrates agent telemetry with Amazon CloudWatch. Runtime metrics and application output do not, by themselves, describe every reasoning step or tool call. Use supported OpenTelemetry instrumentation, such as AWS Distro for OpenTelemetry (ADOT), to capture agent, model, and tool spans. ADOT instruments application operations; CloudWatch receives and analyzes the telemetry.

A multi-agent task can stop between delegation, tool execution, and result delivery. A correlated execution trace connects these boundaries: carry a workflow run ID and task ID through every handoff, then record an attempt ID and parent-child span relationships. This lets you distinguish a retry from a different task and locate the last confirmed action.

AWS CodePipeline coordinates a release workflow. Its stages and actions describe how a change moves through activities such as source retrieval, testing, approval, and deployment. It invokes the configured action providers and tracks their results.

A pipeline deployment needs both a revision and a destination. In this EC2 example, the revision is an archive containing application files, scripts, and an AppSpec file. The pipeline stores its artifacts in Amazon S3, and its deploy action names a CodeDeploy application and deployment group.

Blue-green deployment keeps the current inference environment, blue, available while a replacement, green, is prepared. A stable production entry point sends requests to the active environment. The colors describe release roles, not permanent infrastructure identities.

A canary deployment lets a candidate model serve a small share of live requests while the stable model serves the rest. For example, an initial weight split could be 10% candidate and 90% stable. These are routing proportions, not a guarantee of exactly that split in every short interval.

Blue-green deployment separates the current release from its replacement. Blue serves production while engineers prepare and validate green in a separate release environment. A classic blue-green cutover then switches production traffic to green. The colors name roles, not permanent versions.

AWS CodeDeploy does not offer the same rollout choices everywhere. On the EC2/On-Premises compute platform, an in-place deployment updates software on existing hosts. Its deployment configuration controls how many hosts can be updated while maintaining the required healthy capacity. This is host-by-host rollout control, not percentage-based request routing.

AWS CodePipeline coordinates a software release workflow. Its stages and actions describe how a change moves from source through activities such as building, testing, approval, and deployment. It tracks action results and controls when dependent work can proceed.

AWS CodeCommit hosts Git repositories. Developers push commits to it; a CI/CD source action retrieves a revision for later stages. These operations use different identities: a successful developer push does not prove that the pipeline can read the repository.

AWS CodeConnections connects supported Git providers to AWS services that consume repository content or events. A consuming service references a connection by its ARN. The connection is not a copy of the repository, and knowing its ARN does not grant access.

A CodeBuild project is reusable configuration for build runs. Its source identifies the build input, its environment determines where commands run, and its service role supplies AWS permissions. A pipeline can invoke the project successfully while the build still fails to prepare its environment or download its input.

A buildspec is a YAML instruction file that tells AWS CodeBuild which commands to run and which files to collect. By default, CodeBuild looks for buildspec.yml in the source root. Use version: 0.2 so commands can share shell state, such as a directory changed by an earlier command.

A pipeline execution processes a source revision through configured actions. For a CodeCommit source, configure the repository and branch, then use an EventBridge rule to start the pipeline for matching repository changes. The rule's target role needs permission to start that pipeline. Disable source polling when using event-based detection to avoid duplicate executions.

A deterministic unit test checks a small behavior with known inputs and an expected result that stays the same across runs. In an ML application, that behavior might be feature calculation, prompt assembly, or inference-response parsing. Keep this logic separate from network calls and runtime handlers so the test can exercise it without deploying infrastructure.

An automated validation gate makes a successful check a requirement for reaching a later pipeline stage. Put fast code checks early, evaluate the candidate after it is available, and allow production promotion only after every required gate succeeds.

Champion/challenger testing decides whether a candidate model deserves to replace an accepted model. The champion is usually the production model, but it can be an approved baseline before the first deployment. A challenger is a candidate replacement, perhaps trained on newer data or built with revised features, hyperparameters, or an algorithm.

Data provenance connects a model to the data it used and the work that produced that data. For reproducibility, two kinds of evidence matter: artifact identity, which identifies exact inputs and outputs, and execution history, which records how those artifacts were produced. A dataset name such as customer-data-latest is not enough because its contents can change.

A prompt template combines fixed instructions with named variables that an application supplies at runtime. Amazon Bedrock Prompt Management lets teams manage these prompts as reusable resources instead of copying prompt text into every application.

Iterative prompt refinement improves a prompt through measured changes, not just better-sounding answers. Start with a baseline: the current prompt and its observed results. Identify a specific failure, describe the behavior you want instead, and propose one focused change.

An agent release is more than a copy of its code. Its behavior also depends on prompts, tool definitions, model settings, and software libraries. A release manifest is a record that binds these inputs to one release ID, such as R17. It links the source commit, packaged artifact, configuration revision, dependency lockfile, and pipeline execution.

A fine-tuned model release is more than a file named model-latest. An immutable release has a unique identity whose model contents and dependency references do not change. Its release manifest, a record describing the release, links the exact base model, training-data snapshot, training configuration, and resulting adapted artifacts.

A retrieval-augmented generation, or RAG, index holds searchable chunks derived from source documents. After an initial load, incremental processing refreshes only the documents or chunks affected by changes. A scheduled refresh can be incremental; an event-triggered full rebuild is not.

An agent execution trace connects the recorded work for one request: agent decisions to invoke operations, model calls, tool calls, and handoffs between agents. Each timed operation is a span. Its name, timing, status, and recorded attributes help you locate where execution stopped progressing as expected.

A trace records operations from one execution of a distributed workflow. Each span represents an operation, such as validating records, fetching features, or calling a model. A span records timing and can include status, error events, and attributes that help you investigate that operation.

Data drift is a change in the statistical properties of inputs relative to a reference distribution. A monitor compares recent inference inputs with a baseline, often training data or a validated period of production traffic. Valid field names and data types are not enough: employment categories can remain valid while their proportions change.

A higher measured score does not automatically mean a better model. Outcomes vary between samples, so hypothesis testing asks whether an observed difference provides evidence against a specified claim. Define the effect as candidate minus current model on a metric where higher is better. For a one-sided superiority test, the null hypothesis is that the effect is zero or negative; the alternative is that it is positive. A two-sided test instead looks for a difference in either direction. Choose the direction before examining results.

An A/B test compares a current variant A with a candidate B using randomized production traffic. Run them during the same period, keep unrelated settings stable, and define the change being tested. If both the model and prompt change, the result measures that combined change, not either change separately.

A/B testing and hypothesis testing are complementary, not competing methods. A/B testing creates a randomized comparison between alternatives. Hypothesis testing evaluates the statistical evidence in measured results.

Trajectory evaluation checks the observable path an application took, not just its final answer. A refund confirmation can sound correct even when the agent skipped approval. The evidence includes tool selections, inputs, results, handoffs, retries, and termination events—not the model’s private internal reasoning.

AWS X-Ray assembles recorded work into a trace: the connected operations for one request. OpenTelemetry instrumentation creates spans with start and end times, parent relationships, and attributes. Automatic instrumentation covers supported frameworks and clients; custom spans describe application work it cannot see.

An Amazon Managed Service for Prometheus workspace stores time series: timestamped samples identified by a metric name and labels. This lesson starts with metrics already ingested. PromQL selects and calculates values from those series; Grafana displays the results.

An Amazon Managed Grafana dashboard contains panels, and each panel queries a connected data source. The metrics remain in that source; creating a panel does not collect missing metrics. Before editing queries, verify that the connection works and that the workspace's data-source credentials have permission to read the metrics.

An Auto Scaling group manages EC2 instances as one fleet. Its minimum capacity sets the floor, desired capacity is the amount it tries to maintain, and maximum capacity limits how far scaling policies can grow it. With minimum 2, desired 4, and maximum 8, the group initially works toward four instances—not eight.

Failover capacity is the capacity still available to do useful work after a failure. Spreading instances across Availability Zones protects their placement, but does not prove that the survivors can handle the workload. For immediate takeover, the ready capacity outside any one failed zone must cover the demand you intend to support.

Workload cost analysis starts by defining which charges belong to the workload. In AWS Cost Explorer, filter and group historical costs by linked account, service, Region, and activated cost allocation tags. Compare equivalent periods to locate a change, then separate increased demand from increased cost per unit of work. Include an agreed share of shared infrastructure; a workload tag alone does not capture every related charge.

For ML cost reporting, choose a small set of business labels such as CostCenter=CC104, Project=Forecasting, and Environment=Production. Define approved spelling, allowed values, and who maintains them. Tag keys and values are case-sensitive: Project and project create separate keys, while Forecasting and forecasting split values. Use deployment templates and provisioning workflows to apply the standard consistently.

Cost allocation assigns spending to an owner or workload. For an ML workload, that can mean bringing training, inference, storage, and supporting service costs into one financial view. Start with a clear ownership rule: which team pays for each account, and which workload names will be used consistently?

Resource tagging adds metadata: key-value labels that describe a resource. For example, Application=Forecasting identifies a workload and Environment=Production describes its environment. These labels can help people find and manage resources without making any financial decision.

AWS Cost Explorer helps you explore historical AWS cost and usage data. Start with a date range: monthly views reveal longer-term trends, while daily views help locate spending spikes. An ML workload can incur charges for SageMaker, EC2, storage, and other services, so looking only at SageMaker can miss part of its cost.

The AWS Cost and Usage Report (CUR) breaks spending into billing line items rather than giving one total per ML workload. A training workflow can generate separate charges for compute, storage, requests, and data transfer. One resource can also appear in many rows across time and usage types. CUR delivers report files to Amazon S3 for analysis, including SQL analysis with Amazon Athena.

AWS Cost Explorer is the starting point for interactive cost analysis. It lets you explore spending with built-in charts, filters, and groupings without preparing a separate dataset. For example, you can investigate whether an increase in ML spending came from compute or storage.

AWS Budgets compares selected costs or usage against a planned amount over a budget period. A cost budget measures money; a usage budget measures consumption, such as instance hours. Neither is a reservation of capacity or a hard spending cap.

AWS Budgets is the choice for threshold-based spending oversight. Start with a spending target, then configure notifications so an owner knows when spending needs attention. For an ML team with a monthly spending target, this replaces repeatedly opening a report to check progress against that target.

Pay-as-you-go pricing charges for measured resource consumption rather than infrastructure purchased for anticipated demand. For intermittent ML experiments or uncertain inference traffic, this lets you provision resources as needed without a standard long-term usage commitment. It does not mean that every service charges only when a prediction or training step succeeds.

Pay-as-you-go pricing charges for measured consumption without a long-term usage commitment. For example, standard On-Demand compute charges follow billable runtime, subject to the service's billing rules. You do not promise to keep buying that compute for a term.

A commitment discount applies only to usage that meets its eligibility rules. Start with the billed resource, not the application's purpose: model training on Amazon EC2 produces EC2 usage, while training with Amazon SageMaker produces SageMaker usage. Both perform computation, but they do not qualify for the same commitments.

Savings Plans exchange a spending commitment in dollars per hour for discounted rates on matching usage. They change billing, not resource performance, and do not reserve capacity. For ML workloads, first identify where the compute runs: an EC2 training instance and a SageMaker AI training instance do not use the same plan coverage.

A token budget is an allowance for the input a model receives and the output it generates. Tokens are not words: punctuation, code, and different languages affect counts. For a reliable preflight count, use the selected model’s tokenizer or a provider-supported counting interface, rather than a words-to-tokens estimate.

For text models using token-based billing, request count alone does not determine inference cost. A token is a piece of text, not necessarily a word. Input tokens include instructions, conversation history, user messages, and retrieved context sent to the model. Output tokens are the response it generates. A short new message can therefore accompany a large input if the application resends a long conversation.

AWS Pricing Calculator converts service configurations and usage assumptions into planning estimates. Start by adding the inference service and selecting the deployment Region. For Amazon Bedrock, check the model and inference options available in the calculator. Only use pricing dimensions that it supports; verify missing options against the service pricing documentation rather than treating them as free.

Caching reduces foundation model costs by reusing work instead of repeating it. A cache hit finds reusable work; a cache miss requires new work. For inference, distinguish reusing the processing of input from reusing a finished answer.

Vector quantization replaces a vector with a compact identifier for a representative vector in a codebook. Codebooks are learned from representative embedding data. This is lossy compression: the representative usually differs from the original, so estimated distances and neighbor rankings can change.

Amazon CodeGuru and Amazon Inspector address different security questions. For source code security analysis, CodeGuru Reviewer examines supported source code and recommends changes to potentially unsafe coding patterns, such as code that handles untrusted input insecurely. CodeGuru Profiler analyzes runtime performance; it is not the source code security review tool.

Amazon CodeGuru Reviewer analyzes source code and recommends improvements, including fixes for potential security vulnerabilities. In a pull request workflow, it gives developers feedback while a change is still under review, before that change reaches the shared branch.

Static Application Security Testing (SAST) examines application code without executing the application under test. Depending on the analyzer, it reads source code, intermediate representations, or compiled artifacts. It can identify weaknesses such as untrusted input reaching a database query through unsafe string construction.

Amazon Inspector identifies known vulnerabilities in supported operating system packages and programming language packages inside container images stored in Amazon ECR. It examines image contents; the container does not need to be running.

Vulnerability assessment asks, “What security weaknesses were found?” It examines a defined scope, such as application code or a container image, and produces findings. Those findings describe weaknesses and help people understand what needs attention.

A security quality gate makes promotion depend on completed security checks and explicit acceptance rules. For a container release, the pipeline needs both code-level evidence and image vulnerability evidence: checking source code does not establish that the image's operating-system packages are acceptable.

AWS Identity and Access Management (IAM) manages identity and access configuration. You use it to create users and roles and define their access policies. AWS Security Token Service (AWS STS) issues temporary security credentials for authorized requests.

Least privilege means granting only the actions, resources, and request conditions a task needs. For an ML workload, reading training data, writing model artifacts, and invoking an endpoint are separate permissions. An identity-based policy attached to a role defines permissions for that role; attaching a policy to a group gives those permissions to its IAM users.

An IAM role is an identity that a workload can assume without owning permanent access keys. Its trust policy defines which principal may assume it and under what conditions. A principal can be an AWS service, another AWS identity, or a federated identity. Trust does not turn arbitrary application code into an authorized workload: the caller must match the trusted principal and satisfy the applicable conditions.

An encrypted model or training dataset has separate access controls for the artifact and its KMS key. For example, reading an S3 object encrypted with SSE-KMS normally requires both s3:GetObject permission and kms:Decrypt permission. Permission to read the object does not automatically authorize decryption. This lesson uses a customer managed symmetric KMS key, whose access controls you can configure.

Least privilege starts by separating the identity that submits a SageMaker job from the execution role used by that job. The submitter needs the required SageMaker operations, such as sagemaker:CreateTrainingJob, scoped to supported job resources. It also needs iam:PassRole on the approved execution role, restricted with iam:PassedToService set to sagemaker.amazonaws.com.

Attribute-Based Access Control (ABAC) uses attributes in policy conditions to restrict access. For ML artifacts, a shared policy can permit selected operations only when the requester’s project matches the artifact’s project. Tags do not grant access by themselves: an applicable policy must use them.

Cross-account artifact sharing lets an identified consumer read resources owned by another account without making them public. Here, the source account contains the consumer, and the destination account owns the ML artifacts. Network connectivity and membership in the same AWS organization do not grant access.

Least privilege means giving each person or application only the operations and resources its task needs. An IAM role provides an identity with permissions without requiring permanent access keys. When an authorized requester assumes a role, it receives temporary credentials. Use federated access for people and workload roles for applications rather than embedding long-lived keys in notebooks or code.

Role-Based Access Control (RBAC) assigns permissions to responsibilities, then authorizes identities to use those roles. Start with tasks, not a broad service permission such as “SageMaker access.” A training workload, a deployment operator, and an application requesting predictions have different jobs.

To assume a role is to obtain temporary credentials for a role session rather than use the caller's original identity for subsequent work. A user can do this through the AWS CLI, and an application can do it through an AWS SDK. For AssumeRole, the caller signs the request with credentials it already has; these can themselves be temporary credentials.

An ML workload needs an identity to call AWS APIs. An execution role is an IAM role that a service uses to perform work, such as reading training data or saving a model. The workload uses temporary role credentials rather than an IAM user's long-lived access keys.

AWS CloudTrail records supported AWS API activity, including calls from the console, command line, SDKs, and AWS services. An event describes an action, not just a resource's current state. Before depending on a trail for an audit, choose the event categories and filters that cover the actions you need.

AWS CloudTrail provides activity records for auditing AWS actions. Choose these records when you need to establish which identity called an AWS API, which action it requested, and when the request occurred. For example, investigating who requested an update to a SageMaker endpoint calls for API activity evidence, not a performance graph.

AWS Config records the configuration of supported resources so you can inspect changes and assess security requirements. Coverage is not automatic for every resource in a workload. Check that each required resource type is supported in the target Region, then include it in the configuration recorder. For example, a workload's security groups use the resource type AWS::EC2::SecurityGroup.

AWS CloudTrail records evidence of API actions and other account activity. Use it to investigate who requested an action, what they requested, and when. An event can include the caller identity, API operation, source address, request details, and error information.

Configuration drift means that an actual configuration no longer matches an approved configuration. For an ML environment, an approved policy might require SageMaker notebook instances to have direct internet access disabled. This is configuration drift, not a change in model accuracy or input-data distribution.

Log retention defines how long records must remain available and when they may expire. Start by identifying the audit events needed for investigations and compliance, including the relevant accounts and Regions. Enabling collection alone does not guarantee that those records will be kept.

An audit trail helps explain who did what, to which resource, when, and with what result. For an ML workload, that could mean identifying who changed an endpoint, which role read training data, or which application approved a model release. Latency metrics and training-loss logs describe operation, but do not replace this evidence.

Effective permissions are the authorization result for a particular principal, action, resource, and request context. An IAM policy containing Allow is evidence of a grant, not proof that the request will succeed. Ordinary IAM authorization starts with an implicit deny: without an applicable grant, access is denied. An applicable explicit deny overrides an allow.

In AWS IAM authorization, an explicit deny is a matching policy statement whose Effect is Deny. If it applies to a request, an allow elsewhere cannot authorize that request. A more specific allow, an administrator policy, or a later statement does not override it.

Event correlation links records that may describe the same activity, resource, or sequence. For an ML workload, an API audit record, a network flow record, and a training application's message each show a different part of what happened. Aggregation brings those records together; correlation explains which records appear related and why.

A security group filters traffic for associated network interfaces. It is stateful: after it permits a connection, it automatically permits that connection’s return traffic, even if the opposite-direction rules would not independently allow it. This does not permit an unrelated connection started in the reverse direction.

Start with a communication requirement: which ML workload must reach which destination, using which protocol and port? Put workloads with different trust requirements into separate VPCs when they need separate network boundaries. Separate VPCs have no private connection by default; any connection you add needs deliberate routing and filtering.

Threat modeling connects valuable assets to plausible ways someone could harm them. For an ML workload, assets include training-data integrity, confidential records, model artifacts, prediction quality, and endpoint availability. Start with the business consequence: incorrect predictions can matter even when no data is stolen.

A vulnerability assessment looks for security weaknesses in an existing ML workload. Assess both dependencies, such as operating-system packages, training libraries, and inference container base images, and configurations, such as access permissions, storage policies, and network exposure. A package scan does not establish that the workload's permissions are safe.

Threat modeling asks how a workload could be abused and what the design should do about it. It uses the workload’s purpose, valuable assets, data flows, and trust assumptions to describe plausible attacks and their consequences. Its outputs include attack scenarios, security requirements, and decisions about remaining risk.

Data poisoning is deliberate manipulation of information an AI system relies on. An attacker can insert examples, change labels or facts, or selectively remove records. Unlike an accidental quality problem, the aim is to influence behavior, although intent can be difficult to establish from the data alone.

Amazon GuardDuty continuously analyzes account and workload activity using threat intelligence, anomaly detection, and machine learning. For ML systems, suspicious activity could involve credentials used by an ML pipeline or malicious network activity from an EC2 training host. GuardDuty detects security threats; it does not assess model accuracy or guarantee detection of poisoned training data.

Amazon GuardDuty detects potential threats: signs that someone or something may be misusing your AWS environment. Amazon Inspector identifies weaknesses: vulnerable software and network exposure that could allow an attack. The key distinction is evidence of suspicious activity versus a condition that creates risk.

AWS Security Hub CSPM consolidates security findings and security-control results. These capabilities were previously named Security Hub; the enhanced Security Hub adds risk correlation and response capabilities. This lesson follows the CSPM finding lifecycle, where findings become a worklist for investigation and remediation.

Amazon GuardDuty answers, “Is activity in this environment suspicious?” It analyzes activity signals using threat intelligence and detection techniques to produce threat findings. For example, it can identify API activity consistent with compromised credentials used by an ML workload.

To call a foundation model through Amazon Bedrock, an application needs both authentication and permission. An IAM role defines an identity that an authorized caller or workload can assume. Its trust policy controls who can assume it; its permissions policies define allowed actions, subject to other applicable controls.

Credential rotation replaces an authentication secret and retires its predecessor. For model access, persistent credentials can include a long-term Amazon Bedrock API key or an IAM user access key. Use the authentication method supported by the model service and client; these credential types are not interchangeable.

Credential rotation replaces a persistent authentication secret and retires its predecessor. For an application accessing a model, creating a new key is not enough: the application must adopt the replacement, and the old key must stop being accepted. Persistent credentials require explicit lifecycle management.

Amazon Bedrock API keys simplify authentication for supported Bedrock API calls. The client sends the key as a bearer token: possession of the token lets someone use the access it grants. This is convenient for a trusted development tool or a small prototype that accepts an API key.

Amazon Macie discovers sensitive data in Amazon S3. Its data identifiers define what to look for in object content. Discovery identifies potential exposure; it does not mask, redact, or anonymize the objects.

Data Loss Prevention (DLP) protects sensitive information from crossing a boundary where it is not permitted. For an AI application, important boundaries include sending input to a model and releasing a response to a user. A policy identifies protected information, its permitted use or destination, and the action to take when a violation is detected.

AI guardrails evaluate content against application policy rather than asking the model to police itself. An instruction such as “never reveal personal details” is useful guidance, but it is not enforcement. Configure input checks before inference and output checks before release: an acceptable request can still produce an unacceptable answer.