# Semi-Structured Data

Semi-structured data uses keys, tags, or other markers to organize information without requiring every record to follow the same fixed tabular schema.

Cloud concept. Updated 2026-10-29. Page: https://blowstack.com/tools/cloud-concepts-hub/semi-structured-data

> Semi-structured data combines identifiable organization with flexible fields, nesting, and optional attributes, as commonly seen in JSON events and XML documents. It matters because cloud ingestion, analytics, and AI pipelines must preserve that flexibility while validating and transforming records into forms that downstream consumers can use reliably.

Practice exams covering Semi-Structured Data: [AIB-C01 (beta) AWS Certified AI Business Strategist](https://blowstack.com/practice-exams/aws-certified-ai-business-strategist), [DEA-C01 AWS Certified Data Engineer - Associate](https://blowstack.com/practice-exams/aws-certified-data-engineer-associate), [MLA-C02 (beta) AWS Certified Machine Learning Engineer - Associate](https://blowstack.com/practice-exams/aws-certified-machine-learning-engineer-associate).

## 6 key aspects of Semi-Structured Data

Semi-structured data combines flexible, identifiable structure with a continuing need for schema management, validation, and consumer-specific transformation.

1. **Flexible record structure**: Records can contain optional fields, nested objects, and arrays rather than sharing an identical set of columns. Define stable required fields for identifiers and routing while allowing controlled variation elsewhere.
2. **Explicit structural markers**: Keys and tags make fields and relationships identifiable to parsers. They do not guarantee consistent meanings, valid values, or correct types, so validation remains necessary.
3. **Schema evolution and compatibility**: Producers can introduce new attributes without redesigning every stored record. Consumers should tolerate compatible additions and explicitly handle incompatible changes such as renaming a field or changing its type.
4. **Read-time interpretation and transformation**: Raw records can be retained and interpreted differently for separate consumers. Analytics and ML pipelines often normalize types, extract nested fields, and create curated tables or model-ready inputs.
5. **Mixed-content representation**: A record can combine structured metadata with free-form text or references to images and audio. Treat each component according to its processing needs rather than assuming the enclosing format makes all content structured.
6. **Operational quality controls**: Monitor malformed payloads, missing attributes, schema drift, and unexpected array expansion. Preserve raw records and provenance so failed transformations can be investigated and replayed.

## Cloud services for Semi-Structured Data

A common pipeline ingests JSON events through Amazon Kinesis Data Streams and retains raw payloads in Amazon S3. AWS Glue can catalog and transform these records, producing curated datasets that Amazon Athena queries for analysis. Amazon DynamoDB can serve application access patterns for records with varying attributes, rather than acting as a required stage in the analytical pipeline. Across these services, engineers must manage event contracts, validate types, preserve provenance, and handle incompatible changes. Keeping raw and curated data separate supports replay while giving analytics and AI consumers predictable inputs.

- **[Amazon S3](https://blowstack.com/tools/cloud-services-hub/amazon-s3)**: Stores raw JSON, XML, and other payloads alongside images, audio, and curated datasets. S3 preserves objects but does not validate their internal record schemas; ingestion and processing components must provide that validation.
- **[AWS Glue](https://blowstack.com/tools/cloud-services-hub/aws-glue)**: Provides data integration capabilities for discovering, cataloging, and transforming semi-structured datasets. Crawlers can infer schemas, but engineers should review the results and control how schema changes affect downstream jobs.
- **[Amazon Athena](https://blowstack.com/tools/cloud-services-hub/amazon-athena)**: Queries supported semi-structured formats in S3 using table definitions and appropriate readers. Engineers can access nested fields and expand arrays, then convert frequently queried datasets into columnar formats where appropriate.
- **[Amazon DynamoDB](https://blowstack.com/tools/cloud-services-hub/amazon-dynamodb)**: Stores items with varying non-key attributes and supports nested maps and lists. Its flexible attributes suit semi-structured application records, but primary keys, index design, access patterns, and item-size limits still require deliberate planning.
- **[Amazon Kinesis Data Streams](https://blowstack.com/tools/cloud-services-hub/amazon-kinesis-data-streams)**: Ingests streams of records whose payloads can contain semi-structured events such as JSON. The service transports data without interpreting its application schema, so producers and consumers must coordinate validation, compatibility, and processing logic.

## Semi-Structured Data exam questions

Real practice exam questions on Semi-Structured Data, with the correct answer and its explanation.

### Question 1 (AIB-C01 (beta))

**Question:**

A lender's AI application estimates loan risk using monthly income and requested loan amount. A partner supplies JSON records with these values under distinct named fields. The partner changes the order in which the fields appear but does not change their names, nesting, types, or meanings. A pilot integration assigns values by their position in the file and begins swapping income with loan amount. The lender cannot require the partner to use a particular field order. Which recommendation best addresses the problem?

**Correct answer:** Parse the records and map values to model inputs using their field names and nesting rather than their position.

**Explanation:** Semi-structured records provide organizational markers, such as named fields and nesting, that identify values without requiring a fixed field order. Mapping by those markers preserves the distinction between income and loan amount.

### Question 2 (DEA-C01)

**Question:**

A shipping company uses an AWS Glue job to process XML manifests from multiple carriers. Each manifest contains a shipment identifier, a dispatch timestamp, and nested package elements. Carriers can add optional elements without coordinating releases. A recent carrier update added a customs element, causing the job's strict, fixed-schema validation to reject otherwise usable manifests.

The company must make new optional elements available to downstream consumers without requiring a schema update for every carrier release. Manifests with missing shipment identifiers or invalid dispatch timestamps must not enter the curated dataset.

Which processing approach meets these requirements?

**Correct answer:** Parse the XML, validate the required identifier and timestamp, and quarantine manifests that fail those checks. Preserve additional elements and their hierarchy in accepted records.

**Explanation:** This accommodates the flexible, hierarchical organization of semi-structured data while enforcing a stable contract for essential fields. New optional elements remain available without weakening required-field validation.

## Practice Semi-Structured Data in a full exam

Questions on Semi-Structured Data in each practice exam, with every answer option and an explanation for each:

- [AIB-C01 (beta)](https://blowstack.com/practice-exams/aws-certified-ai-business-strategist): 6 questions
- [DEA-C01](https://blowstack.com/practice-exams/aws-certified-data-engineer-associate): 3 questions
- [MLA-C02 (beta)](https://blowstack.com/practice-exams/aws-certified-machine-learning-engineer-associate): 1 question

## What is Semi-Structured Data

Semi-structured data has recognizable organization but does not require the uniform rows and columns typically associated with structured relational data. Keys, tags, nested objects, and arrays describe relationships within individual records. Common examples include JSON API responses, XML documents, and application logs that contain named fields. Two records can represent the same kind of event while carrying different optional attributes.

Consider an online store emitting purchase events. Each event might include an event identifier, timestamp, customer identifier, and an array of purchased items. One event might also contain a coupon code, while another includes delivery instructions. This is semi-structured because the records have meaningful fields and hierarchy without requiring every event to contain an identical set of attributes. By contrast, a relational purchase table usually defines consistent columns and types. Free-form customer comments, photographs, and audio recordings are generally treated as unstructured content for analytics and AI, even though their files have technical formats.

In cloud architectures, engineers commonly ingest these records through APIs, message streams, or batch file uploads. Object storage can retain the original payloads for replay and auditing, while document or key-value databases can support application access. Processing jobs parse records, validate important fields, normalize timestamps, resolve inconsistent types, and extract attributes for analytics. Frequently queried data can be transformed into columnar formats such as Parquet to reduce scanning costs and improve analytical performance.

Schema-on-read means applying an interpretation when reading data rather than enforcing all structure before storing it. It is useful for retaining evolving source payloads, but it does not mean there is no schema. Producers and consumers still need agreements about identifiers, field meanings, types, and compatibility. Schema validation can also occur at ingestion. A JSON document may be semi-structured yet governed by a strict schema; the format alone does not determine how flexible a dataset actually is.

The three data characteristics of volume, velocity, and variety describe different engineering pressures. Volume concerns how much data must be stored and processed. Velocity concerns how quickly data arrives and how soon it must be processed. Variety concerns differences in formats and representations, including structured, semi-structured, and unstructured data. A stream of JSON events can have high velocity and volume, while changing event fields add variety. These dimensions affect buffering, partitioning, processing capacity, and validation design.

For AI and machine learning, semi-structured records often connect raw content to labels, metadata, and business context. A JSON training manifest might reference an image in object storage and provide its class label, capture time, and provenance. The manifest is semi-structured; the referenced image remains unstructured content. Similarly, wrapping free-form text or an audio reference in JSON does not make the underlying content ready for a model. Text may need tokenization, images may need decoding and resizing, and audio may need resampling or feature extraction.

Flexibility shifts work downstream. Missing fields, unexpected nesting, numeric values encoded as strings, and incompatible schema changes can break queries or corrupt training features. Engineers should distinguish absent fields from explicit null values, quarantine invalid records, version incompatible event contracts, and monitor parsing failures. Flattening arrays also requires care because it can multiply rows and distort aggregates. For model preparation, preserve labels and provenance, prevent target leakage, and apply consistent transformations during training and inference. Semi-structured data reduces the need for a rigid source layout, not the need for data quality, governance, or deliberate modeling.

## Learn Semi-Structured Data in lessons

8 short lessons on Semi-Structured Data in 3 certification courses.

- AIB-C01 (beta): Semi-Structured Data: flexible records for AI (Premium), https://app.blowstack.com/lessons/semi-structured-data-flexible-records-for-ai-aib-c01
- MLA-C02 (beta): Semi-Structured Data: preserving flexible records during ingestion (Premium), https://app.blowstack.com/lessons/semi-structured-data-preserving-flexible-records-during-ingestion-mla-c02
- DEA-C01: Semi-Structured Data: flexible record structure (Premium), https://app.blowstack.com/lessons/semi-structured-data-flexible-record-structure-dea-c01
- AIB-C01 (beta): Semi-Structured Data vs Structured Data (Premium), https://app.blowstack.com/lessons/semi-structured-data-vs-structured-data-aib-c01
- MLA-C02 (beta): Semi-Structured Data vs Unstructured Data (Premium), https://app.blowstack.com/lessons/semi-structured-data-vs-unstructured-data-mla-c02
- DEA-C01: Semi-Structured Data vs Unstructured Data (Premium), https://app.blowstack.com/lessons/semi-structured-data-vs-unstructured-data-dea-c01
