CLOUD CONCEPT

Object Storage

Object storage stores data as individually addressable objects containing a payload, metadata, and a unique key within a bucket or container.

Cloud Concepts Hub →

What is Object Storage

Object storage is designed to hold large collections of data without requiring applications to manage disks or traditional file systems. It is commonly used for images, videos, backups, logs, static website assets, and data lakes, with cost and access characteristics determined by the chosen storage class and configuration.

Object storage organizes data into objects rather than disk blocks or a traditional file-system hierarchy. Each object contains the data itself, associated metadata, and a key that identifies it within a bucket or container. Applications typically upload, retrieve, list, and delete objects through HTTP-based APIs, software development kits, or command-line tools. The storage provider manages the underlying storage infrastructure.

Object keys can resemble file paths, such as images/products/item123.jpg, but separators generally represent key prefixes rather than actual directories. Some services offer hierarchical namespace options, so engineers should check the behavior of the selected service. Metadata can describe properties such as content type, while tags can support classification, access policies, or lifecycle management where supported.

Typical cloud uses include storing user uploads, distributing media, retaining backups, collecting logs, and building data lakes. For example, an application can store product images in object storage and keep their keys in a relational database. A content delivery network can cache those images close to users. An analytics platform can query structured files stored as objects without first loading every file into a database.

For foundational certification questions, distinguish object storage from block and file storage. Object storage is usually the appropriate choice for large collections of independently accessed files, backup archives, and static content. Block storage provides disk-like volumes for workloads such as operating systems and transactional databases. File storage provides shared file-system access, commonly through protocols such as NFS or SMB. An object-storage bucket is not inherently a mounted disk or a POSIX-compatible shared file system.

Objects are generally uploaded or replaced as units rather than modified through arbitrary in-place writes. Some services support specialized append or partial-write operations, but these are not portable assumptions. Applications requiring frequent small updates, file locking, or very low-latency random disk access usually need block storage, file storage, or a database instead. Object storage can hold database backups without being suitable as the database's live disk.

Storage classes allow engineers to trade access speed and retrieval flexibility for lower storage costs. Frequently accessed data belongs in an appropriate online class, while older backups may move to archival classes through lifecycle policies. Some archival classes require a restore operation before reading data. Calculate total cost using stored capacity, request volume, retrieval charges, minimum storage durations, and network transfer—not capacity pricing alone.

Durability, availability, and recoverability are separate concerns. Redundant storage protects against infrastructure failures, but it does not automatically protect against authorized deletion or application mistakes. Versioning, retention controls, replication, and tested recovery procedures address different risks. Access should normally remain private, with narrowly scoped permissions and encryption configured as required. Object storage does not mean public storage, and it does not imply eventual consistency: consistency guarantees depend on the service and operation. Static website assets are a common use, but object storage alone does not execute server-side application code.

Key aspects of Object Storage

Object storage combines key-based access, managed scale, configurable storage classes, and protection controls, with access semantics that differ from disks and shared file systems.

Objects, keys, and metadata

Each object combines a data payload with metadata and is addressed by a key within a bucket or container. Choose predictable key conventions so applications can locate and organize objects without relying on traditional directories.

API-based access

Applications typically access objects through HTTP APIs and SDKs rather than disk operations or native file-system calls. Use temporary, narrowly scoped access mechanisms when clients need to upload or download objects directly.

Scalable storage for independent data items

Object storage accommodates large collections of media, documents, logs, backups, and analytical files without provisioning individual disks. Workload design still needs to account for service quotas, request patterns, object sizes, and throughput requirements.

Storage classes and lifecycle management

Storage classes provide different cost, availability, and retrieval characteristics. Lifecycle policies can transition aging objects or expire them, but retrieval fees and minimum storage durations can affect savings.

Protection and access controls

Permissions, encryption, versioning, and retention controls address different security and recovery requirements. High durability does not replace protection against accidental deletion or the need to test recovery.

Different semantics from disks and file systems

Object storage generally favors uploading and retrieving objects over frequent in-place updates. Do not assume support for file locking, directory renames, or POSIX semantics merely because an interface displays folders.

Cloud services for Object Storage

Amazon S3, Azure Blob Storage, and Google Cloud Storage provide alternative implementations of the object-storage model; an architecture does not need all three. In an AWS design, S3 can hold application assets and analytical files, CloudFront can cache and deliver those assets, and Athena can query the analytical data. These services have distinct roles: persistent storage, content delivery, and query processing. Configure permissions between them deliberately, and evaluate storage, request, retrieval, transfer, and query costs together.

Amazon Simple Storage Service image Amazon S3

Amazon S3 stores objects in buckets and supports uses such as application uploads, backups, static assets, and data lakes. Its storage classes, lifecycle rules, versioning, and access controls let engineers align storage behavior with access patterns and recovery requirements.

Open Amazon S3 →
Amazon CloudFront image Amazon CloudFront

Amazon CloudFront can use an S3 bucket as an origin and cache objects at edge locations. With origin access control, supported S3 origins can remain private while CloudFront serves authorized requests; S3 website endpoints have different access requirements.

Open Amazon CloudFront →
Amazon Athena image Amazon Athena

Amazon Athena can query supported data formats stored in Amazon S3 using SQL. Partitioning, compression, and columnar formats can reduce scanned data and improve the efficiency of an object-storage-based analytics design.

Open Amazon Athena →