S3
Simple Storage Service - S3
- Storage service - a backbone service of AWS, provides secure, durable, highly-scalable.
- Easy to use object storage with a simple web service interface
- Object-based storage
- A safe place to store your files
- The data is spread across multiple device and facilities
- File: 0-5TB
- Unlimited storage
- Files saved in buckets
- Globally (like domains)
- Web folder (a flat folder) - containers for objects, root namespace for AWS S3.
- The name must be unique across all AWS accounts (like domains).
- Objects are the entities for files stored in S3
- Size: up to 5TB
- Store data as binary
- Object has:
- keys: a unique identifier (like a file name)
- data: binary data
- metadata: information about the object
- system metadata: auto-generated by S3
- user metadata (optional): generated by the user
- versionId: for versioning
- subresources:
- ACL: access control list
- torrent: torrent supported info
- Data can be accessed by REST API using URL:
- Not support object locking → request that has the latest timestamp is executed
- read-after-write consistency for PUTS new objects
- eventually consistency for PUTS to an existing object or DELETE objects
- S3 is secure by default (public access is turned off)
- Access List Controls (ACLs)
- Logging for buckets
- Hosting static webs
- Bucket policies
- IP range
- AWS account
- Objects with prefixes
- IAM
- Bucket policies are the recommended access control mechanism
- S3 uses a flat structure in a bucket
- Using prefixes and delimiters in order to make a file and folder hierarchy
- Not a file system
Name | Durability | Availability | Usecases |
STANDARD | 9(11)% | 9(4)% | Short term, long term storage for frequently accessed data |
STANDARD_IA | 9(11)% | 9(4)% | Long-lived, less frequently accessed data (infrequently accessed data that is stored for longer than 30 days) |
RRS | 9(4)% | 9(4)% | Can be easily reproduced data(thumbnails,...) |
GLACIER | 9(9)% | 9(2)% | Not available for real-time access. You must first restore archived objects before you can access them. |
- Automated tiering
- Should be used to reduce cost
- SSE-S3 (AWS-Managed Keys): "check-box-style" encryption AWS handles the management and protection of the key. A new key is issued monthly.
- SSE-KMS (AWS KMS Keys): Handle user-provided keys management and protection, provides auditing → access logging
- SSE-C (Customer-Provided Keys): Encrypt/decrypt data by using user-provided keys
- AWS KMS key
- Client-side master key
For maximum simplicity and ease of use, use SSE-S3 or SSE-KMS
- Protects data against accidental or malicious deletion
- Once enabled, can not be removed, it can only be suspended
- Bucket level setting
- Save versions of the file by using IDs
- Additional authentication prevents accidental or malicious deletion
- Can only be enabled by the root account
- Use owner security credential to grant time-limited permission to access objects
- Protect against "content scraping."
- Used for large file
- 3 steps process
- Initiation
- Uploading the parts
- Completion
- Should be used for objects larger than 100Mbytes
- Must be used for objects larger than 5GB
- Asynchronously replicate to a target bucket in another region
- Versioning must be turned on
- Must use IAM policy to give Amazon the permission
- Replicate only new object (created after activation), and delete marker is replicated
- Delete a delete marker is not replication between buckets
- Trigger action by S3 events:
- SNS
- SQS
- Lambda
- Off by default
- Should use with prefixed (/logs, bucketname/logs)
- Information
- Request user account
- Bucket name
- Request time
- Action (GET, PUT, LIST)
- Response status, error code
- Use for hybrid IT environments and applications (data in on-premise file systems, databases back-ups,...)
- bulk blob for data
- If request rate > 100 requests per second, use a hash prefix
- Static web hosting should use CloudFront as a cache layer
- domain: bucket-name.s3-website-regions.amazonaws.com
- Extremely low-cost storage service provides durable, secure, and flexible storage for data archiving and online backup.
- Retrieval time of three to five hours
- 99.9(9)% durability
- Data is stored in archives
- Up to 40TB and an unlimited number of archives
- Assigned a unique archive ID at the time of creation
- Automatically encrypted and immutable
- Vaults are containers for archives
- Each AWS accounts can have up to 1000 vaults
- Enforce compliance controls with a vault lock policy
- Once locked, the policy can no longer be changed
- Up to 5% of data can be stored for free each month
| S3 | Glacier |
Data size| 40TB archive | 5TB object | Identifier|system-generated Ids|friendly key names| Encryption| automatically encrypted|encrypted at rest|
Review questions: 90% → PASSED!

AWS Storage Mindmap
Last modified 4yr ago