Amazon Web Services on Software Engineering Learning Paths

AWS Global Infrastructure

Mon, 01 Jan 0001 00:00:00 +0000

The Physical Layer#

AWS operates the world’s largest cloud infrastructure — a global network of data centers organized into a hierarchy designed for high availability, low latency, and data sovereignty.

flowchart TD
 Global["AWS Global Infrastructure"]
 Global --> Regions["33 Regions<br>(geographic areas)"]
 Global --> Edge["450+ Edge Locations<br>(CDN endpoints)"]
 
 Regions --> AZs["105+ Availability Zones<br>(isolated data centers)"]
 Regions --> Local["Local Zones<br>(city extensions)"]
 Regions --> Wavelength["Wavelength Zones<br>(5G edge)"]

Regions#

A Region is a geographic area containing multiple isolated data centers. Each region is completely independent — a failure in one region doesn’t affect others.

IAM (Identity and Access Management)

Mon, 01 Jan 0001 00:00:00 +0000

What is IAM?#

IAM controls who (authentication) can do what (authorization) on which AWS resources. It’s the security foundation of every AWS account — misconfigure IAM and nothing else matters.

flowchart LR
 Who["WHO?<br>(Principal)"] --> Auth["Authenticated?"]
 Auth --> What["WHAT?<br>(Action)"]
 What --> Which["ON WHICH?<br>(Resource)"]
 Which --> Conditions["UNDER WHAT CONDITIONS?<br>(Conditions)"]
 Conditions --> Decision{"ALLOW or DENY?"}

IAM Entities#

Users#

An IAM user represents a person or application that interacts with AWS:

{
 "UserName": "alice",
 "Arn": "arn:aws:iam::123456789012:user/alice",
 "CreateDate": "2024-01-15T10:00:00Z"
}

Best practice: Minimize IAM users. Use IAM Identity Center (SSO) for humans and IAM roles for applications.

Compute

Mon, 01 Jan 0001 00:00:00 +0000

Compute Options Overview#

AWS offers compute at every abstraction level — from bare metal to fully managed serverless:

flowchart TD
 subgraph Control["More Control"]
 BM["Bare Metal<br>(EC2 .metal)"]
 EC2["EC2 Instances<br>(Virtual Machines)"]
 end
 
 subgraph Managed["Managed"]
 ECS["ECS / EKS<br>(Containers)"]
 Batch["AWS Batch<br>(Job scheduling)"]
 end
 
 subgraph Serverless["Serverless"]
 Lambda["Lambda<br>(Functions)"]
 Fargate["Fargate<br>(Serverless containers)"]
 end
 
 Control -.->|"Less ops overhead →"| Managed
 Managed -.->|"Less ops overhead →"| Serverless

EC2 (Elastic Compute Cloud)#

Virtual machines in the cloud. You choose the OS, instance type, storage, and networking.

Storage

Mon, 01 Jan 0001 00:00:00 +0000

Storage Types#

AWS offers storage at different layers — block, file, and object — each optimized for different access patterns:

flowchart TD
 Storage["AWS Storage"]
 Storage --> Block["Block Storage<br>(EBS, Instance Store)"]
 Storage --> Object["Object Storage<br>(S3, S3 Glacier)"]
 Storage --> File["File Storage<br>(EFS, FSx)"]
 
 Block --> |"Attached to EC2"| Disk["Like a hard drive"]
 Object --> |"HTTP API"| Bucket["Like a file server with infinite capacity"]
 File --> |"NFS/SMB"| Share["Like a network share"]

S3 (Simple Storage Service)#

Object storage with virtually unlimited capacity. The most-used AWS service.

Databases

Mon, 01 Jan 0001 00:00:00 +0000

Database Options#

AWS offers purpose-built databases for different data models and access patterns:

flowchart TD
 DB["AWS Databases"]
 DB --> Relational["Relational (SQL)"]
 DB --> KV["Key-Value"]
 DB --> Document["Document"]
 DB --> Cache["In-Memory Cache"]
 DB --> Graph["Graph"]
 DB --> TimeSeries["Time Series"]
 
 Relational --> RDS["RDS / Aurora"]
 KV --> DDB["DynamoDB"]
 Document --> DocDB["DocumentDB"]
 Cache --> EC["ElastiCache"]
 Graph --> Neptune["Neptune"]
 TimeSeries --> TS["Timestream"]

RDS (Relational Database Service)#

Managed relational databases — AWS handles patching, backups, failover, and replication.

Networking: VPC

Mon, 01 Jan 0001 00:00:00 +0000

What is a VPC?#

A Virtual Private Cloud (VPC) is your isolated network within AWS. It’s a logically isolated section of the AWS cloud where you launch resources in a virtual network that you define.

flowchart TD
 subgraph VPC["VPC: 10.0.0.0/16"]
 subgraph AZ1["AZ 1a"]
 PubSub1["Public Subnet<br>10.0.1.0/24"]
 PrivSub1["Private Subnet<br>10.0.10.0/24"]
 end
 subgraph AZ2["AZ 1b"]
 PubSub2["Public Subnet<br>10.0.2.0/24"]
 PrivSub2["Private Subnet<br>10.0.20.0/24"]
 end
 
 IGW["Internet Gateway"]
 NAT["NAT Gateway"]
 end
 
 Internet["Internet"] --> IGW
 IGW --> PubSub1
 IGW --> PubSub2
 PubSub1 --> NAT
 NAT --> PrivSub1
 NAT --> PrivSub2

VPC Components#

CIDR Blocks#

CIDR (Classless Inter-Domain Routing) defines the IP address range for your VPC:

Load Balancing

Mon, 01 Jan 0001 00:00:00 +0000

Why Load Balancers?#

Load balancers distribute incoming traffic across multiple targets, providing:

High availability — if one target fails, traffic routes to healthy ones
Scalability — add/remove targets without changing client configuration
SSL termination — handle encryption at the LB, not each server
Health checks — automatically detect and remove unhealthy targets

flowchart TD
 Users["Users"] --> LB["Load Balancer"]
 LB --> T1["Target 1 ✅"]
 LB --> T2["Target 2 ✅"]
 LB --> T3["Target 3 ❌ unhealthy"]
 
 LB -.->|"Health check fails"| T3

ELB Types#

Type	Layer	Protocol	Use Case
ALB (Application)	7 (HTTP)	HTTP, HTTPS, gRPC	Web apps, microservices, APIs
NLB (Network)	4 (TCP)	TCP, UDP, TLS	Ultra-low latency, gaming, IoT
GLB (Gateway)	3 (IP)	IP	Third-party appliances (firewalls)

Application Load Balancer (ALB)#

The most common choice for web applications:

DNS: Route 53

Mon, 01 Jan 0001 00:00:00 +0000

What is Route 53?#

Route 53 is AWS’s managed DNS service. It translates domain names (like api.example.com) into IP addresses, and provides health checking and traffic routing.

flowchart LR
 User["User types<br>api.example.com"] --> Resolver["DNS Resolver"]
 Resolver --> R53["Route 53"]
 R53 --> IP["Returns: 52.28.100.50"]
 User -->|"Connects to IP"| ALB["ALB at 52.28.100.50"]

The name “Route 53” comes from TCP/UDP port 53, the standard DNS port.

Core Concepts#

Hosted Zones#

A hosted zone is a container for DNS records for a domain:

Messaging

Mon, 01 Jan 0001 00:00:00 +0000

Why Messaging?#

Direct synchronous communication between services creates tight coupling — if service B is down, service A fails too. Messaging decouples services by introducing a buffer between them.

flowchart LR
 subgraph Synchronous["Synchronous (coupled)"]
 A1["Service A"] -->|"HTTP call"| B1["Service B"]
 B1 -->|"If down → A fails"| A1
 end
 
 subgraph Asynchronous["Asynchronous (decoupled)"]
 A2["Service A"] -->|"Send message"| Q["Queue/Topic"]
 Q -->|"Consume when ready"| B2["Service B"]
 end

SQS (Simple Queue Service)#

Fully managed message queue — producers send messages, consumers poll and process them.

Monitoring

Mon, 01 Jan 0001 00:00:00 +0000

The Three Pillars of Observability#

flowchart TD
 Obs["Observability"]
 Obs --> Metrics["Metrics<br>(CloudWatch Metrics)"]
 Obs --> Logs["Logs<br>(CloudWatch Logs)"]
 Obs --> Traces["Traces<br>(X-Ray / Datadog APM)"]
 
 Metrics --> |"What is happening?"| Dashboard["Dashboards & Alarms"]
 Logs --> |"Why is it happening?"| Search["Log Insights"]
 Traces --> |"Where is it slow?"| Map["Service Map"]

CloudWatch Metrics#

Metrics are time-series data points that measure resource behavior.

Built-in Metrics (Free)#

Service	Metrics
EC2	CPUUtilization, NetworkIn/Out, DiskRead/Write
ALB	RequestCount, TargetResponseTime, HTTPCode_5XX
RDS	CPUUtilization, FreeableMemory, ReadIOPS
Lambda	Invocations, Duration, Errors, Throttles
SQS	ApproximateNumberOfMessages, ApproximateAgeOfOldestMessage

Custom Metrics#

Publish application-specific metrics:

Containers: ECS, ECR, EKS

Mon, 01 Jan 0001 00:00:00 +0000

Why Containers on AWS?#

Containers package your application with all its dependencies into a portable unit. AWS provides managed services to run containers without managing the underlying infrastructure.

flowchart TD
 Code["Application Code"] --> Docker["Docker Image"]
 Docker --> ECR["ECR<br>(Store Image)"]
 ECR --> Orchestrator{"Orchestrator"}
 Orchestrator --> ECS["ECS<br>(AWS-native)"]
 Orchestrator --> EKS["EKS<br>(Kubernetes)"]
 
 ECS --> Fargate["Fargate<br>(Serverless)"]
 ECS --> EC2["EC2<br>(Self-managed)"]
 EKS --> FargateK["Fargate"]
 EKS --> EC2K["EC2 Nodes"]

ECR (Elastic Container Registry)#

Managed Docker registry — store, manage, and deploy container images.

Serverless Patterns

Mon, 01 Jan 0001 00:00:00 +0000

What is Serverless?#

Serverless doesn’t mean “no servers” — it means you don’t manage them. AWS handles provisioning, scaling, patching, and availability. You write code and define triggers.

flowchart LR
 subgraph Traditional["Traditional"]
 Provision["Provision servers"]
 Patch["Patch OS"]
 Scale["Configure scaling"]
 Monitor["Monitor capacity"]
 end
 
 subgraph Serverless["Serverless"]
 Code["Write code"]
 Deploy["Deploy"]
 Done["Done ✅"]
 end

Serverless Services#

Service	Purpose
Lambda	Compute (functions)
API Gateway	HTTP APIs
DynamoDB	Database
S3	Storage
SQS/SNS	Messaging
EventBridge	Event routing
Step Functions	Workflow orchestration
Fargate	Containers (serverless)

Common Serverless Patterns#

API Backend#

flowchart LR
 Client["Client"] --> APIGW["API Gateway"]
 APIGW --> Lambda["Lambda"]
 Lambda --> DDB["DynamoDB"]
 Lambda --> S3["S3"]

API Gateway handles: routing, authentication, rate limiting, request validation, CORS. Lambda handles: business logic. DynamoDB handles: data persistence.

Well-Architected Framework

Mon, 01 Jan 0001 00:00:00 +0000

What is the Well-Architected Framework?#

AWS’s Well-Architected Framework is a set of best practices for designing and operating reliable, secure, efficient, cost-effective, and sustainable systems in the cloud. It provides a consistent approach to evaluating architectures.

flowchart TD
 WAF["Well-Architected Framework"]
 WAF --> OE["Operational Excellence"]
 WAF --> Sec["Security"]
 WAF --> Rel["Reliability"]
 WAF --> PE["Performance Efficiency"]
 WAF --> CO["Cost Optimization"]
 WAF --> Sus["Sustainability"]

Pillar 1: Operational Excellence#

Focus: Run and monitor systems to deliver business value, and continually improve processes.

Security Services

Mon, 01 Jan 0001 00:00:00 +0000

Security Services#

AWS provides a deep security services portfolio — from encryption key management to threat detection. This section covers the services you need to protect production workloads.

AWS KMS (Key Management Service)#

Centrally managed encryption keys for data at rest and in transit.

Concept	Description
CMK (Customer Master Key)	The root key — never leaves KMS hardware
Data key	Generated by KMS, used to encrypt your data (envelope encryption)
Key policy	IAM-like policy attached to the key (who can use/manage it)
Key rotation	Automatic annual rotation (AWS-managed keys) or manual
Key alias	Human-readable name (`alias/my-app-key`)

Envelope Encryption Pattern#

1. App calls KMS → GenerateDataKey → plaintext data key + encrypted data key
2. App encrypts data with plaintext data key
3. App stores encrypted data + encrypted data key together
4. App discards plaintext data key from memory
5. To decrypt: send encrypted data key to KMS → get plaintext data key → decrypt data

Services That Integrate with KMS#

S3 (SSE-KMS), EBS, RDS, DynamoDB, Secrets Manager, SQS, SNS, CloudWatch Logs, ECS, Lambda environment variables.

Cost Management

Mon, 01 Jan 0001 00:00:00 +0000

Cost Management#

AWS bills can grow faster than your infrastructure if unmanaged. Cost optimisation is a continuous practice — not a one-time cleanup. This section covers the tools, patterns, and organisational practices for keeping AWS spend under control.

Understanding Your Bill#

AWS Cost Structure#

Dimension	What You Pay For
Compute	EC2 hours, Lambda invocations, Fargate vCPU/memory-hours
Storage	S3 GB-months, EBS GB-months, snapshots
Data transfer	Outbound to internet, cross-AZ, cross-region
Requests	S3 GET/PUT, API Gateway calls, SQS messages
Provisioned capacity	RDS instances (running 24/7), NAT Gateways, Elasticsearch

Common Cost Surprises#

Surprise	Cause	Fix
NAT Gateway data fees	All private subnet outbound traffic goes through NAT	Use VPC endpoints for AWS services, minimise cross-AZ
Idle RDS instances	Dev/staging databases running 24/7	Stop/start on schedule, or use Aurora Serverless v2
Unattached EBS volumes	Volumes from terminated EC2 instances	Automated cleanup
Old snapshots	Years of daily EBS/RDS snapshots	Lifecycle policies
Cross-AZ data transfer	$0.01/GB between AZs adds up	Keep chatty services in the same AZ
CloudWatch Logs	High log volume without retention	Set retention policies (7/14/30 days)

AWS Cost Tools#

Cost Explorer#

Visualise and analyse spend over time: