Software Engineering Learning Paths

Architecture Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

Architecture Fundamentals#

Software architecture is the set of significant decisions about the organization of a software system — the selection of structural elements and their interfaces, their behavior as specified in collaborations, and the composition of these elements into progressively larger subsystems.

What Is Software Architecture?#

Architecture is not about the code. It’s about the decisions that are costly to change. These decisions shape the system’s ability to meet both functional requirements and quality attributes.

Arrays & Strings

Mon, 01 Jan 0001 00:00:00 +0000

Arrays & Strings#

Arrays and strings are the most common data structures in interviews. Mastering a handful of patterns covers the vast majority of problems.

Pattern Complexity Table#

Pattern	Time	Space	When to Use
Two Pointers	O(n)	O(1)	Sorted array, pair finding, partitioning
Sliding Window	O(n)	O(k)	Subarray/substring with constraint
Prefix Sums	O(n)	O(n)	Range sum queries, subarray sum equals K
Kadane’s Algorithm	O(n)	O(1)	Maximum subarray sum
Sort + Scan	O(n log n)	O(1)	When order doesn’t matter, grouping

Two Pointers#

The two-pointer technique uses two indices that move toward each other (or in the same direction) to solve problems in a single pass.

AWS Global Infrastructure

Mon, 01 Jan 0001 00:00:00 +0000

The Physical Layer#

AWS operates the world’s largest cloud infrastructure — a global network of data centers organized into a hierarchy designed for high availability, low latency, and data sovereignty.

flowchart TD
 Global["AWS Global Infrastructure"]
 Global --> Regions["33 Regions<br>(geographic areas)"]
 Global --> Edge["450+ Edge Locations<br>(CDN endpoints)"]
 
 Regions --> AZs["105+ Availability Zones<br>(isolated data centers)"]
 Regions --> Local["Local Zones<br>(city extensions)"]
 Regions --> Wavelength["Wavelength Zones<br>(5G edge)"]

Regions#

A Region is a geographic area containing multiple isolated data centers. Each region is completely independent — a failure in one region doesn’t affect others.

C Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

C Fundamentals#

C is a small language — about 30 keywords, no classes, no exceptions, no garbage collector. It compiles to machine code, gives you direct hardware access, and trusts you to manage everything yourself. This is both its power and its danger.

Hello World#

#include <stdio.h>

int main(void) {
 printf("Hello, World!\n");
 return 0;
}

gcc -o hello hello.c
./hello

Anatomy#

Part	Purpose
`#include <stdio.h>`	Preprocessor directive — includes standard I/O declarations
`int main(void)`	Entry point, returns int (0 = success, nonzero = error)
`printf()`	Formatted output to stdout
`return 0`	Exit code to the OS

Compilation Pipeline#

source.c → [Preprocessor] → source.i → [Compiler] → source.s → [Assembler] → source.o → [Linker] → executable

Stage	Tool	What It Does
Preprocessing	`cpp`	Expands `#include`, `#define`, conditional compilation
Compilation	`cc1`	Translates C to assembly
Assembly	`as`	Translates assembly to machine code (object file)
Linking	`ld`	Combines object files + libraries into executable

gcc -E hello.c > hello.i # preprocess only
gcc -S hello.c # compile to assembly
gcc -c hello.c # compile to object file
gcc -o hello hello.c # full pipeline

Compiler Flags You Should Always Use#

gcc -Wall -Wextra -Werror -std=c17 -pedantic -O2 -o myapp main.c

Flag	Purpose
`-Wall`	Enable most warnings
`-Wextra`	Additional warnings
`-Werror`	Treat warnings as errors
`-std=c17`	Use C17 standard
`-pedantic`	Strict standard compliance
`-O2`	Optimisation level 2
`-g`	Include debug symbols
`-fsanitize=address`	AddressSanitizer (catches memory bugs)

Types#

Integer Types#

Type	Minimum Size	Typical Size	Range (typical)
`char`	1 byte	1 byte	-128 to 127 or 0 to 255
`short`	2 bytes	2 bytes	-32,768 to 32,767
`int`	2 bytes	4 bytes	-2³¹ to 2³¹-1
`long`	4 bytes	4 or 8 bytes	Platform-dependent
`long long`	8 bytes	8 bytes	-2⁶³ to 2⁶³-1

Use fixed-width types from <stdint.h> for portability:

Cloud Computing Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

Cloud Computing Fundamentals#

Cloud computing is renting someone else’s computers — but the architecture decisions you make on those rented computers are entirely different from on-premises. This section covers the service models, shared responsibilities, and design principles that underpin every cloud-native system.

Cloud Service Models#

Every cloud provider offers services across a spectrum of abstraction. Understanding where each model sits determines what you manage versus what the provider manages.

┌─────────────────────────────────────────────────────────────────┐
│ YOU MANAGE EVERYTHING │
│ On-Premises │
│ ┌──────────┬──────────┬──────────┬──────────┬──────────┐ │
│ │ App │ App │ App │ App │ App │ │
│ │ Runtime │ Runtime │ Runtime │ Runtime │ Runtime │ │
│ │ OS │ OS │ OS │ OS │ OS │ │
│ │ Compute │ Compute │ Compute │ Compute │ Compute │ │
│ │ Storage │ Storage │ Storage │ Storage │ Storage │ │
│ │ Network │ Network │ Network │ Network │ Network │ │
│ └──────────┴──────────┴──────────┴──────────┴──────────┘ │
├─────────────────────────────────────────────────────────────────┤
│ IaaS PaaS FaaS SaaS │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ App ▲ │ │ App ▲ │ │ Code ▲ │ │ ▲ │ │
│ │ Runtime│ │ │ │ │ │ │ │ │ │ │ │
│ │ OS │Y│ │ Runtime│P│ │ Runtime│P│ │ App │P│ │
│ │ ───────│O│ │ OS │R│ │ OS │R│ │ Runtime│R│ │
│ │ Compute│U│ │ ───────│O│ │ ───────│O│ │ OS │O│ │
│ │ Storage│ │ │ Compute│V│ │ Compute│V│ │ Compute│V│ │
│ │ Network│ │ │ Storage│ │ │ Storage│ │ │ Storage│ │ │
│ └────────┘ │ │ Network│ │ │ Network│ │ │ Network│ │ │
│ │ └────────┘ │ └────────┘ │ └────────┘ │ │
│ YOU ▲ │ │ │ │ │
│ PROVIDER ─ │ │ │ │ │
└─────────────────────────────────────────────────────────────────┘

Service Model Comparison#

Model	You Manage	Provider Manages	Examples
IaaS	OS, runtime, app, data	Hardware, networking, virtualisation	EC2, Compute Engine, Azure VMs
PaaS	App code, data	OS, runtime, scaling, patching	App Engine, Elastic Beanstalk, Azure App Service
FaaS	Function code	Everything else, including scaling to zero	Lambda, Cloud Functions, Azure Functions
SaaS	Configuration, data	The entire application	Gmail, Salesforce, Datadog
CaaS	Container images, orchestration config	Infrastructure, container runtime	ECS, GKE, AKS
DBaaS	Schema, queries, data	Engine, patching, backups, scaling	RDS, Cloud SQL, Cosmos DB

Choosing a Service Model#

Is your team managing the OS and patching today?
├── Yes, and we want to keep control → IaaS
├── Yes, but we'd rather not → PaaS or CaaS
└── We just want to ship code
 ├── Stateless, event-driven → FaaS
 ├── Long-running processes → PaaS or CaaS
 └── Standard business app → SaaS (buy, don't build)

The Shared Responsibility Model#

Security and operations responsibilities split between you and the provider. The split point varies by service model.

Communication

Mon, 01 Jan 0001 00:00:00 +0000

Communication#

The single highest-leverage skill for any engineer. Clear communication reduces bugs, prevents rework, accelerates decisions, and builds trust. Poor communication does the opposite — at scale.

Written Communication#

Most engineering communication is written: Slack messages, emails, pull request descriptions, design documents, tickets, and documentation. Written words persist, get forwarded, and are read without your tone of voice for context.

The Pyramid Principle#

Lead with the conclusion, then provide supporting details. Busy readers need the answer first.

Computer Architecture

Mon, 01 Jan 0001 00:00:00 +0000

The Von Neumann Model#

All modern computers follow this architecture: a processor fetches instructions from memory, decodes them, executes them, and stores results.

flowchart LR
 subgraph CPU["CPU"]
 CU["Control Unit"]
 ALU["ALU"]
 REG["Registers"]
 Cache["Cache"]
 end
 
 CPU <-->|"Bus"| RAM["RAM (Memory)"]
 CPU <-->|"Bus"| IO["I/O (Disk, Network, Display)"]

CPU (Central Processing Unit)#

Component	Role	Speed
Registers	Tiny storage inside CPU	1 cycle
L1 Cache	Per-core cache (32-64KB)	~4 cycles
L2 Cache	Per-core (256KB-1MB)	~12 cycles
L3 Cache	Shared across cores (8-64MB)	~40 cycles
RAM	Main memory (8-128GB)	~200 cycles
SSD	Persistent storage	~100,000 cycles
HDD	Spinning disk	~10,000,000 cycles

The Instruction Cycle#

flowchart LR
 F["Fetch"] --> D["Decode"]
 D --> E["Execute"]
 E --> S["Store"]
 S --> F

Fetch — read next instruction from memory (pointed to by Program Counter)
Decode — interpret what the instruction means
Execute — perform the operation (ALU for math, load/store for memory)
Store — write result to register or memory

Modern CPUs execute billions of cycles per second (GHz = billion cycles/sec).

Core Concepts

Mon, 01 Jan 0001 00:00:00 +0000

Core Concepts#

Understanding what Kubernetes solves, how a cluster is structured, and the fundamental building blocks that everything else rests upon.

What Kubernetes Solves#

Before Kubernetes, deploying containers at scale meant manually managing placement, networking, scaling, and failure recovery. Kubernetes automates all of this:

Problem	Without K8s	With K8s
Container placement	Manual SSH, docker run	Scheduler decides optimal node
Scaling	Write custom scripts	`kubectl scale` or autoscaler
Self-healing	Monitor + restart manually	Automatic restart, reschedule
Service discovery	Hardcode IPs, use Consul	Built-in DNS and Services
Rolling updates	Downtime or custom tooling	Declarative rolling update strategy
Configuration	Baked into image or env files	ConfigMaps, Secrets, mounted at runtime
Storage	Manual volume management	Dynamic provisioning via StorageClasses

Kubernetes provides a declarative API — you describe the desired state, and the system continuously reconciles actual state to match it.

Data Engineering Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

What Is Data Engineering?#

Data engineering is the discipline of designing, building, and maintaining the systems that collect, store, transform, and serve data. While data scientists build models and analysts extract insights, data engineers build the infrastructure and pipelines that make those activities possible.

The Data Engineer’s Responsibilities#

Responsibility	Description
Ingestion	Collecting data from source systems (APIs, databases, files, streams)
Storage	Choosing and managing data stores (lakes, warehouses, object storage)
Transformation	Cleaning, enriching, and reshaping data for consumption
Orchestration	Scheduling and monitoring pipeline execution
Data quality	Validating, testing, and monitoring data correctness
Infrastructure	Provisioning compute, networking, and storage for data workloads
Security	Managing access control, encryption, and compliance

Data Engineering vs Adjacent Roles#

Role	Focus	Typical Tools
Data Engineer	Pipelines, infrastructure, data platforms	Spark, Airflow, Kafka, SQL, dbt
Data Scientist	Models, experiments, statistical analysis	Python, R, Jupyter, scikit-learn
Data Analyst	Reporting, dashboards, ad-hoc queries	SQL, Tableau, Looker, Excel
Analytics Engineer	Transform + model data for analysts	dbt, SQL, version control
ML Engineer	Productionise ML models	MLflow, Kubeflow, TensorFlow Serving
Platform Engineer	Infrastructure for data teams	Terraform, Kubernetes, CI/CD

The Data Lifecycle#

Data moves through a series of stages from creation to consumption. Understanding this lifecycle is essential for building reliable systems.

DevOps Principles

Mon, 01 Jan 0001 00:00:00 +0000

DevOps Principles#

DevOps is not a tool, a job title, or a team — it is a set of practices and cultural philosophies that unify software development (Dev) and IT operations (Ops). The goal is to shorten the systems development life cycle while delivering features, fixes, and updates frequently and reliably.

The Three Ways of DevOps#

The foundational principles come from The Phoenix Project and The DevOps Handbook:

Way	Principle	Practice
First Way	Systems thinking (flow)	Optimise left-to-right flow from Dev to Ops to customer. Reduce batch sizes, limit WIP, eliminate waste.
Second Way	Amplify feedback loops	Fast feedback from right to left. Telemetry, monitoring, peer review, testing in production.
Third Way	Continual experimentation and learning	Foster a culture of experimentation, risk-taking, and learning from failure. Blameless postmortems.

DevOps Culture — CALMS#

The CALMS framework describes the cultural pillars of DevOps:

Distributed Systems Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

Distributed Systems Fundamentals#

A distributed system is a collection of independent computers that appears to its users as a single coherent system. This deceptively simple definition hides enormous complexity — complexity that has broken production systems at every scale, from startups to hyperscalers.

This section covers why distributed systems are fundamentally harder than single-node systems, the theoretical limits that constrain what is possible, and the models we use to reason about them.

Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

Why System Design?#

Writing code that works on your laptop is one skill. Designing systems that serve millions of users reliably is another. System design is about making tradeoffs — every choice has consequences for scalability, reliability, cost, and complexity.

Core Metrics#

Latency#

Time from request to response. Measured in milliseconds (ms).

Operation	Typical Latency
L1 cache reference	0.5 ns
RAM access	100 ns
SSD random read	150 μs
HDD seek	10 ms
Same-datacenter roundtrip	0.5 ms
US coast-to-coast roundtrip	40 ms
Europe to US roundtrip	80 ms

P50 vs P95 vs P99: Don’t measure averages — measure percentiles. P99 latency (worst 1% of requests) is what users feel during peak load.

Getting Started

Mon, 01 Jan 0001 00:00:00 +0000

Getting Started#

Rust’s toolchain is self-contained and well-designed. rustup manages versions, cargo handles building, testing, and dependency management. Within minutes of installation you can compile, run, test, and lint Rust code.

Installation#

# Install rustup (manages Rust versions)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Verify
rustc --version # compiler
cargo --version # build tool + package manager
rustup --version # toolchain manager

Toolchain Management#

rustup update # update to latest stable
rustup default stable # use stable channel
rustup default nightly # use nightly (for bleeding-edge features)
rustup component add clippy # add linter
rustup component add rustfmt # add formatter
rustup target add wasm32-unknown-unknown # add WebAssembly target

Hello World#

fn main() {
 println!("Hello, World!");
}

rustc main.rs # compile directly
./main # run

# But in practice, always use cargo:
cargo new hello # creates a project
cd hello
cargo run # compile + run

What’s `println!`?#

The ! indicates a macro, not a function. Macros are expanded at compile time — println! accepts variable argument counts and format strings that are checked at compile time.

Git

Mon, 01 Jan 0001 00:00:00 +0000

What Is Git?#

Git is a distributed version control system that tracks changes in source code. Created by Linus Torvalds in 2005 for Linux kernel development, it’s now the universal standard for software collaboration.

Why Git?#

Feature	Benefit
Distributed	Every clone is a full repository — work offline, no single point of failure
Branching	Create, merge, and delete branches instantly
Speed	Local operations (log, diff, commit) are near-instant
Integrity	Every object is checksummed with SHA-1/SHA-256
Staging area	Fine-grained control over what goes into each commit

Core Concepts#

flowchart LR
 WD["Working Directory"] -->|"git add"| SA["Staging Area<br/>(Index)"]
 SA -->|"git commit"| LR["Local Repository<br/>(.git)"]
 LR -->|"git push"| RR["Remote Repository"]
 RR -->|"git pull / fetch"| LR
 LR -->|"git checkout"| WD

The Three States#

Every file in a Git repository is in one of three states:

Git Internals

Mon, 01 Jan 0001 00:00:00 +0000

Git Internals#

Understanding how Git stores and manages data transforms you from someone who memorises commands into someone who truly understands the tool. Git is fundamentally a content-addressable filesystem with a version control interface built on top.

The .git Directory Anatomy#

When you run git init, Git creates the .git directory — the entire repository lives here:

.git/
├── HEAD # Points to the current branch ref
├── config # Repository-specific configuration
├── description # Used by GitWeb (rarely relevant)
├── hooks/ # Client-side and server-side hook scripts
├── info/ # Global exclude patterns
├── objects/ # All content (blobs, trees, commits, tags)
│ ├── info/
│ └── pack/ # Packed objects for efficiency
├── refs/ # Pointers to commit objects
│ ├── heads/ # Branch tips
│ ├── tags/ # Tag references
│ └── remotes/ # Remote-tracking branches
├── index # Staging area (binary file)
└── logs/ # Reflog history

Component	Purpose	Human-readable?
`HEAD`	Current branch pointer	Yes — contains `ref: refs/heads/main`
`objects/`	Content store (all versions of all files)	No — compressed binary
`refs/`	Named pointers to commits	Yes — contains SHA-1 hashes
`index`	Staging area snapshot	No — binary format
`logs/`	History of ref changes	Yes — one entry per line

The Four Object Types#

Git stores everything as one of four object types, each identified by a 40-character SHA-1 hash:

Go Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

Go Fundamentals#

Go is deliberately simple. The language specification fits in a few dozen pages. There are no classes, no inheritance, no exceptions, no operator overloading, and no implicit type conversions. This is not a limitation — it is the design.

Philosophy#

Go was born from frustration with the compile times, complexity, and dependency management of C++ and Java at Google scale. Its design principles:

Principle	Manifestation
Simplicity	25 keywords, one loop construct, no ternary operator
Readability	`gofmt` enforces a single style — no formatting debates
Fast compilation	Compiles a million lines of code in seconds
Explicit over implicit	No hidden constructors, no magic methods, no implicit conversions
Composition over inheritance	Struct embedding instead of class hierarchies
Concurrency as a first-class concept	Goroutines and channels built into the language
Batteries included	Standard library covers HTTP, JSON, crypto, testing, profiling

The Go Proverbs#

From Rob Pike’s 2015 talk, distilled wisdom:

HTML Semantics and Accessibility

Mon, 01 Jan 0001 00:00:00 +0000

Why Semantics Matter#

HTML is not just about visual structure — it’s a communication layer between your content and machines (browsers, screen readers, search engines, AI). Semantic HTML provides meaning that <div> and <span> cannot.

flowchart LR
 HTML["Semantic HTML"] --> B["Browser<br>(rendering, navigation)"]
 HTML --> SR["Screen Reader<br>(accessibility tree)"]
 HTML --> SE["Search Engine<br>(content understanding)"]
 HTML --> Dev["Developer<br>(readable code)"]

Document Structure#

The Semantic Skeleton#

<!DOCTYPE html>
<html lang="en">
<head>
 <meta charset="UTF-8">
 <meta name="viewport" content="width=device-width, initial-scale=1.0">
 <title>Page Title — Site Name</title>
 <meta name="description" content="Page description for SEO">
</head>
<body>
 <header>
 <nav aria-label="Main navigation">
 <ul>
 <li><a href="/" aria-current="page">Home</a></li>
 <li><a href="/products">Products</a></li>
 <li><a href="/about">About</a></li>
 </ul>
 </nav>
 </header>
 
 <main>
 <article>
 <h1>Article Title</h1>
 <p>Content...</p>
 
 <section aria-labelledby="features-heading">
 <h2 id="features-heading">Features</h2>
 <p>Section content...</p>
 </section>
 </article>
 
 <aside aria-label="Related articles">
 <h2>Related</h2>
 <ul>...</ul>
 </aside>
 </main>
 
 <footer>
 <nav aria-label="Footer navigation">...</nav>
 <p>&copy; 2024 Company Name</p>
 </footer>
</body>
</html>

Heading Hierarchy#

Headings create an outline. Screen readers use them for navigation.

Infrastructure as Code Concepts

Mon, 01 Jan 0001 00:00:00 +0000

The Problem IaC Solves#

Before IaC, infrastructure was managed through manual processes — clicking through cloud consoles, running ad-hoc scripts, and documenting steps in wikis that quickly became outdated. This created several problems:

flowchart TD
 Manual["Manual Infrastructure"] --> Drift["Configuration Drift<br>Servers differ over time"]
 Manual --> Undoc["Undocumented Changes<br>Who changed what? When?"]
 Manual --> Slow["Slow Provisioning<br>Hours/days to set up environments"]
 Manual --> Error["Human Error<br>Typos, missed steps, wrong order"]
 Manual --> NoRepro["Not Reproducible<br>Can't recreate environments reliably"]

IaC eliminates these problems by treating infrastructure the same way we treat application code — written in files, stored in version control, reviewed by peers, and deployed through automation.

Introduction to Machine Learning

Mon, 01 Jan 0001 00:00:00 +0000

Introduction to Machine Learning#

Machine learning is the study of algorithms that improve their performance on a task through experience (data), without being explicitly programmed for that task. This section establishes the taxonomy, core tradeoffs, and workflow that every subsequent section builds upon.

What Is Machine Learning?#

Traditional programming: you write rules, the program applies them to data and produces output.

Machine learning: you provide data and the desired output, and the algorithm learns the rules.

Introduction to Observability

Mon, 01 Jan 0001 00:00:00 +0000

Introduction to Observability#

Observability is the measure of how well you can understand a system’s internal state from its external outputs. The term originates from control theory, where a system is “observable” if its internal state can be reconstructed from its outputs alone. In software engineering, observability means your production systems emit enough telemetry — metrics, logs, and traces — that you can diagnose any failure without deploying new code.

Java: JVM Architecture

Mon, 01 Jan 0001 00:00:00 +0000

From Source to Execution#

flowchart LR
 A["Hello.java"] -->|"javac"| B["Hello.class<br>(bytecode)"]
 B -->|"java"| C["Class Loader"]
 C --> D["Bytecode Verifier"]
 D --> E["Interpreter"]
 D --> F["JIT Compiler"]
 E --> G["Native Execution"]
 F --> G

Compilation vs Interpretation#

Java uses a two-stage execution model:

Ahead-of-time (AOT): javac compiles .java → .class (bytecode)
Just-in-time (JIT): The JVM compiles hot bytecode → native machine code at runtime

# Compile
javac Hello.java # Produces Hello.class

# Run
java Hello # JVM loads, verifies, interprets/JIT-compiles

# Inspect bytecode
javap -c Hello.class # Disassemble bytecode

// Hello.java
public class Hello {
 public static void main(String[] args) {
 System.out.println("Hello, JVM!");
 }
}

Disassembled bytecode:

JavaScript Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

The Language at a Glance#

JavaScript is a dynamically typed, prototype-based, multi-paradigm language. It was created in 10 days in 1995 and has evolved into the most widely deployed programming language in history — running in every browser, on servers (Node.js), mobile apps, desktop apps, and even embedded systems.

flowchart LR
 subgraph Engine["JavaScript Engine (V8, SpiderMonkey, JSC)"]
 P["Parser"] --> AST["Abstract Syntax Tree"]
 AST --> I["Interpreter (Ignition)"]
 I --> BC["Bytecode"]
 BC --> OC["Optimizing Compiler (TurboFan)"]
 OC --> MC["Machine Code"]
 end
 
 Source[".js source"] --> P
 MC --> Exec["Execution"]

Types and the Type System#

JavaScript has 7 primitive types and 1 structural type (object). Primitives are immutable and compared by value. Objects are mutable and compared by reference.

Kotlin Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

Kotlin Fundamentals#

Kotlin is a statically-typed language that runs on the JVM, designed to be concise, safe, and fully interoperable with Java. It eliminates entire categories of bugs through its type system, particularly its handling of null references through nullable types.

Variables: val and var#

Kotlin distinguishes between immutable and mutable bindings at the declaration site:

val maximumAttempts = 3 // Immutable — cannot be reassigned
var currentAttempt = 0 // Mutable — can be reassigned

currentAttempt += 1 // OK
// maximumAttempts = 5 // Compile error: Val cannot be reassigned

Prefer val over var whenever possible. The compiler enforces immutability, making code easier to reason about and safer in concurrent contexts.

Neural Networks and Deep Learning

Mon, 01 Jan 0001 00:00:00 +0000

This guide covers the foundational concepts of neural networks — from the single perceptron to deep architectures. By the end, you’ll understand how networks learn, why they work, and how to prevent them from failing.

Prerequisites#

Linear algebra basics (vectors, matrices, dot products)
Calculus (derivatives, chain rule)
Basic probability and statistics
Python familiarity

The Perceptron#

The perceptron is the simplest neural unit — a linear classifier inspired by biological neurons.

Numbers & Arithmetic

Mon, 01 Jan 0001 00:00:00 +0000

Numbers & Arithmetic#

Every computation a computer performs reduces to arithmetic on numbers. Understanding number systems — what they represent, how they are stored, and where they break — is the foundation of everything that follows.

Number Systems#

Mathematics organises numbers into nested sets, each extending the previous one to solve problems the smaller set cannot.

Symbol	Name	Contains	Example
ℕ	Natural numbers	0, 1, 2, 3, …	Counting items in a list
ℤ	Integers	…, -2, -1, 0, 1, 2, …	Signed offsets, deltas
ℚ	Rational numbers	Any number expressible as p/q where p, q ∈ ℤ and q ≠ 0	1/3, 7/4, -22/7
ℝ	Real numbers	All rationals plus irrationals (π, √2, e)	Continuous measurements
ℂ	Complex numbers	a + bi where a, b ∈ ℝ and i² = -1	Signal processing, quantum computing

Why This Matters#

Array indices are natural numbers (ℕ).
Database offsets and deltas are integers (ℤ).
Financial calculations use rationals (ℚ) — never floating-point.
Physics simulations operate on reals (ℝ), approximated by floats.
Fourier transforms and quantum gates use complex numbers (ℂ).

Key Properties#

Every number system inherits the properties of the sets below it and adds new ones:

OSI and TCP/IP Models

Mon, 01 Jan 0001 00:00:00 +0000

Why Layered Models?#

Networking is complex — it involves physical signals, addressing, routing, reliability, and application logic. Layered models break this complexity into manageable pieces where each layer has a single responsibility and communicates only with adjacent layers.

flowchart TD
 App["Application: What to send"] --> Transport["Transport: How to deliver reliably"]
 Transport --> Network["Network: Where to send (routing)"]
 Network --> DataLink["Data Link: Local delivery (MAC)"]
 DataLink --> Physical["Physical: Bits on the wire"]

The OSI Model (7 Layers)#

A theoretical reference model — useful for understanding and troubleshooting:

Processes & Threads

Mon, 01 Jan 0001 00:00:00 +0000

Processes & Threads#

A process is the OS’s abstraction for a running program. It encapsulates the program’s code, data, and execution state into an isolated unit that the OS can manage, schedule, and protect.

The Process Model#

A process is more than just executing code. It includes:

Component	Description
Text segment	The compiled program instructions
Data segment	Global and static variables
Heap	Dynamically allocated memory (grows upward)
Stack	Function call frames, local variables (grows downward)
Registers	CPU register state (PC, SP, general-purpose)
Open files	File descriptors the process holds
Credentials	User ID, group ID, permissions

Process Memory Layout#

High addresses
┌──────────────────┐
│ Stack │ ← grows downward
│ ↓ │
├──────────────────┤
│ │
│ (free space) │
│ │
├──────────────────┤
│ ↑ │
│ Heap │ ← grows upward
├──────────────────┤
│ BSS (uninit) │
├──────────────────┤
│ Data (init) │
├──────────────────┤
│ Text │ ← program code
└──────────────────┘
Low addresses

Process States#

Every process transitions through a well-defined set of states:

PWA Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

What Makes a Progressive Web App#

A Progressive Web App is a web application that uses modern browser APIs and a strategic design approach to deliver a native-app-like experience. The term was coined by Alex Russell and Frances Berriman in 2015, but the underlying principles are rooted in the web’s core strengths: linkability, discoverability, and universal access.

A PWA is not a framework or a specific technology. It is a set of characteristics:

Python: Syntax, Types, and the Interpreter

Mon, 01 Jan 0001 00:00:00 +0000

How Python Executes Code#

The CPython Pipeline#

flowchart LR
 A["source.py"] --> B["Lexer/Tokenizer"]
 B --> C["Parser (AST)"]
 C --> D["Compiler"]
 D --> E["Bytecode (.pyc)"]
 E --> F["CPython VM"]
 F --> G["Execution"]

Lexer — breaks source into tokens (def, foo, (, ), :)
Parser — builds an Abstract Syntax Tree (AST)
Compiler — generates bytecode (stored in __pycache__/)
VM — executes bytecode instruction by instruction

Inspecting Bytecode#

import dis

def add(a, b):
 return a + b

dis.dis(add)
# Output:
# 2 0 LOAD_FAST 0 (a)
# 2 LOAD_FAST 1 (b)
# 4 BINARY_ADD
# 6 RETURN_VALUE

Python Implementations#

Implementation	Language	Use Case
CPython	C	Default, reference implementation
PyPy	Python/RPython	JIT-compiled, 4-10x faster for long-running code
Jython	Java	JVM integration
MicroPython	C	Microcontrollers, embedded systems
GraalPy	Java	GraalVM polyglot

The Type System#

Everything is an Object#

# Even integers are objects
x = 42
type(x) # <class 'int'>
x.bit_length() # 6 (42 in binary = 101010)
id(x) # memory address

# Functions are objects
def greet():
 pass

type(greet) # <class 'function'>
greet.__name__ # 'greet'
greet.__code__.co_varnames # ()

Object Identity, Type, and Value#

a = [1, 2, 3]
b = a # b references the SAME object
c = [1, 2, 3] # c is a DIFFERENT object with same value

a is b # True — same identity (same object in memory)
a is c # False — different identity
a == c # True — same value

id(a) == id(b) # True
id(a) == id(c) # False

Mutability#

Mutable	Immutable
`list`	`tuple`
`dict`	`frozenset`
`set`	`str`
`bytearray`	`bytes`
Custom objects (default)	`int`, `float`, `bool`

Why it matters:

Relational Database Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

What Is a Database?#

A database is an organized collection of structured data, managed by a Database Management System (DBMS) that handles storage, retrieval, security, and concurrent access. Without a DBMS, applications would need to manage files, locking, indexing, and crash recovery themselves.

Types of Databases#

Type	Model	Examples	Best For
Relational (RDBMS)	Tables with rows/columns	PostgreSQL, MySQL, Oracle, SQL Server	Structured data, complex queries, transactions
Document	JSON/BSON documents	MongoDB, CouchDB, Firestore	Flexible schemas, nested data
Key-Value	Simple key → value pairs	Redis, DynamoDB, Memcached	Caching, sessions, simple lookups
Column-Family	Wide columns, sparse rows	Cassandra, HBase, ScyllaDB	Time-series, high write throughput
Graph	Nodes and edges	Neo4j, Amazon Neptune	Relationships, social networks, recommendations
Time-Series	Timestamped data points	InfluxDB, TimescaleDB, Prometheus	Metrics, IoT, monitoring

The Relational Model#

Proposed by Edgar F. Codd in 1970, the relational model organizes data into relations (tables) with mathematical foundations in set theory and predicate logic.

REST Principles

Mon, 01 Jan 0001 00:00:00 +0000

REST Principles#

REST (Representational State Transfer) is an architectural style defined by Roy Fielding in his 2000 doctoral dissertation. It describes a set of constraints that, when applied to web services, yield systems that are scalable, loosely coupled, and simple to understand.

The Six REST Constraints#

REST is not a protocol or a standard — it’s a set of architectural constraints. A service that satisfies all constraints is called RESTful.

Shell Basics

Mon, 01 Jan 0001 00:00:00 +0000

What is a Shell?#

A shell is a command-line interpreter — a program that reads text commands, interprets them, and executes them by calling the operating system’s kernel. It serves two roles: an interactive environment for running commands one at a time, and a programming language for automating sequences of commands.

flowchart LR
 User["User Input"] --> Shell["Shell (bash, zsh)"]
 Shell --> Kernel["OS Kernel"]
 Kernel --> HW["Hardware"]
 Kernel --> FS["File System"]
 Kernel --> Net["Network"]
 Shell --> Output["Output (stdout/stderr)"]

The shell is NOT the terminal. The terminal is the window application (iTerm2, GNOME Terminal). The shell is the program running inside it.

Storage Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

Why Storage Matters#

Every database operation — reads, writes, joins, index lookups — ultimately translates into I/O against a storage device. Understanding how data is physically organised determines whether a query takes microseconds or minutes.

This section covers the lowest layer of a database: how bytes are arranged on disk, how the database divides storage into fixed-size pages, and how individual rows are packed into those pages.

Disk vs SSD#

Mechanical Hard Drives (HDD)#

HDDs store data on spinning magnetic platters read by a moving head. The cost of an I/O operation is dominated by seek time (moving the head to the right track) and rotational latency (waiting for the platter to spin to the right sector).

Swift Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

Swift Fundamentals#

Swift is a compiled, statically-typed language designed for safety and performance. It eliminates entire categories of bugs through its type system, particularly its handling of nil values through optionals.

Variables and Constants#

Swift distinguishes between mutable and immutable bindings at the declaration site:

let maximumAttempts = 3 // Constant — cannot be reassigned
var currentAttempt = 0 // Variable — can be reassigned

currentAttempt += 1 // OK
// maximumAttempts = 5 // Compile error: cannot assign to 'let' constant

Prefer let over var whenever possible. The compiler enforces immutability, making code easier to reason about and safer in concurrent contexts.

Syntax, Types, and the Runtime

Mon, 01 Jan 0001 00:00:00 +0000

The PHP Runtime#

PHP originally stood for “Personal Home Page” (now “PHP: Hypertext Preprocessor” — a recursive acronym). It’s an interpreted language compiled to opcodes at request time, executed by the Zend Engine virtual machine.

Execution Model#

flowchart TD
 SRC[".php source file"] --> LEXER["Lexer (Tokenizer)"]
 LEXER --> PARSER["Parser (AST)"]
 PARSER --> COMPILER["Compiler"]
 COMPILER --> OPCODES["Opcodes"]
 OPCODES --> VM["Zend VM"]
 VM --> OUTPUT["Output (HTML, JSON, etc.)"]
 
 OPCODES -.->|"Cached by OPcache"| CACHE["Shared Memory"]
 CACHE -.->|"Subsequent requests"| VM

Key insight: Without OPcache, PHP re-compiles every file on every request. With OPcache (enabled by default since PHP 5.5), compiled opcodes persist in shared memory across requests — massive performance win.

Testing Philosophy

Mon, 01 Jan 0001 00:00:00 +0000

Testing Philosophy#

Before writing a single test, you need to understand why testing matters, what to test, and how much testing is enough. This section establishes the mental models that guide every testing decision you’ll make.

Why Test?#

Testing exists to answer one question: does this software do what it should?

But the deeper motivations are:

Motivation	What It Means
Confidence in change	Refactor, upgrade, or extend without fear of breaking existing behaviour
Documentation	Tests describe what the system actually does, not what someone intended
Design feedback	Hard-to-test code is often poorly designed — tests push toward better architecture
Defect prevention	Catching bugs in development costs 10–100× less than in production
Speed	Automated checks run in seconds; manual verification takes minutes to hours

The Cost of Bugs Over Time#

The later a defect is found, the more expensive it is to fix:

Threat Landscape & Attacker Mindset

Mon, 01 Jan 0001 00:00:00 +0000

Why This Matters#

You cannot defend what you don’t understand. Security is fundamentally adversarial — there are intelligent humans actively trying to break your systems. Understanding their motivations, methods, and tooling is the first step to building effective defenses.

Threat Actors#

Classification by Motivation#

Actor	Motivation	Resources	Persistence	Typical Targets
Script Kiddies	Fun, bragging rights	Low (public tools)	Low	Anything vulnerable
Hacktivists	Political/ideological	Low–Medium	Medium	Government, corporations
Cybercriminals	Financial gain	Medium–High	High	Anyone with money/data
Insiders	Revenge, money, negligence	High (already inside)	Varies	Their own organization
Nation-States (APT)	Espionage, sabotage, influence	Very High (unlimited)	Very High	Critical infrastructure, IP
Competitors	Business advantage	Medium	Medium	Trade secrets, strategies

Advanced Persistent Threats (APTs)#

APTs are the most dangerous — state-sponsored groups with:

Variables, Data Types, and Memory

Mon, 01 Jan 0001 00:00:00 +0000

What is a Variable?#

A variable is a named reference to a location in memory where a value is stored. The name is for humans; the computer only cares about the memory address.

Declaration: int age = 25;

What happens:
1. Compiler allocates 4 bytes (for int) at address 0x7FFF001A
2. Stores the value 25 in those bytes
3. Associates the name "age" with address 0x7FFF001A

Variable Lifecycle#

flowchart LR
 D["Declaration<br/>(allocate space)"] --> I["Initialization<br/>(assign first value)"]
 I --> U["Usage<br/>(read/write)"]
 U --> F["Destruction<br/>(free memory)"]

In most languages, destruction is automatic — either when the variable goes out of scope (stack) or when the garbage collector reclaims it (heap).

Web Security Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

Why Web Security Matters#

Every web application exposed to the internet is under constant attack. Automated scanners probe for vulnerabilities within minutes of a new service going live. A single SQL injection or misconfigured header can expose millions of user records, destroy a company’s reputation, and trigger regulatory fines measured in percentages of global revenue.

Security is not a feature — it is a property of the system. You cannot bolt it on at the end. This section establishes the foundational mental models that underpin every subsequent topic.

Algebra

Mon, 01 Jan 0001 00:00:00 +0000

Algebra#

Algebra is the machinery for manipulating symbols and solving equations — the language in which algorithms express their relationships, complexity bounds are derived, and configuration parameters are computed.

Expressions, Equations, and Identities#

Expressions#

An expression is a mathematical phrase that combines numbers, variables, and operations. It does not have an equals sign.

3x + 7
n² - 1
log₂(n) + 1

Equations#

An equation asserts that two expressions are equal. It can be solved for unknowns.

API Design

Mon, 01 Jan 0001 00:00:00 +0000

API Design#

Good API design makes the right thing easy and the wrong thing hard. This section covers the practical patterns for URL structure, request/response shaping, pagination, error handling, and bulk operations that production APIs rely on.

URL Structure#

A well-designed URL is predictable, readable, and consistent.

Anatomy of an API URL#

https://api.example.com/v1/customers/42/orders?status=shipped&page=2
└─────────┬──────────┘ └┬┘ └────┬────┘ └┘ └──┬──┘ └──────┬───────┘
 base URL ver collection id sub-res query params

Design Rules#

Rule	Good	Bad
Use nouns for resources	`/invoices`	`/getInvoices`
Plural for collections	`/users`	`/user`
IDs in path for specifics	`/users/42`	`/users?id=42`
Nest for relationships	`/users/42/posts`	`/posts?user_id=42` (acceptable for filtering)
Max 2 levels of nesting	`/users/42/posts`	`/users/42/posts/7/comments/3/likes`
Actions as sub-resources	`/orders/42/cancel`	`/cancelOrder?id=42`
Consistent naming	`snake_case` or `camelCase` (pick one)	Mixing conventions

When Nesting Gets Too Deep#

If you need more than two levels, flatten:

B-Trees & B+Trees

Mon, 01 Jan 0001 00:00:00 +0000

The Index Problem#

Given a heap file with millions of rows, finding a specific row without an index requires scanning every page — a full table scan. B-trees and B+trees solve this by maintaining a sorted, balanced tree structure that narrows the search to a handful of page reads.

The Databases & SQL path covered when to use indexes. This section covers how they work internally.

B-Tree Fundamentals#

A B-tree (Bayer & McCreight, 1972) is a self-balancing tree where:

CI Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

CI Fundamentals#

Continuous Integration (CI) is the practice of merging all developer working copies to a shared mainline frequently — ideally multiple times per day — and verifying each integration with an automated build and test suite.

What CI Solves#

Without CI, teams encounter these problems:

Problem	Symptom	How CI Solves It
Integration hell	Merging weeks of work causes conflicts and breakage	Small, frequent merges reduce conflict surface
“Works on my machine”	Code passes locally but fails elsewhere	Consistent build environment for every commit
Late bug discovery	Bugs found in QA weeks after introduction	Automated tests catch regressions immediately
Slow feedback	Developers wait days for test results	Pipeline runs in minutes on every push
Quality drift	Standards erode without enforcement	Linting and formatting enforced automatically
Release anxiety	Nobody knows if the codebase is deployable	The build is always green (or immediately fixed)

The CI Contract#

Every team practising CI agrees to:

Control Flow

Mon, 01 Jan 0001 00:00:00 +0000

Control flow determines the execution order of statements. Without it, programs would be strictly sequential — no decisions, no repetition, no branching.

The Three Fundamental Structures#

Every algorithm can be expressed using only three structures (Böhm-Jacopini theorem):

flowchart LR
 subgraph Sequence
 A1["Statement A"] --> B1["Statement B"] --> C1["Statement C"]
 end
 
 subgraph Selection
 D1{"Condition?"} -->|True| E1["Path A"]
 D1 -->|False| F1["Path B"]
 end
 
 subgraph Iteration
 G1{"Condition?"} -->|True| H1["Body"]
 H1 --> G1
 G1 -->|False| I1["Exit"]
 end

Conditionals (Selection)#

If/Else#

The most basic decision structure:

Control Flow and Functions

Mon, 01 Jan 0001 00:00:00 +0000

Conditionals#

if / elseif / else#

<?php
if ($age >= 18) {
 echo "Adult";
} elseif ($age >= 13) {
 echo "Teenager";
} else {
 echo "Child";
}

match Expression (PHP 8.0+)#

match is a strict-comparison alternative to switch that returns a value and doesn’t fall through:

<?php
$status = match ($code) {
 200 => 'OK',
 301 => 'Moved Permanently',
 404 => 'Not Found',
 500 => 'Internal Server Error',
 default => 'Unknown',
};

// match with complex conditions
$label = match (true) {
 $temp > 35 => 'Hot',
 $temp > 20 => 'Warm',
 $temp > 10 => 'Cool',
 default => 'Cold',
};

match vs switch:

Feature	switch	match
Comparison	Loose (`==`)	Strict (`===`)
Fall-through	Yes (needs `break`)	No
Returns value	No	Yes
Exhaustive	No	Yes (throws `UnhandledMatchError`)

Ternary and Short Ternary#

<?php
$label = $count > 0 ? 'has items' : 'empty';

// Short ternary (Elvis operator) — returns left if truthy, else right
$name = $input ?: 'Anonymous';
// Equivalent to: $name = $input ? $input : 'Anonymous';

Loops#

for#

<?php
for ($i = 0; $i < 10; $i++) {
 echo $i;
}

foreach#

The workhorse loop for arrays:

CPU Scheduling

Mon, 01 Jan 0001 00:00:00 +0000

CPU Scheduling#

The CPU scheduler decides which ready process gets the CPU next. This decision directly impacts system responsiveness, throughput, and fairness. A good scheduler makes a fast machine feel fast; a bad one makes it feel sluggish regardless of hardware.

Scheduling Criteria#

Criterion	Definition	Optimize
CPU utilization	Percentage of time CPU is busy	Maximize
Throughput	Processes completed per time unit	Maximize
Turnaround time	Total time from submission to completion	Minimize
Waiting time	Time spent in the ready queue	Minimize
Response time	Time from submission to first response	Minimize

These criteria often conflict. Maximizing throughput may hurt response time; optimizing for interactive responsiveness may reduce batch throughput.

Cryptography

Mon, 01 Jan 0001 00:00:00 +0000

Why This Matters#

Cryptography is the mathematical foundation of all digital security. Every time you visit a website, send a message, or authenticate to a service, cryptographic algorithms are protecting you. Understanding how they work — and how they fail — is essential.

Core Concepts#

Concept	Purpose	Property Provided
Encryption	Hide data from unauthorized readers	Confidentiality
Hashing	Create fixed-size fingerprint of data	Integrity
Digital Signatures	Prove who created/approved data	Authentication + Non-repudiation
Key Exchange	Securely agree on a shared secret	Confidentiality (setup)

Symmetric Encryption#

Same key for encryption and decryption. Fast, efficient for bulk data.

CSS Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

How CSS Works#

CSS (Cascading Style Sheets) controls the visual presentation of HTML. Understanding the cascade, specificity, and box model is essential — without them, you’re guessing why styles apply or don’t.

flowchart TD
 HTML["HTML Document"] --> Parse["Parse CSS"]
 Parse --> Cascade["Cascade<br>(resolve conflicts)"]
 Cascade --> Spec["Specificity<br>(which rule wins)"]
 Spec --> Inherit["Inheritance<br>(parent → child)"]
 Inherit --> Computed["Computed Values"]
 Computed --> Layout["Layout (Box Model)"]
 Layout --> Paint["Paint"]
 Paint --> Composite["Composite"]

The Cascade#

When multiple rules target the same element, the cascade determines which wins:

Data Modelling for Analytics

Mon, 01 Jan 0001 00:00:00 +0000

Why Analytical Data Modelling?#

Transactional databases (OLTP) are designed for fast reads and writes of individual records. Analytical databases (OLAP) are designed for aggregating large volumes of data across many dimensions. The schemas that serve OLTP well — highly normalised, with many joins — are inefficient for analytical queries. Analytical data modelling restructures data to optimise for read-heavy, aggregation-heavy workloads.

OLTP vs OLAP#

Aspect	OLTP	OLAP
Purpose	Transactional operations	Analytical queries
Query pattern	Point lookups, small updates	Full-table scans, aggregations
Schema	Normalised (3NF+)	Denormalised (star/snowflake)
Users	Applications, microservices	Analysts, BI tools, dashboards
Optimised for	Write throughput, consistency	Read throughput, query speed
Example	Insert an order	Total revenue by region per quarter

Dimensional Modelling#

Dimensional modelling, pioneered by Ralph Kimball, organises data into fact tables (measurements) and dimension tables (context). It is the most widely used approach for analytical data warehouses.

Data Preparation

Mon, 01 Jan 0001 00:00:00 +0000

Data Preparation#

Data preparation consumes 60–80% of a typical ML project’s time. A model trained on poorly prepared data will produce poor results regardless of how sophisticated the algorithm is. This section covers the essential transformations between raw data and model-ready features.

Exploratory Data Analysis (EDA)#

Before any transformation, understand your data:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

df = pd.read_csv("housing.csv")

# Shape, types, memory
print(df.shape)
print(df.dtypes)
print(df.describe())

# Missing values
print(df.isnull().sum().sort_values(ascending=False))

# Distribution of target
df["price"].hist(bins=50)
plt.xlabel("Price")
plt.ylabel("Count")
plt.title("Target Distribution")
plt.show()

# Correlation matrix
corr = df.select_dtypes(include=[np.number]).corr()
sns.heatmap(corr, annot=True, fmt=".2f", cmap="coolwarm", center=0)
plt.title("Feature Correlations")
plt.tight_layout()
plt.show()

EDA Checklist#

Check	What to Look For
Shape	Number of rows and features
Types	Numeric vs categorical vs datetime
Missing values	Percentage per column, patterns (MCAR/MAR/MNAR)
Distributions	Skew, outliers, multimodality
Target balance	Class imbalance for classification
Correlations	Redundant features, leakage signals
Duplicates	Exact or near-duplicate rows

Handling Missing Data#

Missing data is nearly universal. The strategy depends on why the data is missing.

Design Patterns

Mon, 01 Jan 0001 00:00:00 +0000

Design Patterns#

Design patterns are reusable solutions to common problems in software design. They are not ready-made code — they are templates for solving problems that occur repeatedly in different contexts. Understanding when (and when not) to apply them is a core architectural skill.

Pattern Categories#

graph TD
 DP[Design Patterns] --> C[Creational]
 DP --> S[Structural]
 DP --> B[Behavioral]
 
 C --> C1[Singleton]
 C --> C2[Factory Method]
 C --> C3[Abstract Factory]
 C --> C4[Builder]
 C --> C5[Prototype]
 
 S --> S1[Adapter]
 S --> S2[Facade]
 S --> S3[Decorator]
 S --> S4[Proxy]
 S --> S5[Composite]
 
 B --> B1[Strategy]
 B --> B2[Observer]
 B --> B3[Command]
 B --> B4[State]
 B --> B5[Template Method]

Category	Purpose	Key Question
Creational	Object creation mechanisms	How should objects be created?
Structural	Object composition and relationships	How should objects be composed?
Behavioral	Object communication and responsibility	How should objects communicate?

Creational Patterns#

Singleton#

Ensures a class has only one instance and provides a global access point.

Docker and Docker Compose

Mon, 01 Jan 0001 00:00:00 +0000

What Is Docker?#

Docker is a platform for building, running, and distributing applications inside containers — lightweight, isolated environments that package code with all its dependencies. Containers share the host OS kernel, making them far more efficient than virtual machines.

flowchart TB
 subgraph VM["Virtual Machine"]
 GA["Guest OS A"]
 AA["App A + Libs"]
 GA --- AA
 end
 subgraph VM2["Virtual Machine"]
 GB["Guest OS B"]
 AB["App B + Libs"]
 GB --- AB
 end
 subgraph Container["Containers"]
 CA["App A + Libs"]
 CB["App B + Libs"]
 end
 
 HV["Hypervisor"]
 DR["Docker Engine"]
 HOST["Host OS + Kernel"]
 HW["Hardware"]
 
 VM --- HV
 VM2 --- HV
 Container --- DR
 HV --- HOST
 DR --- HOST
 HOST --- HW

Containers vs Virtual Machines#

Aspect	Container	Virtual Machine
Boot time	Seconds	Minutes
Size	MBs (app + libs only)	GBs (full OS)
Isolation	Process-level (shared kernel)	Full hardware-level
Performance	Near-native	Hypervisor overhead
Density	Hundreds per host	Tens per host
Use case	Microservices, CI/CD, dev environments	Full OS isolation, legacy apps

Core Concepts#

Concept	What It Is
Image	Read-only template with app code, runtime, libraries, and config. Blueprint for containers.
Container	Running instance of an image. Isolated process with its own filesystem, networking, PID space.
Dockerfile	Text file with instructions to build an image layer by layer.
Registry	Storage for images (Docker Hub, ECR, GCR, GitLab Registry).
Volume	Persistent storage that survives container restarts/removal.
Network	Virtual network connecting containers.

Essential Commands#

Images#

# Pull from registry
docker pull nginx:1.25
docker pull python:3.12-slim

# List local images
docker images
docker image ls

# Build from Dockerfile
docker build -t myapp:1.0 .
docker build -t myapp:latest -f Dockerfile.prod .

# Remove
docker rmi nginx:1.25
docker image prune # Remove dangling images
docker image prune -a # Remove all unused images

Containers#

# Run (create + start)
docker run nginx # Foreground
docker run -d nginx # Detached (background)
docker run -d -p 8080:80 nginx # Map host:container port
docker run -d --name web -p 8080:80 nginx # Named container
docker run --rm alpine echo "hello" # Remove after exit

# Interactive shell
docker run -it ubuntu bash
docker exec -it web bash # Shell into running container

# Lifecycle
docker start web
docker stop web
docker restart web
docker rm web # Remove stopped container
docker rm -f web # Force remove (running)

# Inspect
docker ps # Running containers
docker ps -a # All (including stopped)
docker logs web # View logs
docker logs -f web # Follow logs
docker inspect web # Full JSON details
docker stats # Live resource usage

Volumes#

# Named volume (managed by Docker)
docker volume create mydata
docker run -d -v mydata:/app/data myapp

# Bind mount (host path)
docker run -d -v $(pwd)/src:/app/src myapp

# List and clean
docker volume ls
docker volume prune

Dockerfile#

Anatomy#

# Base image
FROM python:3.12-slim

# Metadata
LABEL maintainer="you@example.com"

# Set working directory
WORKDIR /app

# Install dependencies (cached layer)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY src/ ./src/

# Environment variables
ENV ENVIRONMENT=production
ENV PORT=8000

# Expose port (documentation)
EXPOSE 8000

# Non-root user (security)
RUN useradd -r appuser
USER appuser

# Default command
CMD ["python", "-m", "uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8000"]

Key Instructions#

Instruction	Purpose
`FROM`	Base image (always first)
`WORKDIR`	Set working directory for subsequent instructions
`COPY`	Copy files from build context into image
`ADD`	Like COPY but can extract tarballs and fetch URLs
`RUN`	Execute command during build (creates layer)
`ENV`	Set environment variable
`EXPOSE`	Document which port the container listens on
`USER`	Switch to non-root user
`CMD`	Default command when container starts (overridable)
`ENTRYPOINT`	Fixed command (CMD becomes arguments)
`ARG`	Build-time variable (not available at runtime)
`VOLUME`	Declare mount point
`HEALTHCHECK`	Define health check command

Multi-Stage Builds#

Produce small production images by separating build from runtime:

Functions and Closures

Mon, 01 Jan 0001 00:00:00 +0000

Functions and Closures#

Functions are first-class citizens in Swift — they can be assigned to variables, passed as arguments, and returned from other functions. Closures are self-contained blocks of functionality that capture their surrounding context.

Function Syntax#

A Swift function has a name, parameter list, and return type:

func greet(person: String) -> String {
 return "Hello, \(person)!"
}

let message = greet(person: "Alice") // "Hello, Alice!"

Functions with no return value implicitly return Void (a type alias for ()):

func logMessage(_ message: String) {
 print("[LOG] \(message)")
}

Single-Expression Functions#

When the body is a single expression, the return keyword is implicit:

Functions and Lambdas

Mon, 01 Jan 0001 00:00:00 +0000

Functions and Lambdas#

Kotlin’s function system combines concise syntax with powerful features like default parameters, extension functions, and first-class lambdas. Functions are the primary building blocks for organising logic, and Kotlin makes them flexible enough to serve as both simple utilities and composable abstractions.

Function Syntax#

Functions are declared with the fun keyword. The return type follows the parameter list:

fun greet(name: String): String {
 return "Hello, $name!"
}

// Single-expression functions omit braces and return
fun greet(name: String): String = "Hello, $name!"

// Return type can be inferred for single-expression functions
fun greet(name: String) = "Hello, $name!"

Functions that return nothing meaningful use Unit (equivalent to void). The return type and return statement can be omitted:

Giving & Receiving Feedback

Mon, 01 Jan 0001 00:00:00 +0000

Giving & Receiving Feedback#

Feedback is how teams get better. Without it, bad patterns persist, good work goes unrecognised, and resentment builds silently. Engineers who can give direct, kind feedback — and receive it without defensiveness — create disproportionate value.

Why Engineers Struggle with Feedback#

Code feels personal — criticising someone’s code can feel like criticising their intelligence
Conflict avoidance — it’s easier to fix the code yourself than have the conversation
Lack of frameworks — “this is bad” is not feedback, it’s a verdict
Remote work — text strips tone, making feedback feel harsher

The SBI Model (Situation-Behavior-Impact)#

The most reliable framework for specific, actionable feedback:

Hashmaps & Sets

Mon, 01 Jan 0001 00:00:00 +0000

Hashmaps & Sets#

Hash-based structures provide O(1) average-case lookup, making them the first tool to reach for when you need to check existence, count frequencies, or group elements.

Pattern Summary#

Pattern	Time	Space	When to Use
Frequency Counting	O(n)	O(k)	Count occurrences, find duplicates, majority element
Two-Sum (Complement)	O(n)	O(n)	Find pair summing to target (unsorted)
Anagram Detection	O(n)	O(1)*	Compare character frequencies
Grouping	O(n)	O(n)	Group elements by shared property
Set Operations	O(n)	O(n)	Intersection, union, difference

*O(1) space when alphabet is fixed (e.g., 26 lowercase letters).

HCL Syntax

Mon, 01 Jan 0001 00:00:00 +0000

What is HCL?#

HCL (HashiCorp Configuration Language) is a domain-specific language designed for infrastructure configuration. It’s not a general-purpose programming language — it’s purpose-built to be human-readable, machine-parseable, and expressive enough to describe complex infrastructure.

Block Structure#

Everything in HCL is organized into blocks:

block_type "label_1" "label_2" {
 argument = value
 
 nested_block {
 nested_argument = value
 }
}

Common Block Types#

Block	Labels	Purpose
`resource`	type, name	Create infrastructure
`data`	type, name	Read existing infrastructure
`variable`	name	Declare input
`output`	name	Expose values
`locals`	(none)	Define computed values
`module`	name	Use a reusable module
`provider`	name	Configure a provider
`terraform`	(none)	Terraform settings

# Resource block: 2 labels (type + local name)
resource "aws_s3_bucket" "data_lake" {
 bucket = "my-data-lake"
}

# Variable block: 1 label (name)
variable "region" {
 type = string
 default = "eu-central-1"
}

# Locals block: no labels
locals {
 prefix = "myapp-${var.environment}"
}

Types and Values#

Primitive Types#

# String
name = "web-server"

# Number (integer or float)
port = 8080
timeout = 3.5

# Boolean
enabled = true

Collection Types#

# List (ordered, duplicates allowed)
availability_zones = ["eu-central-1a", "eu-central-1b", "eu-central-1c"]

# Map (key-value pairs)
tags = {
 Environment = "prod"
 Team = "platform"
}

# Set (unordered, no duplicates)
allowed_ips = toset(["10.0.0.1", "10.0.0.2"])

Structural Types#

# Object (fixed structure with named attributes)
variable "database" {
 type = object({
 engine = string
 instance_class = string
 storage_gb = number
 multi_az = bool
 })
}

# Tuple (fixed-length, mixed types)
variable "rule" {
 type = tuple([string, number, bool])
 # Example: ["allow", 443, true]
}

Type Constraints#

variable "ports" {
 type = list(number) # list of numbers only
}

variable "tags" {
 type = map(string) # map with string values only
}

variable "services" {
 type = list(object({
 name = string
 port = number
 path = optional(string, "/") # optional with default
 }))
}

Expressions#

String Interpolation and Templates#

# Simple interpolation
name = "app-${var.environment}-${var.region}"

# Heredoc (multi-line strings)
policy = <<-EOT
 {
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Action": "s3:GetObject",
 "Resource": "arn:aws:s3:::${var.bucket_name}/*"
 }]
 }
EOT

# Directive (template logic)
user_data = <<-EOT
 #!/bin/bash
 %{ for port in var.ports ~}
 ufw allow ${port}
 %{ endfor ~}
EOT

Operators#

# Arithmetic
total = var.base_count * var.multiplier

# Comparison
is_large = var.instance_count > 10

# Logical
needs_scaling = var.is_production && var.high_traffic

Conditional Expressions#

# condition ? true_value : false_value
instance_type = var.environment == "prod" ? "t3.large" : "t3.micro"

# Nested (avoid — use locals for readability)
size = var.tier == "large" ? 100 : var.tier == "medium" ? 50 : 20

For Expressions#

# Transform a list
upper_names = [for name in var.names : upper(name)]

# Filter a list
prod_instances = [for i in var.instances : i if i.environment == "prod"]

# List to map
instance_map = { for i in var.instances : i.name => i.id }

# Map transformation
uppercased = { for k, v in var.tags : k => upper(v) }

Splat Expressions#

# Shorthand for [for o in list : o.attribute]
instance_ids = aws_instance.web[*].id
subnet_cidrs = aws_subnet.private[*].cidr_block

# Equivalent to:
instance_ids = [for i in aws_instance.web : i.id]

References#

Resource References#

resource "aws_vpc" "main" {
 cidr_block = "10.0.0.0/16"
}

resource "aws_subnet" "public" {
 vpc_id = aws_vpc.main.id # reference another resource
 cidr_block = cidrsubnet(aws_vpc.main.cidr_block, 8, 0)
}

Reference Types#

Reference	Syntax	Example
Resource attribute	`type.name.attribute`	`aws_vpc.main.id`
Variable	`var.name`	`var.environment`
Local	`local.name`	`local.common_tags`
Module output	`module.name.output`	`module.vpc.vpc_id`
Data source	`data.type.name.attribute`	`data.aws_ami.latest.id`
Count index	`count.index`	`count.index`
Each key/value	`each.key`, `each.value`	`each.value.port`
Self (provisioners)	`self.attribute`	`self.public_ip`

Dynamic Blocks#

Generate repeated nested blocks from a collection:

IAM (Identity and Access Management)

Mon, 01 Jan 0001 00:00:00 +0000

What is IAM?#

IAM controls who (authentication) can do what (authorization) on which AWS resources. It’s the security foundation of every AWS account — misconfigure IAM and nothing else matters.

flowchart LR
 Who["WHO?<br>(Principal)"] --> Auth["Authenticated?"]
 Auth --> What["WHAT?<br>(Action)"]
 What --> Which["ON WHICH?<br>(Resource)"]
 Which --> Conditions["UNDER WHAT CONDITIONS?<br>(Conditions)"]
 Conditions --> Decision{"ALLOW or DENY?"}

IAM Entities#

Users#

An IAM user represents a person or application that interacts with AWS:

{
 "UserName": "alice",
 "Arn": "arn:aws:iam::123456789012:user/alice",
 "CreateDate": "2024-01-15T10:00:00Z"
}

Best practice: Minimize IAM users. Use IAM Identity Center (SSO) for humans and IAM roles for applications.

Injection Attacks

Mon, 01 Jan 0001 00:00:00 +0000

What Is Injection?#

Injection occurs when untrusted data is sent to an interpreter as part of a command or query. The interpreter cannot distinguish between intended instructions and injected instructions. This has been the most devastating class of web vulnerability for over two decades.

The root cause is always the same: string concatenation of user input into a structured language (SQL, shell, LDAP, NoSQL query).

SQL Injection#

How It Works#

The application builds a SQL query by concatenating user input directly:

Java: Syntax, Types, and Operators

Mon, 01 Jan 0001 00:00:00 +0000

Primitive Types#

Java has 8 primitive types — they live on the stack, not the heap.

Type	Size	Range	Default	Wrapper
`byte`	8 bits	-128 to 127	0	`Byte`
`short`	16 bits	-32,768 to 32,767	0	`Short`
`int`	32 bits	-2³¹ to 2³¹-1	0	`Integer`
`long`	64 bits	-2⁶³ to 2⁶³-1	0L	`Long`
`float`	32 bits	±3.4×10³⁸	0.0f	`Float`
`double`	64 bits	±1.7×10³⁰⁸	0.0d	`Double`
`char`	16 bits	0 to 65,535 (Unicode)	‘\u0000’	`Character`
`boolean`	JVM-specific	true/false	false	`Boolean`

// Literals
int decimal = 42;
int hex = 0x2A; // 42 in hex
int binary = 0b101010; // 42 in binary
long big = 10_000_000L; // Underscores for readability
double pi = 3.14159;
float f = 3.14f; // Must suffix with f
char c = 'A';
char unicode = '\u0041'; // Also 'A'

// Overflow wraps silently!
int max = Integer.MAX_VALUE; // 2,147,483,647
int overflow = max + 1; // -2,147,483,648 (no exception!)

Autoboxing and Unboxing#

// Autoboxing: primitive → wrapper (automatic)
Integer boxed = 42; // int → Integer
List<Integer> list = new ArrayList<>();
list.add(5); // int → Integer (autoboxed)

// Unboxing: wrapper → primitive (automatic)
int unboxed = boxed; // Integer → int
int sum = boxed + 10; // Unbox, add, result is int

// DANGER: NullPointerException on unboxing null
Integer nullable = null;
int crash = nullable; // NullPointerException!

// Integer cache: -128 to 127 are cached
Integer a = 127;
Integer b = 127;
System.out.println(a == b); // true (same cached object)

Integer c = 128;
Integer d = 128;
System.out.println(c == d); // false (different objects!)
System.out.println(c.equals(d)); // true (value comparison)

Strings#

// Strings are immutable objects
String s = "Hello";
s.toUpperCase(); // Returns NEW string; s is unchanged
String upper = s.toUpperCase(); // "HELLO"

// String pool (interning)
String a = "hello"; // Goes to string pool
String b = "hello"; // Same reference from pool
String c = new String("hello"); // New object on heap (avoid this)
a == b; // true (same reference)
a == c; // false (different objects)
a.equals(c); // true (same content) — ALWAYS use .equals() for strings

// StringBuilder for concatenation in loops
StringBuilder sb = new StringBuilder();
for (int i = 0; i < 1000; i++) {
 sb.append("item ").append(i).append("\n");
}
String result = sb.toString();

// Text blocks (Java 13+)
String json = """
 {
 "name": "Alice",
 "age": 30
 }
 """;

// Useful methods
"Hello World".substring(0, 5); // "Hello"
"Hello World".split(" "); // ["Hello", "World"]
" spaces ".strip(); // "spaces" (Java 11+)
"Hello".contains("ell"); // true
"Hello".startsWith("He"); // true
String.format("Name: %s, Age: %d", name, age);
"Hello %s".formatted("World"); // Java 15+

Type Inference (var)#

// Local variable type inference (Java 10+)
var list = new ArrayList<String>(); // Inferred as ArrayList<String>
var map = Map.of("key", "value"); // Inferred as Map<String, String>
var stream = list.stream(); // Inferred as Stream<String>

// Where var CANNOT be used:
// - Method parameters
// - Return types
// - Fields (instance/class variables)
// - When initializer is null

// Good uses
var connection = DriverManager.getConnection(url); // Type is obvious
var entries = map.entrySet(); // Avoids verbose Set<Map.Entry<K,V>>

// Bad uses (reduces readability)
var result = process(data); // What type is result?
var x = 42; // Just use int

Arrays#

// Declaration and initialization
int[] numbers = new int[5]; // [0, 0, 0, 0, 0]
int[] primes = {2, 3, 5, 7, 11}; // Literal initialization
String[] names = new String[3]; // [null, null, null]

// Multi-dimensional
int[][] matrix = new int[3][4]; // 3 rows, 4 columns
int[][] jagged = { // Rows can have different lengths
 {1, 2, 3},
 {4, 5},
 {6, 7, 8, 9}
};

// Arrays are objects with fixed size
numbers.length; // 5 (field, not method)
// numbers[5]; // ArrayIndexOutOfBoundsException

// Utility methods
import java.util.Arrays;
Arrays.sort(numbers);
Arrays.fill(numbers, 0);
int[] copy = Arrays.copyOf(numbers, 10); // Extends with zeros
Arrays.equals(a, b); // Content comparison
Arrays.toString(numbers); // "[0, 0, 0, 0, 0]"

Control Flow#

// Enhanced switch (Java 14+) — expression form
String dayType = switch (day) {
 case MONDAY, TUESDAY, WEDNESDAY, THURSDAY, FRIDAY -> "Weekday";
 case SATURDAY, SUNDAY -> "Weekend";
};

// Switch with blocks
int numLetters = switch (day) {
 case MONDAY, FRIDAY, SUNDAY -> 6;
 case TUESDAY -> 7;
 case THURSDAY, SATURDAY -> 8;
 case WEDNESDAY -> 9;
 default -> throw new IllegalArgumentException("Invalid day: " + day);
};

// Pattern matching for instanceof (Java 16+)
if (obj instanceof String s) {
 // s is already cast to String — no explicit cast needed
 System.out.println(s.length());
}

// Pattern matching in switch (Java 21+)
String describe(Object obj) {
 return switch (obj) {
 case Integer i when i > 0 -> "Positive integer: " + i;
 case Integer i -> "Non-positive integer: " + i;
 case String s -> "String of length " + s.length();
 case null -> "null";
 default -> "Unknown: " + obj.getClass().getName();
 };
}

// For-each
for (String name : names) {
 System.out.println(name);
}

// Labeled break (for nested loops)
outer:
for (int i = 0; i < rows; i++) {
 for (int j = 0; j < cols; j++) {
 if (matrix[i][j] == target) {
 found = true;
 break outer; // Breaks out of BOTH loops
 }
 }
}

Records (Java 16+)#

// Records: immutable data carriers with auto-generated methods
public record Point(double x, double y) {
 // Auto-generated: constructor, getters (x(), y()), equals(), hashCode(), toString()
 
 // Compact constructor for validation
 public Point {
 if (Double.isNaN(x) || Double.isNaN(y)) {
 throw new IllegalArgumentException("Coordinates cannot be NaN");
 }
 }
 
 // Additional methods
 public double distanceTo(Point other) {
 return Math.sqrt(Math.pow(x - other.x, 2) + Math.pow(y - other.y, 2));
 }
}

// Usage
var p1 = new Point(3, 4);
var p2 = new Point(0, 0);
p1.x(); // 3.0 (accessor, not getX())
p1.distanceTo(p2); // 5.0
p1.toString(); // "Point[x=3.0, y=4.0]"

// Records in pattern matching
record Range(int min, int max) {}

static String classify(Range r) {
 return switch (r) {
 case Range(int min, int max) when min == max -> "Single value: " + min;
 case Range(int min, int max) when max - min < 10 -> "Narrow range";
 case Range(int min, int max) -> "Wide range: " + min + " to " + max;
 };
}

Sealed Classes (Java 17+)#

// Sealed: restricts which classes can extend/implement
public sealed interface Shape permits Circle, Rectangle, Triangle {
 double area();
}

public record Circle(double radius) implements Shape {
 public double area() { return Math.PI * radius * radius; }
}

public record Rectangle(double width, double height) implements Shape {
 public double area() { return width * height; }
}

public record Triangle(double base, double height) implements Shape {
 public double area() { return 0.5 * base * height; }
}

// Exhaustive pattern matching — compiler knows all subtypes
String describe(Shape shape) {
 return switch (shape) {
 case Circle c -> "Circle with radius " + c.radius();
 case Rectangle r -> "Rectangle " + r.width() + "x" + r.height();
 case Triangle t -> "Triangle with base " + t.base();
 // No default needed — sealed type is exhaustive
 };
}

Hypothetical Use Case: Type-Safe Configuration#

public sealed interface ConfigValue permits StringValue, IntValue, BoolValue, ListValue {
 String key();
}

public record StringValue(String key, String value) implements ConfigValue {}
public record IntValue(String key, int value) implements ConfigValue {}
public record BoolValue(String key, boolean value) implements ConfigValue {}
public record ListValue(String key, List<String> values) implements ConfigValue {}

public class Config {
 private final Map<String, ConfigValue> entries = new HashMap<>();
 
 public void set(ConfigValue value) {
 entries.put(value.key(), value);
 }
 
 public String getString(String key) {
 return switch (entries.get(key)) {
 case StringValue sv -> sv.value();
 case IntValue iv -> String.valueOf(iv.value());
 case BoolValue bv -> String.valueOf(bv.value());
 case ListValue lv -> String.join(",", lv.values());
 case null -> throw new NoSuchElementException("Key not found: " + key);
 };
 }
}

Key Takeaways#

Primitives live on the stack — wrappers are objects on the heap (autoboxing has cost)
Always use .equals() for object comparison — == compares references
Strings are immutable — use StringBuilder for repeated concatenation
var is for local variables only — use when the type is obvious from context
Records replace boilerplate data classes — immutable, with auto-generated methods
Sealed types + pattern matching enable exhaustive, type-safe branching
Integer cache (-128 to 127) means == works for small boxed integers but fails for larger ones

Metrics

Mon, 01 Jan 0001 00:00:00 +0000

Metrics#

Metrics are numeric measurements collected at regular intervals and aggregated over time. They are the most efficient observability signal — compact to store, fast to query, and ideal for dashboards and alerting. This section covers metric types, the Prometheus data model, PromQL fundamentals, and naming conventions.

Metric Types#

There are four fundamental metric types. Each serves a different purpose.

Counter#

A counter is a cumulative value that only goes up (or resets to zero on restart). Use counters for things you count: requests, errors, bytes transferred.

Multi-Account & Organisation Strategy

Mon, 01 Jan 0001 00:00:00 +0000

Multi-Account & Organisation Strategy#

A single cloud account for everything is the number one architectural mistake organisations make. It creates a blast radius that spans your entire business: one misconfigured IAM policy can expose production data, development experiments can exhaust production quotas, and cost attribution becomes impossible. This section covers how to structure accounts, enforce guardrails, and automate provisioning.

Why Multiple Accounts?#

A single account conflates environments, teams, and workloads into one security and billing boundary. Multiple accounts provide:

Networking & Protocols

Mon, 01 Jan 0001 00:00:00 +0000

Networking & Protocols#

Understanding how data moves across the network is the foundation of system design. Every architectural decision — from API style to database placement — is constrained by networking realities.

DNS Resolution#

The Domain Name System translates human-readable domain names into IP addresses. A single page load may trigger dozens of DNS lookups.

Resolution Flow#

sequenceDiagram
 participant Client
 participant Resolver as Recursive Resolver
 participant Root as Root NS
 participant TLD as TLD NS (.com)
 participant Auth as Authoritative NS

 Client->>Resolver: Who is api.example.com?
 Resolver->>Root: Who handles .com?
 Root-->>Resolver: Try TLD NS at 192.5.6.30
 Resolver->>TLD: Who handles example.com?
 TLD-->>Resolver: Try Auth NS at 205.251.195.35
 Resolver->>Auth: What is api.example.com?
 Auth-->>Resolver: A record: 93.184.216.34 (TTL 300s)
 Resolver-->>Client: 93.184.216.34

DNS Record Types#

Record	Purpose	Example
A	Maps name to IPv4 address	`api.example.com → 93.184.216.34`
AAAA	Maps name to IPv6 address	`api.example.com → 2606:2800:220:1:...`
CNAME	Alias to another domain name	`www.example.com → example.com`
MX	Mail server routing	`example.com → mail.example.com`
NS	Delegates a zone to nameservers	`example.com → ns1.provider.com`
SRV	Service discovery (host + port)	`_http._tcp.example.com → 8080 web1`
TXT	Arbitrary text (SPF, verification)	`v=spf1 include:_spf.google.com`

DNS Caching Layers#

Browser cache — respects TTL from response
OS resolver cache — shared across all applications
Router/ISP cache — intermediate caching resolver
Recursive resolver — (e.g., 8.8.8.8, 1.1.1.1) caches per TTL

Design implication: Low TTLs (30–60s) enable fast failover but increase lookup latency. High TTLs (3600s+) improve performance but slow propagation of changes.

Operating Systems

Mon, 01 Jan 0001 00:00:00 +0000

What is an Operating System?#

An OS is the software layer between hardware and applications. It manages resources (CPU, memory, disk, I/O) and provides abstractions so programs don’t need to interact with hardware directly.

flowchart TD
 Apps["Applications (browser, editor, server)"]
 Apps --> OS["Operating System"]
 OS --> HW["Hardware (CPU, RAM, Disk, Network)"]

Core Responsibilities#

Function	What It Does
Process management	Create, schedule, and terminate processes
Memory management	Allocate RAM, virtual memory, paging
File system	Organize and access persistent data
I/O management	Interface with devices (disk, network, display)
Security	User permissions, process isolation

Process Management#

What is a Process?#

A process is a running program — it has its own memory space, file descriptors, and execution state.

Physical and Data Link Layers

Mon, 01 Jan 0001 00:00:00 +0000

Physical Layer (Layer 1)#

The physical layer converts bits into signals and transmits them over a medium.

Transmission Media#

Medium	Speed	Distance	Use Case
Cat5e (copper)	1 Gbps	100m	Office LAN
Cat6a (copper)	10 Gbps	100m	Data center
Single-mode fiber	100+ Gbps	80+ km	Long-haul, WAN
Multi-mode fiber	10-100 Gbps	300-550m	Data center
Wi-Fi 6 (802.11ax)	9.6 Gbps (theoretical)	~30m indoor	Wireless LAN

Data Link Layer (Layer 2)#

Responsible for local delivery — getting frames from one device to another on the same network segment.

Pointers & Memory

Mon, 01 Jan 0001 00:00:00 +0000

Pointers & Memory#

Pointers are the most powerful and most dangerous feature of C. A pointer is a variable that holds a memory address. Understanding pointers is understanding how the computer actually manages data.

What Is a Pointer?#

int x = 42;
int *p = &x; // p holds the address of x

printf("%d\n", x); // 42 (the value)
printf("%p\n", (void*)p); // 0x7ffd5e8c3a4c (the address)
printf("%d\n", *p); // 42 (dereference — read value at address)

*p = 100; // write through the pointer
printf("%d\n", x); // 100 (x was modified via p)

Operation	Syntax	Meaning
Declare pointer	`int *p`	p is a pointer to int
Address-of	`&x`	Get the address of x
Dereference	`*p`	Access the value at address p
NULL pointer	`NULL` or `(void*)0`	Points to nothing

Stack vs Heap#

┌─────────────────────┐ High address
│ Stack │ ← function locals, grows downward
│ (automatic, LIFO) │
├─────────────────────┤
│ ↓ │
│ (free space) │
│ ↑ │
├─────────────────────┤
│ Heap │ ← malloc'd memory, grows upward
│ (dynamic, manual) │
├─────────────────────┤
│ Data (BSS + init) │ ← globals, statics
├─────────────────────┤
│ Text │ ← program code (read-only)
└─────────────────────┘ Low address

Memory Region	Allocated By	Freed By	Lifetime
Stack	Function call	Function return	Automatic
Heap	`malloc()`, `calloc()`	`free()`	Manual
Data (global)	Program start	Program end	Program lifetime
Text (code)	Program load	Program end	Program lifetime

Dynamic Memory#

#include <stdlib.h>

// Allocate
int *arr = malloc(10 * sizeof(int)); // 10 ints, UNINITIALIZED
int *arr = calloc(10, sizeof(int)); // 10 ints, ZEROED

// Check for failure
if (arr == NULL) {
 perror("malloc failed");
 exit(1);
}

// Use
arr[0] = 42;
arr[9] = 100;

// Resize
arr = realloc(arr, 20 * sizeof(int)); // grow to 20 ints

// Free
free(arr);
arr = NULL; // prevent use-after-free

The Four Memory Bugs#

Bug	Cause	Consequence
Memory leak	`malloc` without `free`	Unbounded memory growth
Use-after-free	Dereference after `free`	Undefined behaviour (crash, corruption)
Double free	`free` the same pointer twice	Heap corruption
Buffer overflow	Write past allocated bounds	Stack smashing, code execution

// LEAK — forgot to free
void leak(void) {
 int *p = malloc(100);
 return; // p is lost, 100 bytes leaked
}

// USE-AFTER-FREE
int *p = malloc(sizeof(int));
*p = 42;
free(p);
printf("%d\n", *p); // UNDEFINED BEHAVIOUR

// DOUBLE FREE
free(p);
free(p); // UNDEFINED BEHAVIOUR

// BUFFER OVERFLOW
int arr[5];
arr[10] = 42; // writes beyond array bounds — UNDEFINED BEHAVIOUR

Pointer Arithmetic#

Pointers can be incremented and decremented. The step size is the size of the pointed-to type:

Prototypes and `this`

Mon, 01 Jan 0001 00:00:00 +0000

The Prototype Chain#

JavaScript uses prototypal inheritance — objects inherit directly from other objects. There are no classes at the engine level (ES6 class is syntactic sugar).

Every object has an internal [[Prototype]] slot pointing to another object (or null). When you access a property, the engine walks up this chain until it finds the property or reaches null.

flowchart TD
 subgraph Chain["Prototype Chain"]
 direction TB
 myObj["myObj<br>{name: 'Alice'}"]
 Person["Person.prototype<br>{greet(), toString()}"]
 ObjProto["Object.prototype<br>{hasOwnProperty(), valueOf()...}"]
 Null["null"]
 end
 
 myObj -->|"[[Prototype]]"| Person
 Person -->|"[[Prototype]]"| ObjProto
 ObjProto -->|"[[Prototype]]"| Null
 
 style Null fill:#ffcdd2

const animal = {
 type: "animal",
 speak() {
 return `${this.name} says ${this.sound}`;
 }
};

const dog = Object.create(animal); // dog's [[Prototype]] = animal
dog.name = "Rex";
dog.sound = "woof";

dog.speak(); // "Rex says woof" — found on animal via chain
dog.type; // "animal" — inherited
dog.hasOwnProperty("name"); // true — own property
dog.hasOwnProperty("type"); // false — inherited

Property Lookup Mechanics#

sequenceDiagram
 participant Code
 participant obj as myObj
 participant proto as Prototype
 participant objProto as Object.prototype
 
 Code->>obj: Access obj.property
 alt Found on obj
 obj-->>Code: Return value
 else Not found
 obj->>proto: Check [[Prototype]]
 alt Found on prototype
 proto-->>Code: Return value
 else Not found
 proto->>objProto: Check [[Prototype]]
 alt Found
 objProto-->>Code: Return value
 else Reached null
 objProto-->>Code: undefined
 end
 end
 end

Property Shadowing#

const parent = { x: 1, y: 2 };
const child = Object.create(parent);

child.x = 10; // Creates OWN property on child (shadows parent.x)

child.x; // 10 (own property)
child.y; // 2 (inherited from parent)
parent.x; // 1 (unchanged)

// Deleting own property reveals inherited one
delete child.x;
child.x; // 1 (now reads from parent)

Property Descriptors#

const obj = {};

Object.defineProperty(obj, "id", {
 value: 42,
 writable: false, // Cannot reassign
 enumerable: true, // Shows in for...in
 configurable: false // Cannot delete or reconfigure
});

obj.id = 100; // Silently fails (or TypeError in strict mode)
obj.id; // 42

// Getters and setters
const user = {
 _name: "Alice",
 get name() { return this._name; },
 set name(val) {
 if (val.length < 2) throw new Error("Name too short");
 this._name = val;
 }
};

user.name; // "Alice" (calls getter)
user.name = "Bo"; // Error: Name too short

Constructor Functions and `new`#

Before ES6 classes, constructor functions were the standard pattern:

Python: Control Flow, Functions, and Modules

Mon, 01 Jan 0001 00:00:00 +0000

Control Flow#

Conditionals — Beyond Basics#

# Walrus operator (Python 3.8+) — assign and test in one expression
if (n := len(data)) > 100:
 print(f"Processing {n} items in batches")

# Chained comparisons
if 0 <= score <= 100:
 print("Valid score")

# Match statement (Python 3.10+) — structural pattern matching
def handle_command(command):
 match command.split():
 case ["quit"]:
 return "Goodbye"
 case ["hello", name]:
 return f"Hello, {name}!"
 case ["add", *numbers] if all(n.isdigit() for n in numbers):
 return sum(int(n) for n in numbers)
 case _:
 return "Unknown command"

# Pattern matching with objects
match event:
 case {"type": "click", "x": x, "y": y}:
 handle_click(x, y)
 case {"type": "keypress", "key": key}:
 handle_key(key)
 case {"type": "scroll", "direction": "up" | "down" as direction}:
 handle_scroll(direction)

Iteration Patterns#

# zip — iterate multiple sequences in parallel
names = ["Alice", "Bob", "Charlie"]
scores = [95, 87, 92]
for name, score in zip(names, scores):
 print(f"{name}: {score}")

# zip_longest — don't stop at shortest
from itertools import zip_longest
for a, b in zip_longest([1, 2, 3], [10, 20], fillvalue=0):
 print(a, b) # (1,10), (2,20), (3,0)

# enumerate with start
for i, line in enumerate(lines, start=1):
 print(f"Line {i}: {line}")

# reversed and sorted
for item in reversed(items):
 process(item)

for item in sorted(items, key=lambda x: x.priority, reverse=True):
 process(item)

# Dictionary iteration patterns
for key in sorted(config.keys()):
 print(f"{key} = {config[key]}")

# Nested loop with product
from itertools import product
for x, y in product(range(3), range(3)):
 print(f"({x}, {y})")

Comprehensions — Advanced#

# Nested comprehension (flatten)
matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
flat = [x for row in matrix for x in row] # [1, 2, 3, 4, 5, 6, 7, 8, 9]

# Conditional expression in comprehension
labels = ["PASS" if score >= 60 else "FAIL" for score in scores]

# Dictionary comprehension with filtering
env_vars = {k: v for k, v in os.environ.items() if k.startswith("APP_")}

# Set comprehension
unique_domains = {email.split("@")[1] for email in emails}

# Walrus in comprehension (filter + transform)
results = [cleaned for raw in data if (cleaned := clean(raw)) is not None]

Functions — Advanced Patterns#

Parameter Types#

def complex_function(
 required_arg, # positional or keyword
 *args, # variable positional
 keyword_only, # must be passed by name
 default_kwarg="default", # keyword with default
 **kwargs # variable keyword
):
 pass

# Positional-only parameters (Python 3.8+)
def divmod(a, b, /): # a and b MUST be positional
 return a // b, a % b

# Combined
def api_call(url, /, *, method="GET", headers=None, **params):
 # url is positional-only
 # method, headers are keyword-only
 # params catches extra keyword args
 pass

Closures and Factories#

def make_validator(min_val, max_val, required=True):
 """Factory: creates a validation function with captured config."""
 def validate(value):
 if value is None:
 return not required
 return min_val <= value <= max_val
 return validate

validate_age = make_validator(0, 150)
validate_score = make_validator(0, 100)
validate_optional_rating = make_validator(1, 5, required=False)

validate_age(25) # True
validate_age(200) # False
validate_optional_rating(None) # True

Partial Application#

from functools import partial

def power(base, exponent):
 return base ** exponent

square = partial(power, exponent=2)
cube = partial(power, exponent=3)

square(5) # 25
cube(3) # 27

# Practical: configure a logger
import logging
debug = partial(logging.log, logging.DEBUG)
error = partial(logging.log, logging.ERROR)
debug("This is a debug message")

Function Annotations and Overloading#

from typing import overload

@overload
def process(data: str) -> list[str]: ...
@overload
def process(data: list[str]) -> str: ...

def process(data):
 if isinstance(data, str):
 return data.split()
 elif isinstance(data, list):
 return " ".join(data)

Modules and Packages#

Module System#

# Every .py file is a module
# __name__ is "__main__" when run directly, module name when imported

# my_module.py
def helper():
 return "I'm a helper"

_private_var = "not exported by convention"

if __name__ == "__main__":
 # Only runs when executed directly, not when imported
 print(helper())

Package Structure#

my_package/
├── __init__.py # Makes it a package, runs on import
├── core.py
├── utils/
│ ├── __init__.py
│ ├── strings.py
│ └── numbers.py
└── _internal.py # Convention: private module

# __init__.py — control what's exported
from .core import MainClass, main_function
from .utils.strings import clean_text

__all__ = ["MainClass", "main_function", "clean_text"]
# Controls what `from my_package import *` exports

Import Mechanics#

# Python searches for modules in this order:
# 1. sys.modules cache (already imported)
# 2. Built-in modules
# 3. sys.path (current dir, PYTHONPATH, site-packages)

import sys
print(sys.path) # shows search order

# Lazy imports (defer heavy imports)
def process_image(path):
 from PIL import Image # only imported when function is called
 return Image.open(path)

# Conditional imports
try:
 import ujson as json # faster JSON library
except ImportError:
 import json # fallback to stdlib

Hypothetical Use Cases#

Use Case: CLI Argument Parser#

def parse_args(argv: list[str]) -> dict:
 """Simple CLI argument parser using pattern matching."""
 args = {"verbose": False, "output": "stdout", "files": []}
 
 i = 0
 while i < len(argv):
 match argv[i]:
 case "-v" | "--verbose":
 args["verbose"] = True
 case "-o" | "--output":
 i += 1
 args["output"] = argv[i]
 case arg if not arg.startswith("-"):
 args["files"].append(arg)
 case unknown:
 raise ValueError(f"Unknown argument: {unknown}")
 i += 1
 
 return args

Use Case: Plugin System#

import importlib
from pathlib import Path

def load_plugins(plugin_dir: str) -> dict:
 """Dynamically load all plugins from a directory."""
 plugins = {}
 
 for path in Path(plugin_dir).glob("*.py"):
 if path.name.startswith("_"):
 continue
 
 module_name = path.stem
 spec = importlib.util.spec_from_file_location(module_name, path)
 module = importlib.util.module_from_spec(spec)
 spec.loader.exec_module(module)
 
 # Each plugin must have a register() function
 if hasattr(module, "register"):
 plugin_info = module.register()
 plugins[plugin_info["name"]] = module
 
 return plugins

Key Takeaways#

Pattern matching (3.10+) replaces complex if/elif chains for structural data
Comprehensions are idiomatic Python — prefer them over manual loops for transforms
Closures and factories create configured functions without classes
Positional-only and keyword-only parameters make APIs clearer
__init__.py controls package exports — use __all__ for explicit public API
Lazy imports improve startup time for heavy dependencies

Rebase Workflows

Mon, 01 Jan 0001 00:00:00 +0000

Rebase Workflows#

Rebase is Git’s most powerful history-editing tool. It replays commits onto a new base, creating a linear history that’s easier to read and bisect. But with great power comes great responsibility — rewriting shared history can cause serious collaboration problems.

Rebase vs Merge#

Both integrate changes from one branch into another, but they produce different histories:

gitGraph
 commit id: "A"
 commit id: "B"
 branch feature
 commit id: "C"
 commit id: "D"
 checkout main
 commit id: "E"
 checkout feature
 commit id: "F"

After Merge (`git merge feature`)#

gitGraph
 commit id: "A"
 commit id: "B"
 branch feature
 commit id: "C"
 commit id: "D"
 checkout main
 commit id: "E"
 checkout feature
 commit id: "F"
 checkout main
 merge feature id: "M"

After Rebase (`git checkout feature && git rebase main`)#

gitGraph
 commit id: "A"
 commit id: "B"
 commit id: "E"
 commit id: "C'"
 commit id: "D'"
 commit id: "F'"

Aspect	Merge	Rebase
History shape	Non-linear (preserves branches)	Linear (straight line)
Original commits	Preserved exactly	Replaced with new commits (new SHAs)
Merge commits	Creates one	None
Conflict resolution	Once (in merge commit)	Per-commit (during replay)
Traceability	Full branch history visible	Branch topology lost
Safety for shared branches	Safe	Dangerous — rewrites history

The Golden Rule of Rebase#

Never rebase commits that have been pushed to a shared branch.

Service Workers

Mon, 01 Jan 0001 00:00:00 +0000

What Is a Service Worker#

A service worker is a JavaScript file that the browser runs in the background, separate from the web page. It acts as a programmable network proxy — intercepting every network request the page makes and deciding how to respond. This is the foundational technology that enables offline support, background sync, and push notifications.

Key properties:

Property	Detail
Thread	Runs on its own thread — no access to the DOM
Scope	Controls all pages under its registered scope
HTTPS only	Requires a secure origin (localhost exempt for dev)
Event-driven	Wakes up in response to events, then terminates when idle
No persistent state	Cannot rely on global variables between events — use IndexedDB or the Cache API
Async only	Synchronous APIs (e.g. `localStorage`, `XHR`) are not available

Registration#

SQL Basics

Mon, 01 Jan 0001 00:00:00 +0000

What Is SQL?#

SQL (Structured Query Language) is the standard language for interacting with relational databases. Despite variations between vendors, core SQL is remarkably portable. It’s declarative — you describe what you want, not how to get it.

SQL Statement Categories#

Category	Acronym	Statements	Purpose
Data Query Language	DQL	`SELECT`	Read data
Data Manipulation Language	DML	`INSERT`, `UPDATE`, `DELETE`	Modify data
Data Definition Language	DDL	`CREATE`, `ALTER`, `DROP`, `TRUNCATE`	Define structure
Data Control Language	DCL	`GRANT`, `REVOKE`	Manage permissions
Transaction Control	TCL	`BEGIN`, `COMMIT`, `ROLLBACK`	Manage transactions

SELECT — Reading Data#

Basic Syntax#

SELECT columns
FROM table
WHERE condition
ORDER BY column [ASC|DESC]
LIMIT count;

Selecting Columns#

-- All columns
SELECT * FROM employees;

-- Specific columns
SELECT name, salary, department FROM employees;

-- With aliases
SELECT name AS employee_name, salary AS annual_salary FROM employees;

-- Computed columns
SELECT name, salary, salary * 12 AS annual_total FROM employees;

-- Distinct values (remove duplicates)
SELECT DISTINCT department FROM employees;

Filtering with WHERE#

-- Comparison operators
SELECT * FROM employees WHERE salary > 80000;
SELECT * FROM employees WHERE department = 'Engineering';
SELECT * FROM employees WHERE hire_date >= '2021-01-01';

-- Multiple conditions
SELECT * FROM employees WHERE department = 'Engineering' AND salary > 90000;
SELECT * FROM employees WHERE department = 'Sales' OR department = 'Marketing';
SELECT * FROM employees WHERE NOT department = 'Sales';

-- IN (shorthand for multiple OR)
SELECT * FROM employees WHERE department IN ('Engineering', 'Marketing', 'Sales');

-- BETWEEN (inclusive range)
SELECT * FROM employees WHERE salary BETWEEN 70000 AND 100000;

-- LIKE (pattern matching)
SELECT * FROM employees WHERE name LIKE 'A%'; -- Starts with A
SELECT * FROM employees WHERE name LIKE '%son'; -- Ends with son
SELECT * FROM employees WHERE name LIKE '%ar%'; -- Contains ar
SELECT * FROM employees WHERE name LIKE '_ob%'; -- Second and third chars are 'ob'

-- NULL checks (never use = NULL)
SELECT * FROM employees WHERE manager_id IS NULL;
SELECT * FROM employees WHERE manager_id IS NOT NULL;

Sorting with ORDER BY#

-- Ascending (default)
SELECT * FROM employees ORDER BY salary;

-- Descending
SELECT * FROM employees ORDER BY salary DESC;

-- Multiple columns
SELECT * FROM employees ORDER BY department ASC, salary DESC;

-- By column position (less readable)
SELECT name, salary FROM employees ORDER BY 2 DESC;

Limiting Results#

-- First N rows
SELECT * FROM employees ORDER BY salary DESC LIMIT 10;

-- Pagination (skip 20, take 10)
SELECT * FROM employees ORDER BY id LIMIT 10 OFFSET 20;

-- SQL Server uses TOP
SELECT TOP 10 * FROM employees ORDER BY salary DESC;

INSERT — Adding Data#

-- Single row (specify columns)
INSERT INTO employees (name, department, salary, hire_date)
VALUES ('Eve Brown', 'Engineering', 92000, '2023-06-01');

-- Single row (all columns in order — fragile, avoid)
INSERT INTO employees
VALUES (5, 'Eve Brown', 'Engineering', 92000, '2023-06-01');

-- Multiple rows
INSERT INTO employees (name, department, salary, hire_date)
VALUES
 ('Frank Lee', 'Marketing', 75000, '2023-07-15'),
 ('Grace Kim', 'Sales', 71000, '2023-08-01'),
 ('Henry Park', 'Engineering', 98000, '2023-09-10');

-- Insert from SELECT
INSERT INTO archive_employees (name, department, salary)
SELECT name, department, salary
FROM employees
WHERE hire_date < '2020-01-01';

UPDATE — Modifying Data#

-- Update specific rows (always include WHERE!)
UPDATE employees
SET salary = 100000
WHERE id = 3;

-- Update multiple columns
UPDATE employees
SET salary = 95000, department = 'Engineering'
WHERE name = 'Bob Smith';

-- Computed update
UPDATE employees
SET salary = salary * 1.10
WHERE department = 'Engineering';

-- Update from another table
UPDATE employees e
SET department = d.new_department
FROM department_changes d
WHERE e.id = d.employee_id;

⚠️ WARNING: An UPDATE without WHERE modifies ALL rows in the table. Always double-check before executing.

The Transformer Architecture

Mon, 01 Jan 0001 00:00:00 +0000

The Transformer is the architecture behind virtually all modern language models (GPT, BERT, Claude, Llama, T5). Introduced in “Attention Is All You Need” (Vaswani et al., 2017), it replaced recurrent networks by processing all positions in parallel through a mechanism called self-attention.

This guide breaks down every component — from the attention mechanism to the full architecture variants.

Prerequisites#

Neural network basics (forward/backward pass, loss functions)
Matrix multiplication
Basic understanding of sequence processing (what a “token” is)

The Core Idea: Attention#

The fundamental question attention answers: “When processing token X, how much should I look at every other token in the sequence?”

Time & Ordering

Mon, 01 Jan 0001 00:00:00 +0000

Time & Ordering#

In a single-node system, “what happened first?” has a straightforward answer — the CPU’s instruction counter provides a total order. In a distributed system, this question becomes one of the hardest problems in computer science. There is no global clock, no shared counter, and no way to determine the exact order of events across nodes without communication.

This section covers the tools we use to establish ordering in distributed systems — from physical clocks (and why they fail) to logical clocks that capture causality without relying on time at all.

Types & Variables

Mon, 01 Jan 0001 00:00:00 +0000

Types & Variables#

Go is statically typed with type inference. Every variable has a fixed type determined at compile time, but you rarely need to write types explicitly thanks to the := short declaration.

Declaring Variables#

Long Form#

var name string = "Alice"
var age int = 30
var active bool = true

Short Declaration (Most Common)#

name := "Alice" // type inferred as string
age := 30 // type inferred as int
active := true // type inferred as bool

:= declares and assigns in one step. It only works inside functions (not at package level).

Types & Variables

Mon, 01 Jan 0001 00:00:00 +0000

Types & Variables#

Rust is statically typed with powerful type inference. The compiler deduces most types, but you must be explicit in function signatures. Every value has exactly one owner, a fixed type, and a fixed size known at compile time (or stored behind a pointer).

Scalar Types#

Type	Size	Range	Example
`i8`	1 byte	-128 to 127	`let x: i8 = -42;`
`i16`	2 bytes	-32,768 to 32,767
`i32`	4 bytes	-2³¹ to 2³¹-1	Default integer type
`i64`	8 bytes	-2⁶³ to 2⁶³-1
`i128`	16 bytes	Very large
`isize`	Pointer-sized	Platform-dependent	Index into collections
`u8`	1 byte	0 to 255	Bytes, ASCII
`u16`	2 bytes	0 to 65,535
`u32`	4 bytes	0 to 2³²-1
`u64`	8 bytes	0 to 2⁶⁴-1
`u128`	16 bytes	Very large
`usize`	Pointer-sized	Platform-dependent	Array indices, sizes
`f32`	4 bytes	~7 digits
`f64`	8 bytes	~15 digits	Default float type
`bool`	1 byte	`true` / `false`
`char`	4 bytes	Unicode scalar value	`'a'`, `'🦀'`

Integer Literals#

let decimal = 98_222; // underscores for readability
let hex = 0xff;
let octal = 0o77;
let binary = 0b1111_0000;
let byte_literal = b'A'; // u8 only

Integer Overflow#

In debug mode, overflow panics. In release mode, it wraps (two’s complement). To be explicit:

Unit Testing

Mon, 01 Jan 0001 00:00:00 +0000

Unit Testing#

Unit tests verify the smallest pieces of your code in isolation. They’re fast, focused, and form the foundation of any test suite. This section covers patterns and practices for writing effective unit tests.

The AAA Pattern#

Every well-structured unit test follows three phases:

graph LR
 A[Arrange<br/>Set up inputs & dependencies] --> B[Act<br/>Execute the code under test]
 B --> C[Assert<br/>Verify the outcome]

Python Example#

def test_calculate_total_with_discount():
 # Arrange
 items = [
 {"name": "Widget", "price": 25.00, "quantity": 2},
 {"name": "Gadget", "price": 15.00, "quantity": 1},
 ]
 discount_percentage = 10

 # Act
 total = calculate_total(items, discount=discount_percentage)

 # Assert
 assert total == 58.50 # (50 + 15) * 0.90

JavaScript Example#

test('calculate total applies discount correctly', () => {
 // Arrange
 const items = [
 { name: 'Widget', price: 25.00, quantity: 2 },
 { name: 'Gadget', price: 15.00, quantity: 1 },
 ];
 const discount = 10;

 // Act
 const total = calculateTotal(items, { discount });

 // Assert
 expect(total).toBe(58.50);
});

Rules for AAA#

Phase	Guidelines
Arrange	Set up only what’s needed for this test. Use helpers/factories for complex setup.
Act	Exactly one action. If you need multiple actions, it’s probably multiple tests.
Assert	Check one logical concept (may be multiple `assert` statements for one concept).

Test Naming#

Good test names describe the scenario and expected outcome:

Variables, Quoting, and Expansion

Mon, 01 Jan 0001 00:00:00 +0000

Variables#

Shell variables are untyped — everything is a string. The shell interprets context to decide if something is a number or text.

Assignment Rules#

# Correct — NO spaces around =
name="Alice"
count=42
path="/usr/local/bin"

# WRONG — spaces cause errors
name = "Alice" # Error: "name: command not found"

Variable Usage#

name="world"
echo "Hello, $name" # Hello, world
echo "Hello, ${name}" # Same — braces for clarity
echo "${name}_backup" # world_backup (braces needed here)
echo "$name_backup" # Empty! (looks for variable name_backup)

Variable Types#

# Regular variable (local to current shell/script)
greeting="Hello"

# Environment variable (inherited by child processes)
export API_KEY="abc123"

# Readonly (constant)
readonly MAX_RETRIES=3
MAX_RETRIES=5 # Error: readonly variable

# Unset
unset greeting # removes the variable

Special Variables#

Variable	Meaning
`$0`	Script name
`$1`-`$9`	Positional parameters
`$#`	Number of arguments
`$@`	All arguments (as separate words)
`$*`	All arguments (as single string)
`$?`	Exit code of last command
`$$`	Current process PID
`$!`	PID of last background process

Quoting#

Quoting is the #1 source of shell scripting bugs. Understanding it prevents most issues.

Workloads

Mon, 01 Jan 0001 00:00:00 +0000

Workloads#

Higher-level abstractions that manage pods — ensuring the right number are running, handling updates gracefully, and supporting both stateless and stateful applications.

Workload Resources Overview#

Resource	Use Case	Pod Identity	Scaling
Deployment	Stateless apps (APIs, web servers)	Interchangeable	Horizontal (replicas)
ReplicaSet	Ensures N identical pods run (managed by Deployment)	Interchangeable	Horizontal
DaemonSet	One pod per node (agents, log collectors)	Per-node	Tied to node count
StatefulSet	Stateful apps (databases, queues)	Stable, ordered	Horizontal (ordered)
Job	Run-to-completion tasks (batch, migrations)	Ephemeral	Parallelism setting
CronJob	Scheduled Jobs	Ephemeral	Per schedule

Deployments#

The most common workload resource. A Deployment manages ReplicaSets, which manage Pods.

Arrays and Strings

Mon, 01 Jan 0001 00:00:00 +0000

Arrays#

PHP arrays are the most versatile data structure in the language. They’re actually ordered hash maps — they can act as lists, dictionaries, stacks, queues, sets, and more.

Creating Arrays#

<?php
// Short syntax (preferred)
$fruits = ['apple', 'banana', 'cherry'];

// Associative array (key => value)
$person = [
 'name' => 'Alice',
 'age' => 30,
 'city' => 'Barcelona',
];

// Mixed keys (possible but confusing — avoid)
$mixed = [0 => 'zero', 'key' => 'value', 1 => 'one'];

// Nested arrays
$matrix = [
 [1, 2, 3],
 [4, 5, 6],
 [7, 8, 9],
];

Accessing and Modifying#

<?php
$fruits[0]; // 'apple'
$person['name']; // 'Alice'
$matrix[1][2]; // 6

// Append
$fruits[] = 'date'; // adds at next integer key

// Modify
$person['age'] = 31;

// Remove
unset($fruits[1]); // removes 'banana', does NOT reindex!
// After unset: [0 => 'apple', 2 => 'cherry', 3 => 'date']

// Reindex after unset
$fruits = array_values($fruits);

Array Destructuring#

<?php
// List assignment (positional)
[$first, $second] = ['hello', 'world'];

// Skip elements
[, , $third] = [1, 2, 3]; // $third = 3

// Associative destructuring
['name' => $name, 'age' => $age] = $person;

// In foreach
$points = [['x' => 1, 'y' => 2], ['x' => 3, 'y' => 4]];
foreach ($points as ['x' => $x, 'y' => $y]) {
 echo "($x, $y) ";
}

Essential Array Functions#

Searching and Filtering#

<?php
in_array('apple', $fruits); // true (loose comparison)
in_array('apple', $fruits, true); // true (strict — preferred)
array_key_exists('name', $person); // true
array_search('cherry', $fruits); // returns key or false

// Filter with callback
$adults = array_filter($users, fn($u) => $u['age'] >= 18);

// Filter without callback removes falsy values
$clean = array_filter([0, '', null, 'hello', false, 42]); // ['hello', 42]

Transforming#

<?php
// Map: transform each element
$upper = array_map(fn($s) => strtoupper($s), $fruits);

// Map with keys (use array_map with null + multiple arrays, or foreach)
$result = [];
foreach ($data as $key => $value) {
 $result[$key] = transform($value);
}

// Reduce: fold into a single value
$total = array_reduce($prices, fn($carry, $item) => $carry + $item, 0);

// Column: extract a single column from nested arrays
$names = array_column($users, 'name');
$indexed = array_column($users, null, 'id'); // re-index by 'id' field

Sorting#

<?php
sort($arr); // by value, reindex
rsort($arr); // reverse, reindex
asort($arr); // by value, preserve keys
arsort($arr); // reverse by value, preserve keys
ksort($arr); // by key
krsort($arr); // reverse by key

// Custom sort
usort($users, fn($a, $b) => $a['age'] <=> $b['age']);

// Sort by multiple criteria
usort($users, function ($a, $b) {
 return $a['city'] <=> $b['city']
 ?: $a['name'] <=> $b['name'];
});

Combining and Splitting#

<?php
$merged = array_merge($a, $b); // reindexes numeric keys
$combined = $a + $b; // preserves keys, left wins on conflict
$replaced = array_replace($a, $b); // right wins on conflict

$chunk = array_chunk($large, 100); // split into groups of 100
$slice = array_slice($arr, 2, 5); // 5 elements starting at index 2

$keys = array_keys($person); // ['name', 'age', 'city']
$values = array_values($person); // ['Alice', 30, 'Barcelona']
$flipped = array_flip($arr); // swap keys and values
$unique = array_unique($arr); // remove duplicates

The Spread Operator in Arrays (PHP 7.4+)#

<?php
$first = [1, 2, 3];
$second = [4, 5, 6];
$all = [...$first, ...$second]; // [1, 2, 3, 4, 5, 6]

// Works with named keys (PHP 8.1+)
$defaults = ['color' => 'blue', 'size' => 'M'];
$override = ['color' => 'red'];
$config = [...$defaults, ...$override]; // ['color' => 'red', 'size' => 'M']

Strings#

String Functions (Most Common)#

<?php
strlen($str); // Length (bytes, not characters!)
mb_strlen($str); // Length (multibyte-safe — use this for UTF-8)
strtolower($str); // Lowercase
strtoupper($str); // Uppercase
trim($str); // Strip whitespace from both ends
ltrim($str); rtrim($str); // Left/right trim

str_contains($hay, $needle); // PHP 8.0+ (previously: strpos !== false)
str_starts_with($str, 'http'); // PHP 8.0+
str_ends_with($str, '.php'); // PHP 8.0+

substr($str, 0, 5); // First 5 characters
str_replace('old', 'new', $str); // Replace all occurrences
str_pad($str, 10, '0', STR_PAD_LEFT); // Left-pad with zeros

explode(',', $csv); // Split string → array
implode(', ', $arr); // Join array → string

sprintf('Hello, %s! You are %d.', $name, $age); // Formatted string

String Interpolation#

<?php
$name = "World";

// Simple variable
echo "Hello, $name!";

// Complex expressions need curly braces
echo "Item: {$items[0]->name}";
echo "Result: {$obj->getLabel()}"; // Method calls need braces

// Array access in strings
echo "Value: {$config['key']}";

Regular Expressions (PCRE)#

<?php
// Match
preg_match('/^[a-z]+$/i', $input, $matches);

// Match all
preg_match_all('/\d+/', $text, $matches);

// Replace
$clean = preg_replace('/\s+/', ' ', $text);

// Replace with callback
$result = preg_replace_callback(
 '/\{(\w+)\}/',
 fn($m) => $replacements[$m[1]] ?? $m[0],
 $template
);

// Split
$words = preg_split('/[\s,;]+/', $text);

Multibyte Strings (UTF-8)#

PHP’s standard string functions are byte-oriented. For UTF-8 text, always use mb_* functions:

Authentication & Authorization

Mon, 01 Jan 0001 00:00:00 +0000

Authentication & Authorization#

Authentication (AuthN) verifies who you are. Authorization (AuthZ) determines what you can do. Every API needs both. This section covers the mechanisms from simple API keys to full OAuth2 flows.

Authentication vs Authorization#

Aspect	Authentication (AuthN)	Authorization (AuthZ)
Question	“Who is this?”	“Can they do this?”
Happens	First	After authentication
Mechanism	Credentials, tokens	Permissions, roles, policies
Failure code	401 Unauthorized	403 Forbidden

sequenceDiagram
 participant Client
 participant API
 participant AuthServer

 Client->>API: Request with credentials
 API->>AuthServer: Validate credentials (AuthN)
 AuthServer-->>API: Identity confirmed
 API->>API: Check permissions (AuthZ)
 API-->>Client: 200 OK or 403 Forbidden

API Keys#

The simplest authentication mechanism. A long, random string shared between client and server.

Caching Strategies

Mon, 01 Jan 0001 00:00:00 +0000

The Cache API#

The Cache API provides a storage mechanism for request/response pairs, designed specifically for service workers. Unlike the browser’s HTTP cache, you have full programmatic control over what goes in and what comes out.

// Open (or create) a named cache
const cache = await caches.open('my-cache-v1');

// Store a response
await cache.put(request, response);

// Store by fetching the URL
await cache.add('/styles/main.css');

// Store multiple URLs at once
await cache.addAll(['/index.html', '/app.js', '/styles.css']);

// Retrieve a cached response
const response = await caches.match(request);

// Delete an entry
await cache.delete(request);

// List all keys
const keys = await cache.keys();

Important: cache.add() and cache.addAll() will reject if any response is not a successful (2xx) HTTP response. A single 404 in the list fails the entire batch.

CD & Deployment Strategies

Mon, 01 Jan 0001 00:00:00 +0000

CD & Deployment Strategies#

Continuous Delivery (CD) ensures that code is always in a deployable state. Continuous Deployment takes this further by automatically releasing every change that passes the pipeline. The deployment strategy you choose determines how risk is managed during releases.

Continuous Delivery vs Continuous Deployment#

Aspect	Continuous Delivery	Continuous Deployment
Definition	Every commit can be deployed to production	Every commit is deployed to production
Human gate	Manual approval before prod deploy	No manual gate — fully automated
Prerequisite	Comprehensive automated testing	Extreme confidence in test suite + monitoring
Rollback	Manual or semi-automated	Automated rollback on failure
Adoption	Widespread	Mostly tech-forward companies

graph LR
 A[Commit] --> B[Build & Test]
 B --> C[Deploy to Staging]
 C --> D{Manual Approval?}
 D -->|Continuous Delivery| E[Deploy to Prod]
 D -->|Continuous Deployment| E
 
 style D fill:#fff3cd

Deployment Strategies#

Blue-Green Deployment#

Maintain two identical production environments. Route traffic entirely to one (blue) while deploying to the other (green). Switch when ready.

Clean Architecture

Mon, 01 Jan 0001 00:00:00 +0000

Clean Architecture#

Clean Architecture, introduced by Robert C. Martin, organizes code so that business rules are independent of frameworks, databases, and delivery mechanisms. The core idea is simple: dependencies point inward, and the inner layers know nothing about the outer layers.

The Dependency Rule#

Source code dependencies must point only inward, toward higher-level policies.

Nothing in an inner circle can know anything about something in an outer circle. The name of something declared in an outer circle must not be mentioned by code in an inner circle — functions, classes, variables, or any named software entity.

Collaboration & Teamwork

Mon, 01 Jan 0001 00:00:00 +0000

Collaboration & Teamwork#

Software is built by teams, not individuals. The most impactful engineers are not lone geniuses — they are multipliers who make everyone around them more effective. Collaboration is the skill that turns a group of engineers into a team.

Working in Cross-Functional Teams#

Modern product development involves engineers, designers, product managers, QA, data analysts, and sometimes legal, security, or compliance. Each role brings different priorities and vocabulary.

Collections and Enumerations

Mon, 01 Jan 0001 00:00:00 +0000

Collections and Enumerations#

Swift provides three primary collection types — Array, Set, and Dictionary — alongside enumerations that are far more powerful than their counterparts in most languages.

Arrays#

Arrays are ordered collections of values of the same type:

// Type-inferred creation
var temperatures = [72.0, 68.5, 74.2, 69.8]

// Explicit type annotation
var names: [String] = []

// Array with repeated values
let zeros = Array(repeating: 0, count: 5) // [0, 0, 0, 0, 0]

Array Operations#

var fruits = ["Apple", "Banana", "Cherry"]

// Access
let first = fruits[0] // "Apple"
let last = fruits.last // Optional("Cherry")

// Modification
fruits.append("Date") // Add to end
fruits.insert("Avocado", at: 1) // Insert at index
fruits.remove(at: 0) // Remove at index
fruits.removeLast() // Remove last element

// Querying
let count = fruits.count // Number of elements
let empty = fruits.isEmpty // false
let hasCherry = fruits.contains("Cherry") // true

// Transformation
let uppercased = fruits.map { $0.uppercased() }
let short = fruits.filter { $0.count < 6 }
let sorted = fruits.sorted() // New sorted array (non-mutating)
fruits.sort() // In-place sort (mutating)

Array Slicing#

let numbers = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

let firstThree = numbers[0..<3] // [0, 1, 2] — ArraySlice
let lastThree = numbers.suffix(3) // [7, 8, 9]
let middle = numbers[3...6] // [3, 4, 5, 6]

// Convert slice back to Array
let sliceArray = Array(numbers[2..<5]) // [2, 3, 4]

Sets#

Sets are unordered collections of unique values. They provide O(1) lookup and support mathematical set operations:

Collections and Sequences

Mon, 01 Jan 0001 00:00:00 +0000

Collections and Sequences#

Kotlin’s collections API is one of the language’s strongest features. It provides a rich set of operations for transforming, filtering, and aggregating data, while making a clear distinction between read-only and mutable collections. Sequences extend this model with lazy evaluation for processing large datasets efficiently.

Collection Hierarchy#

Kotlin separates read-only interfaces from mutable ones at the type level:

Read-Only	Mutable	Factory function
`List<T>`	`MutableList<T>`	`listOf()` / `mutableListOf()`
`Set<T>`	`MutableSet<T>`	`setOf()` / `mutableSetOf()`
`Map<K, V>`	`MutableMap<K, V>`	`mapOf()` / `mutableMapOf()`

Read-only collections have no add, remove, or put methods. This is enforced at the interface level, not by creating defensive copies:

Compute

Mon, 01 Jan 0001 00:00:00 +0000

Compute Options Overview#

AWS offers compute at every abstraction level — from bare metal to fully managed serverless:

flowchart TD
 subgraph Control["More Control"]
 BM["Bare Metal<br>(EC2 .metal)"]
 EC2["EC2 Instances<br>(Virtual Machines)"]
 end
 
 subgraph Managed["Managed"]
 ECS["ECS / EKS<br>(Containers)"]
 Batch["AWS Batch<br>(Job scheduling)"]
 end
 
 subgraph Serverless["Serverless"]
 Lambda["Lambda<br>(Functions)"]
 Fargate["Fargate<br>(Serverless containers)"]
 end
 
 Control -.->|"Less ops overhead →"| Managed
 Managed -.->|"Less ops overhead →"| Serverless

EC2 (Elastic Compute Cloud)#

Virtual machines in the cloud. You choose the OS, instance type, storage, and networking.

Consistency Models

Mon, 01 Jan 0001 00:00:00 +0000

Consistency Models#

When data is replicated across multiple nodes, different replicas can disagree about the current state. A consistency model defines the rules about what values a read operation can return and when writes become visible. Choosing the right consistency model is one of the most consequential design decisions in a distributed system — it determines the tradeoff between correctness, performance, and availability.

This section covers the hierarchy of consistency models from strongest to weakest, the guarantees each provides, and when to use them.

Control Flow

Mon, 01 Jan 0001 00:00:00 +0000

Control Flow#

Go has minimal control flow constructs: if, for, switch, and defer. There is no while, no do-while, no ternary operator, and no try/catch. This simplicity is intentional.

If/Else#

if x > 0 {
 fmt.Println("positive")
} else if x == 0 {
 fmt.Println("zero")
} else {
 fmt.Println("negative")
}

If with Init Statement#

Go allows a short statement before the condition — scoped to the if/else block:

if err := doSomething(); err != nil {
 return fmt.Errorf("failed: %w", err)
}
// err is not accessible here

This is the idiomatic error-checking pattern. You will write it hundreds of times.

Control Structures

Mon, 01 Jan 0001 00:00:00 +0000

Conditionals: if/elif/else#

if [[ "$status" == "active" ]]; then
 echo "Service is running"
elif [[ "$status" == "stopped" ]]; then
 echo "Service is stopped"
else
 echo "Unknown status: $status"
fi

The semicolon before then replaces a newline — these are equivalent:

if [[ condition ]]; then command; fi

if [[ condition ]]
then
 command
fi

Test Commands: `[[ ]]` vs `[ ]`#

`[[ ]]` — Bash Extended Test (Preferred)#

Category	Operator	Meaning
String	`==`	Equal
String	`!=`	Not equal
String	`=~`	Regex match
String	`-z "$s"`	String is empty
String	`-n "$s"`	String is non-empty
Numeric	`-eq`, `-ne`	Equal, not equal
Numeric	`-lt`, `-le`	Less than, less or equal
Numeric	`-gt`, `-ge`	Greater than, greater or equal
File	`-f`	Is regular file
File	`-d`	Is directory
File	`-e`	Exists (any type)
File	`-r`, `-w`, `-x`	Readable, writable, executable
File	`-s`	Exists and non-empty
Logic	`&&`	AND
Logic	`\|\|`	OR
Logic	`!`	NOT

`[ ]` vs `[[ ]]` Comparison#

Feature	`[ ]` (POSIX)	`[[ ]]` (Bash)
Word splitting on vars	Yes (must quote!)	No
Glob expansion	Yes	No
Regex (`=~`)	❌	✅
Pattern matching (`==` with `*`)	❌	✅
`&&`, `\|\|` inside	❌	✅
Portability	All shells	Bash/Zsh only

Rule: Use [[ ]] in bash scripts. Use [ ] only for POSIX portability.

Cross-Site Scripting (XSS)

Mon, 01 Jan 0001 00:00:00 +0000

What Is XSS?#

Cross-Site Scripting (XSS) occurs when an attacker injects malicious scripts into web pages viewed by other users. The victim’s browser cannot distinguish between legitimate scripts from the application and injected scripts — it executes both with the same origin and permissions.

XSS is the most prevalent web vulnerability. It enables session hijacking, credential theft, defacement, phishing, and client-side malware distribution.

Types of XSS#

Reflected XSS#

The malicious script is part of the HTTP request and reflected back in the response without being stored. The attacker crafts a URL containing the payload and tricks the victim into clicking it.

ES6+ Features

Mon, 01 Jan 0001 00:00:00 +0000

The Modern JavaScript Revolution#

ES6 (ECMAScript 2015) was the biggest single update to JavaScript. Subsequent yearly releases (ES2016–ES2024) added incremental improvements. This file covers the features that fundamentally changed how JavaScript is written.

Destructuring#

Destructuring extracts values from objects and arrays into distinct variables using a pattern that mirrors the data structure.

Object Destructuring#

const user = { name: "Alice", age: 30, city: "Barcelona", role: "admin" };

// Basic
const { name, age } = user;

// Rename
const { name: userName, age: userAge } = user;

// Default values
const { name, country = "Unknown" } = user; // country = "Unknown"

// Nested
const response = {
 data: {
 user: { id: 1, profile: { avatar: "url" } }
 }
};
const { data: { user: { profile: { avatar } } } } = response;

// Rest (collect remaining)
const { name, ...rest } = user; // rest = { age: 30, city: "Barcelona", role: "admin" }

// Computed property names
const key = "name";
const { [key]: value } = user; // value = "Alice"

Array Destructuring#

const coords = [41.3851, 2.1734, 12];

// Basic
const [lat, lng] = coords;

// Skip elements
const [, , altitude] = coords;

// Default values
const [x, y, z, w = 0] = coords;

// Rest
const [first, ...remaining] = [1, 2, 3, 4, 5]; // remaining = [2, 3, 4, 5]

// Swap variables (no temp needed)
let a = 1, b = 2;
[a, b] = [b, a]; // a = 2, b = 1

Function Parameter Destructuring#

// Object parameters with defaults
function createUser({ name, age, role = "user", active = true } = {}) {
 return { name, age, role, active, createdAt: new Date() };
}

createUser({ name: "Alice", age: 30 });
createUser(); // uses all defaults (the = {} handles undefined argument)

// Array parameters
function head([first, ...rest]) {
 return { first, rest };
}

head([1, 2, 3]); // { first: 1, rest: [2, 3] }

Spread and Rest Operators#

The ... syntax serves two purposes depending on context:

File Systems

Mon, 01 Jan 0001 00:00:00 +0000

What is a File System?#

A file system organizes data on storage devices into a hierarchy of files and directories. It manages how data is stored, retrieved, and updated on disk.

flowchart TD
 Root["/"]
 Root --> etc["/etc (config)"]
 Root --> home["/home (users)"]
 Root --> var["/var (variable data)"]
 Root --> usr["/usr (programs)"]
 Root --> tmp["/tmp (temporary)"]
 
 home --> alice["/home/alice"]
 var --> log["/var/log"]

Linux Directory Structure#

Path	Purpose
`/`	Root — everything starts here
`/bin`	Essential binaries (ls, cp, cat)
`/etc`	System configuration files
`/home`	User home directories
`/var`	Variable data (logs, databases, mail)
`/tmp`	Temporary files (cleared on reboot)
`/usr`	User programs and libraries
`/opt`	Optional/third-party software
`/proc`	Virtual filesystem — process info
`/dev`	Device files (disks, terminals)
`/mnt`, `/media`	Mount points for external storage

File System Types#

FS	OS	Features	Use Case
ext4	Linux	Journaling, mature, reliable	Default Linux FS
XFS	Linux	High performance, large files	Enterprise, databases
Btrfs	Linux	Snapshots, compression, RAID	Advanced Linux
NTFS	Windows	ACLs, encryption, journaling	Windows default
APFS	macOS	Snapshots, encryption, SSD-optimized	macOS/iOS
FAT32	Cross-platform	Simple, universal	USB drives, boot partitions
ZFS	Linux/BSD	Checksums, snapshots, RAID-Z	Storage servers

Key Concepts#

Inodes#

Every file has an inode — metadata about the file (NOT the filename):

Functions & Graphs

Mon, 01 Jan 0001 00:00:00 +0000

Functions & Graphs#

A function is a rule that assigns exactly one output to each input. Functions are the abstraction behind every algorithm’s behaviour, every complexity analysis, and every mathematical model in engineering.

What Is a Function?#

Formally, a function f: A → B is a mapping from a set A (the domain) to a set B (the codomain) where every element of A maps to exactly one element of B.

Functions, Scope, and Recursion

Mon, 01 Jan 0001 00:00:00 +0000

Functions#

A function encapsulates a unit of work — it takes inputs, performs computation, and produces output. Functions are the primary tool for managing complexity in software.

Anatomy of a Function#

def calculate_shipping(weight_kg: float, distance_km: float, express: bool = False) -> float:
 """
 Calculate shipping cost based on weight, distance, and speed.
 
 Args:
 weight_kg: Package weight in kilograms
 distance_km: Shipping distance in kilometers
 express: Whether to use express delivery
 
 Returns:
 Shipping cost in euros
 """
 base_rate = 2.50
 weight_rate = 0.75 * weight_kg
 distance_rate = 0.01 * distance_km
 
 cost = base_rate + weight_rate + distance_rate
 
 if express:
 cost *= 1.5
 
 return round(cost, 2)

Components:

IP Addressing

Mon, 01 Jan 0001 00:00:00 +0000

IPv4 Addresses#

A 32-bit number that uniquely identifies a device on a network, written as four decimal octets:

192.168.1.100
 │ │ │ │
 └────┴───┴──┴── Each octet: 0-255 (8 bits)

Binary: 11000000.10101000.00000001.01100100
Decimal: 192 . 168 . 1 . 100

Subnetting and CIDR#

CIDR Notation#

CIDR (Classless Inter-Domain Routing) notation: IP/prefix_length

The prefix identifies the network portion, the remaining bits identify the host:

10.0.1.50/24
├── Network: 10.0.1 (first 24 bits — fixed)
└── Host: .50 (last 8 bits — variable)

Network address: 10.0.1.0 (all host bits = 0)
Broadcast: 10.0.1.255 (all host bits = 1)
Usable hosts: 10.0.1.1 – 10.0.1.254 (254 hosts)

Subnet Mask#

The subnet mask marks which bits are network (1) and which are host (0):

Java: Object-Oriented Programming

Mon, 01 Jan 0001 00:00:00 +0000

Classes and Objects#

Anatomy of a Java Class#

public class Employee {
 // Fields (instance variables)
 private final String id;
 private String name;
 private double salary;
 private Department department;
 
 // Static field (class-level)
 private static int totalEmployees = 0;
 
 // Constructor
 public Employee(String id, String name, double salary) {
 this.id = id;
 this.name = name;
 this.salary = salary;
 totalEmployees++;
 }
 
 // Overloaded constructor (constructor chaining)
 public Employee(String id, String name) {
 this(id, name, 50000.0); // Delegates to main constructor
 }
 
 // Getters and setters (encapsulation)
 public String getName() { return name; }
 
 public void setSalary(double salary) {
 if (salary < 0) throw new IllegalArgumentException("Salary cannot be negative");
 this.salary = salary;
 }
 
 // Business method
 public double annualBonus() {
 return salary * 0.10;
 }
 
 // Static method
 public static int getTotalEmployees() {
 return totalEmployees;
 }
 
 // Object methods override
 @Override
 public String toString() {
 return "Employee{id='%s', name='%s', salary=%.2f}".formatted(id, name, salary);
 }
 
 @Override
 public boolean equals(Object o) {
 if (this == o) return true;
 if (!(o instanceof Employee other)) return false;
 return id.equals(other.id);
 }
 
 @Override
 public int hashCode() {
 return id.hashCode();
 }
}

Inheritance#

public abstract class Shape {
 private final String color;
 
 protected Shape(String color) {
 this.color = color;
 }
 
 // Abstract method — subclasses MUST implement
 public abstract double area();
 public abstract double perimeter();
 
 // Concrete method — inherited as-is
 public String describe() {
 return "%s %s (area=%.2f)".formatted(color, getClass().getSimpleName(), area());
 }
 
 // Final method — CANNOT be overridden
 public final String getColor() {
 return color;
 }
}

public class Circle extends Shape {
 private final double radius;
 
 public Circle(String color, double radius) {
 super(color); // Must call parent constructor
 this.radius = radius;
 }
 
 @Override
 public double area() {
 return Math.PI * radius * radius;
 }
 
 @Override
 public double perimeter() {
 return 2 * Math.PI * radius;
 }
}

public class Rectangle extends Shape {
 private final double width, height;
 
 public Rectangle(String color, double width, double height) {
 super(color);
 this.width = width;
 this.height = height;
 }
 
 @Override
 public double area() { return width * height; }
 
 @Override
 public double perimeter() { return 2 * (width + height); }
}

Interfaces#

// Interfaces define contracts (what, not how)
public interface Sortable<T> {
 int compareTo(T other);
 
 // Default method (Java 8+) — provides implementation
 default boolean isGreaterThan(T other) {
 return compareTo(other) > 0;
 }
 
 // Static method
 static <T extends Sortable<T>> T max(T a, T b) {
 return a.compareTo(b) >= 0 ? a : b;
 }
}

public interface Serializable {
 byte[] serialize();
 
 // Private method (Java 9+) — helper for default methods
 private String formatHeader() {
 return "v1:" + getClass().getName();
 }
 
 default byte[] serializeWithHeader() {
 return (formatHeader() + ":" + new String(serialize())).getBytes();
 }
}

// A class can implement multiple interfaces
public class Product implements Sortable<Product>, Serializable {
 private final String name;
 private final double price;
 
 @Override
 public int compareTo(Product other) {
 return Double.compare(this.price, other.price);
 }
 
 @Override
 public byte[] serialize() {
 return (name + ":" + price).getBytes();
 }
}

Interface vs Abstract Class#

Feature	Interface	Abstract Class
Multiple inheritance	Yes (implements many)	No (extends one)
Fields	Constants only (`static final`)	Any fields
Constructors	No	Yes
Access modifiers	`public` (default methods)	Any
Use when	Defining a capability/contract	Sharing code among related classes

Polymorphism#

// Runtime polymorphism — method dispatch based on actual type
List<Shape> shapes = List.of(
 new Circle("red", 5),
 new Rectangle("blue", 3, 4),
 new Circle("green", 2)
);

for (Shape shape : shapes) {
 // Calls the correct area() based on actual runtime type
 System.out.println(shape.describe());
}

// Covariant return types
class Animal {
 public Animal create() { return new Animal(); }
}

class Dog extends Animal {
 @Override
 public Dog create() { return new Dog(); } // More specific return type OK
}

Composition and Delegation#

// Prefer composition over inheritance
public class OrderService {
 private final OrderRepository repository;
 private final PaymentGateway paymentGateway;
 private final NotificationService notifications;
 
 // Dependencies injected via constructor
 public OrderService(
 OrderRepository repository,
 PaymentGateway paymentGateway,
 NotificationService notifications) {
 this.repository = repository;
 this.paymentGateway = paymentGateway;
 this.notifications = notifications;
 }
 
 public Order placeOrder(OrderRequest request) {
 Order order = Order.from(request);
 
 PaymentResult payment = paymentGateway.charge(
 request.paymentMethod(), order.total()
 );
 
 if (!payment.isSuccessful()) {
 throw new PaymentFailedException(payment.errorMessage());
 }
 
 order = order.withStatus(OrderStatus.CONFIRMED);
 repository.save(order);
 notifications.sendOrderConfirmation(order);
 
 return order;
 }
}

Enums — Type-Safe Constants#

public enum HttpStatus {
 OK(200, "OK"),
 CREATED(201, "Created"),
 BAD_REQUEST(400, "Bad Request"),
 UNAUTHORIZED(401, "Unauthorized"),
 NOT_FOUND(404, "Not Found"),
 INTERNAL_ERROR(500, "Internal Server Error");
 
 private final int code;
 private final String message;
 
 HttpStatus(int code, String message) {
 this.code = code;
 this.message = message;
 }
 
 public int code() { return code; }
 public String message() { return message; }
 
 public boolean isSuccess() { return code >= 200 && code < 300; }
 public boolean isClientError() { return code >= 400 && code < 500; }
 public boolean isServerError() { return code >= 500; }
 
 // Lookup by code
 public static HttpStatus fromCode(int code) {
 for (HttpStatus status : values()) {
 if (status.code == code) return status;
 }
 throw new IllegalArgumentException("Unknown HTTP status: " + code);
 }
}

// Enum with abstract method (strategy pattern)
public enum Operation {
 ADD {
 @Override public double apply(double a, double b) { return a + b; }
 },
 SUBTRACT {
 @Override public double apply(double a, double b) { return a - b; }
 },
 MULTIPLY {
 @Override public double apply(double a, double b) { return a * b; }
 };
 
 public abstract double apply(double a, double b);
}

double result = Operation.ADD.apply(3, 4); // 7.0

SOLID Principles in Java#

Single Responsibility#

// BAD: UserService does too much
class UserService {
 void createUser(User u) { /* ... */ }
 void sendEmail(User u, String msg) { /* ... */ }
 String generateReport(List<User> users) { /* ... */ }
}

// GOOD: Each class has one reason to change
class UserService { void createUser(User u) { /* ... */ } }
class EmailService { void send(String to, String msg) { /* ... */ } }
class UserReportGenerator { String generate(List<User> users) { /* ... */ } }

Open/Closed (via interfaces)#

// Open for extension, closed for modification
interface DiscountStrategy {
 double apply(double price);
}

class PercentageDiscount implements DiscountStrategy {
 private final double percent;
 PercentageDiscount(double percent) { this.percent = percent; }
 public double apply(double price) { return price * (1 - percent / 100); }
}

class FixedDiscount implements DiscountStrategy {
 private final double amount;
 FixedDiscount(double amount) { this.amount = amount; }
 public double apply(double price) { return Math.max(0, price - amount); }
}

// Adding new discount types doesn't modify existing code
class BuyOneGetOneFree implements DiscountStrategy {
 public double apply(double price) { return price / 2; }
}

Dependency Inversion#

// HIGH-level modules should not depend on LOW-level modules.
// Both should depend on abstractions.

// BAD: OrderService depends on concrete MySQLRepository
class OrderService {
 private MySQLOrderRepository repo = new MySQLOrderRepository();
}

// GOOD: Depend on abstraction
class OrderService {
 private final OrderRepository repo; // Interface
 
 OrderService(OrderRepository repo) { // Injected
 this.repo = repo;
 }
}
// Now works with MySQL, Postgres, InMemory, Mock...

Hypothetical Use Case: Event-Driven Domain Model#

public sealed interface DomainEvent permits OrderCreated, OrderShipped, OrderCancelled {
 String orderId();
 Instant occurredAt();
}

public record OrderCreated(String orderId, String customerId, Instant occurredAt) 
 implements DomainEvent {}

public record OrderShipped(String orderId, String trackingNumber, Instant occurredAt) 
 implements DomainEvent {}

public record OrderCancelled(String orderId, String reason, Instant occurredAt) 
 implements DomainEvent {}

public class Order {
 private final String id;
 private OrderStatus status;
 private final List<DomainEvent> events = new ArrayList<>();
 
 public void ship(String trackingNumber) {
 if (status != OrderStatus.CONFIRMED) {
 throw new IllegalStateException("Cannot ship " + status + " order");
 }
 status = OrderStatus.SHIPPED;
 events.add(new OrderShipped(id, trackingNumber, Instant.now()));
 }
 
 public List<DomainEvent> drainEvents() {
 var drained = List.copyOf(events);
 events.clear();
 return drained;
 }
}

Key Takeaways#

Encapsulation — fields private, access via methods (getters/setters or business methods)
Favor composition over inheritance — inject dependencies, delegate behavior
Interfaces define contracts — use them for polymorphism and testability
Enums are powerful — they can have fields, methods, and implement interfaces
Records (Java 16+) eliminate data class boilerplate
Sealed types (Java 17+) enable exhaustive pattern matching
SOLID principles guide class design — especially Dependency Inversion for testability

Joins and Subqueries

Mon, 01 Jan 0001 00:00:00 +0000

Why Joins?#

In a normalized database, related data lives in separate tables. Joins combine rows from two or more tables based on a related column, letting you reconstruct the full picture from properly separated data.

Join Types#

Visual Overview#

Assume two tables:

employees (id, name, department_id)
departments (id, name)

flowchart LR
 subgraph "INNER JOIN"
 A1["Only matching<br/>rows from both"]
 end
 subgraph "LEFT JOIN"
 A2["All from left +<br/>matching from right"]
 end
 subgraph "RIGHT JOIN"
 A3["All from right +<br/>matching from left"]
 end
 subgraph "FULL JOIN"
 A4["All from both,<br/>NULLs where no match"]
 end

INNER JOIN#

Returns only rows where there’s a match in both tables:

Language Models

Mon, 01 Jan 0001 00:00:00 +0000

This guide traces the evolution of language models — from counting word sequences to trillion-parameter neural networks that exhibit emergent reasoning. Understanding this history reveals why modern LLMs work the way they do and what trade-offs each generation made.

Prerequisites#

Basic probability (conditional probability, Bayes’ theorem)
Neural network fundamentals (layers, training, loss functions)
Transformer architecture basics (attention, encoder/decoder)

What Is a Language Model?#

A language model assigns probabilities to sequences of tokens:

Layout: Flexbox and Grid

Mon, 01 Jan 0001 00:00:00 +0000

The Layout Problem#

Before Flexbox and Grid, layout relied on floats, tables, and positioning hacks. Modern CSS provides two purpose-built layout systems:

Flexbox — one-dimensional (row OR column)
Grid — two-dimensional (rows AND columns simultaneously)

flowchart TD
 Q{Layout need?} -->|"Items in a line<br>(nav, toolbar, card row)"| Flex["Flexbox"]
 Q -->|"Page layout<br>(header, sidebar, content)"| Grid["Grid"]
 Q -->|"Both dimensions<br>(dashboard, gallery)"| Grid
 Q -->|"Content-driven<br>(items determine size)"| Flex
 Q -->|"Layout-driven<br>(grid determines size)"| Grid

Flexbox#

Core Concept#

A flex container distributes space among its children along a main axis:

Linear Models

Mon, 01 Jan 0001 00:00:00 +0000

Linear Models#

Linear models are the foundation of machine learning. They are fast, interpretable, and surprisingly effective. Every ML practitioner should understand them thoroughly — both because they serve as strong baselines and because the concepts (gradient descent, regularisation, loss functions) transfer to every other algorithm.

Linear Regression#

The simplest supervised model. It fits a linear function to predict a continuous target:

ŷ = w₁x₁ + w₂x₂ + ... + wₙxₙ + b

Where w are the weights (coefficients), b is the bias (intercept), and ŷ is the prediction.

Linked Lists

Mon, 01 Jan 0001 00:00:00 +0000

Linked Lists#

Linked list problems test pointer manipulation skills. The key insight: most solutions use combinations of just a few core techniques.

Pattern Summary#

Pattern	Time	Space	When to Use
Fast/Slow Pointers	O(n)	O(1)	Cycle detection, middle finding, nth from end
Reversal	O(n)	O(1)	Reverse whole list or sub-section
Merge	O(n + m)	O(1)	Combine sorted lists, interleave
Sentinel Node	O(n)	O(1)	Simplify edge cases (empty list, head deletion)
Runner Technique	O(n)	O(1)	Split list, reorder

Node Definition#

class ListNode:
 def __init__(self, val=0, next=None):
 self.val = val
 self.next = next

Fast/Slow Pointers (Floyd’s Tortoise and Hare)#

Two pointers move at different speeds. The fast pointer moves 2 steps for every 1 step of the slow pointer.

Load Balancing

Mon, 01 Jan 0001 00:00:00 +0000

Load Balancing#

Load balancers distribute incoming traffic across multiple servers to improve availability, reliability, and performance. They are one of the most fundamental building blocks in scalable system design.

Why Load Balancers?#

Problem	How Load Balancing Solves It
Single point of failure	Traffic reroutes to healthy servers if one dies
Capacity limits	Spread load across multiple machines
Uneven load distribution	Algorithms ensure balanced utilization
Maintenance downtime	Drain connections before taking servers offline
Geographic latency	Route to the nearest healthy endpoint

Layer 4 vs Layer 7 Load Balancing#

graph TD
 subgraph "OSI Model"
 L7[Layer 7 — Application: HTTP, gRPC, WebSocket]
 L4[Layer 4 — Transport: TCP, UDP]
 L3[Layer 3 — Network: IP]
 end

 subgraph "L4 Load Balancer"
 L4LB[Routes based on IP + Port<br/>No payload inspection]
 end

 subgraph "L7 Load Balancer"
 L7LB[Routes based on headers, URL path,<br/>cookies, content type]
 end

Comparison Table#

Feature	L4 Load Balancer	L7 Load Balancer
Decision basis	IP address + TCP/UDP port	HTTP headers, URL, cookies, payload
Performance	Faster (no payload parsing)	Slightly slower (must parse application data)
SSL termination	Typically no (pass-through)	Yes (terminates TLS, re-encrypts optionally)
Content routing	No	Yes (path-based, host-based routing)
WebSocket support	Pass-through (no awareness)	Upgrade-aware, can route by path
Health checks	TCP connect / port check	HTTP status codes, response body checks
Connection pooling	No	Yes (reduces backend connections)
Sticky sessions	IP-based only	Cookie-based, header-based
Use case	High-throughput TCP (databases, gRPC)	Web applications, APIs, microservices
AWS equivalent	Network Load Balancer (NLB)	Application Load Balancer (ALB)

Load Balancing Algorithms#

Round Robin#

Distributes requests sequentially across all servers.

Logging

Mon, 01 Jan 0001 00:00:00 +0000

Logging#

Logs are timestamped, immutable records of discrete events. They provide the richest context of any telemetry signal — the actual error message, the request payload that triggered it, the stack trace. This section covers structured logging, log levels, JSON format, aggregation pipelines, and the major log management stacks.

Unstructured vs Structured Logging#

The Problem with Unstructured Logs#

Traditional logs are free-form text:

2026-08-27 08:15:32 ERROR OrderService - Failed to process order ORD-9182 for user 4521: PaymentGatewayTimeout after 30042ms

This is human-readable but machine-hostile. To extract the order ID, user ID, or duration, you need regex parsing — which is fragile, slow, and breaks when the message format changes.

LSM Trees & Log-Structured Storage

Mon, 01 Jan 0001 00:00:00 +0000

The Write Optimisation Problem#

B+trees provide excellent read performance, but writes are expensive: inserting a single key may require reading a leaf page, modifying it, and writing it back — all random I/O. For write-heavy workloads (time-series, event logs, messaging), this becomes a bottleneck.

Log-Structured Merge-trees (LSM-trees) flip the trade-off: absorb all writes in memory, then flush sorted runs to disk sequentially. Reads become more expensive, but writes are dramatically cheaper.

Memory Management

Mon, 01 Jan 0001 00:00:00 +0000

Memory Management#

The OS must multiplex physical memory among many processes while giving each the illusion of a large, private, contiguous address space. Memory management handles allocation, protection, and efficient use of RAM.

Address Spaces#

Every process operates in its own virtual address space — a logical view of memory that the hardware translates to physical addresses.

Concept	Description
Logical (virtual) address	What the process sees (generated by CPU)
Physical address	Actual location in RAM
Address translation	Mapping logical → physical (done by MMU)
Protection	One process cannot access another’s memory

Paging#

Paging divides memory into fixed-size blocks, eliminating external fragmentation:

Mocking & Test Doubles

Mon, 01 Jan 0001 00:00:00 +0000

Mocking & Test Doubles#

When code depends on external systems — databases, APIs, file systems, or clocks — you need strategies to test in isolation. Test doubles replace real dependencies with controlled substitutes. This section covers types of doubles, when to use them, and how to avoid the over-mocking trap.

Types of Test Doubles#

graph TB
 TD[Test Doubles] --> DUMMY[Dummy<br/>Fills a parameter slot]
 TD --> STUB[Stub<br/>Returns canned data]
 TD --> SPY[Spy<br/>Records interactions]
 TD --> MOCK[Mock<br/>Verifies expectations]
 TD --> FAKE[Fake<br/>Working simplified implementation]

Comparison Table#

Double	Purpose	Verifies Behaviour?	Has Logic?	Example
Dummy	Satisfies a type signature, never actually used	No	No	A null logger passed to a constructor
Stub	Returns pre-configured responses	No	Minimal	`get_user()` always returns `{"id": 1, "name": "Alice"}`
Spy	Records calls for later verification	Yes (after the fact)	No	Records that `send_email` was called with specific args
Mock	Expects specific calls — fails if not met	Yes (built-in)	No	Expects `repository.save(order)` to be called exactly once
Fake	Simplified but functional implementation	No	Yes	In-memory database, local file system

When to Use Each#

Situation	Best Double
Dependency is irrelevant to this test	Dummy
Need to control what dependency returns	Stub
Need to verify how something was called (args, count)	Spy or Mock
Need a working substitute with real behaviour	Fake
Need to prevent real side effects (emails, charges)	Stub or Fake

Practical Examples#

Stubs — Controlling Return Values#

from unittest.mock import patch

def get_weather_message(city: str) -> str:
 """Fetches weather and formats a message."""
 data = weather_api.get_current(city) # External call
 return f"{city}: {data['temp']}°C, {data['condition']}"

@patch("src.weather.weather_api.get_current")
def test_weather_message_formatting(mock_api):
 # Stub: control the return value
 mock_api.return_value = {"temp": 22, "condition": "sunny"}

 result = get_weather_message("Madrid")

 assert result == "Madrid: 22°C, sunny"

// Jest stub
jest.spyOn(weatherApi, 'getCurrent').mockResolvedValue({
 temp: 22,
 condition: 'sunny',
});

test('formats weather message correctly', async () => {
 const result = await getWeatherMessage('Madrid');
 expect(result).toBe('Madrid: 22°C, sunny');
});

Spies — Verifying Interactions#

from unittest.mock import MagicMock

def test_order_sends_confirmation_email():
 email_service = MagicMock()
 order_service = OrderService(email_service=email_service)

 order_service.complete_order(order_id="ord_123")

 # Spy: verify the interaction happened correctly
 email_service.send.assert_called_once_with(
 to="customer@example.com",
 subject="Order Confirmed",
 template="order_confirmation",
 data={"order_id": "ord_123"},
 )

test('sends confirmation email on order completion', () => {
 const sendEmail = jest.fn();
 const service = new OrderService({ sendEmail });

 service.completeOrder('ord_123');

 expect(sendEmail).toHaveBeenCalledWith(
 expect.objectContaining({
 to: 'customer@example.com',
 subject: 'Order Confirmed',
 })
 );
});

Fakes — Simplified Real Implementations#

class FakeUserRepository:
 """In-memory implementation for testing."""

 def __init__(self):
 self._users = {}

 def save(self, user):
 self._users[user["id"]] = user

 def find_by_id(self, user_id):
 return self._users.get(user_id)

 def find_by_email(self, email):
 return next(
 (u for u in self._users.values() if u["email"] == email),
 None,
 )

def test_register_user_saves_and_retrieves():
 repo = FakeUserRepository()
 service = UserService(repository=repo)

 service.register(name="Alice", email="alice@test.com")

 user = repo.find_by_email("alice@test.com")
 assert user is not None
 assert user["name"] == "Alice"

When to Mock#

Good Reasons to Mock#

Reason	Example
Non-deterministic	Current time, random values, UUIDs
Slow	Network calls, database queries in unit tests
Side effects	Sending emails, charging credit cards, writing files
Unavailable	Third-party API not accessible in CI
Hard to trigger	Network timeouts, disk full, race conditions

Bad Reasons to Mock#

Reason	Better Approach
“It’s complex”	Simplify the design instead
“It’s another class I wrote”	Test them together (integration test)
“To get coverage”	Write a meaningful test instead
“Everyone else does it”	Question whether it adds value

The Over-Mocking Anti-Pattern#

Over-mocking happens when tests mock so many things that they test nothing real:

Network Security

Mon, 01 Jan 0001 00:00:00 +0000

Why This Matters#

Networks carry all data between systems. If an attacker controls the network, they can intercept, modify, or block any communication. Network security creates boundaries, monitors traffic, and ensures only authorized communication occurs.

Defense in Depth#

No single control is sufficient. Layer defenses so that if one fails, others still protect:

flowchart TD
 Internet["Internet (untrusted)"]
 Internet --> Edge["Edge Security<br/>DDoS protection, CDN, WAF"]
 Edge --> Perimeter["Perimeter Firewall<br/>Stateful inspection, IPS"]
 Perimeter --> DMZ["DMZ<br/>Reverse proxies, public-facing services"]
 DMZ --> Internal["Internal Firewall<br/>Micro-segmentation"]
 Internal --> App["Application Layer<br/>Input validation, auth"]
 App --> Data["Data Layer<br/>Encryption at rest, access controls"]

Firewalls#

Types#

Type	OSI Layer	Inspects	Strengths	Limitations
Packet Filter	L3–L4	IP, port, protocol	Fast, simple	No state, no content inspection
Stateful	L3–L4	Connection state + packets	Tracks sessions	Can’t inspect encrypted traffic
Application/WAF	L7	HTTP content, headers, body	Deep inspection	Slower, complex rules
Next-Gen (NGFW)	L3–L7	All + threat intel + user identity	Comprehensive	Expensive, complex

Firewall Rule Design#

Rules are evaluated top-to-bottom, first match wins:

Networking

Mon, 01 Jan 0001 00:00:00 +0000

Networking#

How pods communicate with each other, how services expose pods to the cluster and the outside world, and how to control traffic flow.

Pod Networking Model#

Kubernetes imposes a flat networking model with three fundamental rules:

Every pod gets its own IP address — no NAT between pods
All pods can communicate with all other pods without NAT (across nodes)
Agents on a node can communicate with all pods on that node

This is implemented by CNI (Container Network Interface) plugins:

Networking Architecture

Mon, 01 Jan 0001 00:00:00 +0000

Networking Architecture#

Cloud networking is not just “VPC plus subnets.” It’s the connective tissue between every service, account, region, and on-premises data centre. Mistakes here are expensive to fix because networking underpins everything above it. This section covers the design patterns that scale from one VPC to hundreds of accounts across multiple regions.

VPC Design Patterns#

A Virtual Private Cloud (VPC / VNet) is your isolated network within the provider’s infrastructure. The design of your VPC determines how workloads communicate, how traffic enters and exits, and where security boundaries lie.

Ownership & Borrowing

Mon, 01 Jan 0001 00:00:00 +0000

Ownership & Borrowing#

This is the concept that makes Rust unique. The ownership system is a set of compile-time rules that ensure memory safety without a garbage collector. Every value has exactly one owner, and when the owner goes out of scope, the value is dropped (freed). Borrowing lets you temporarily access values without taking ownership.

The Three Rules of Ownership#

Each value has exactly one owner.
When the owner goes out of scope, the value is dropped.
Ownership can be transferred (moved) but not duplicated (for non-Copy types).

{
 let s = String::from("hello"); // s owns the String
 // s is valid here
} // s goes out of scope — String is dropped, memory freed

No garbage collector. No manual free(). The compiler inserts drop calls at the right places.

Providers, Resources, and Data Sources

Mon, 01 Jan 0001 00:00:00 +0000

Providers#

Providers are Terraform’s plugin system. Each provider is a bridge between Terraform and an external API — AWS, Azure, GCP, Kubernetes, GitHub, Datadog, or any service with an API.

flowchart TD
 TF["Terraform Core"] --> AWS["AWS Provider"]
 TF --> Azure["Azure Provider"]
 TF --> K8s["Kubernetes Provider"]
 TF --> GH["GitHub Provider"]
 
 AWS --> AWSAPI["AWS API"]
 Azure --> AzureAPI["Azure API"]
 K8s --> K8sAPI["Kubernetes API"]
 GH --> GHAPI["GitHub API"]

Provider Configuration#

terraform {
 required_providers {
 aws = {
 source = "hashicorp/aws"
 version = "~> 5.0"
 }
 random = {
 source = "hashicorp/random"
 version = "~> 3.6"
 }
 }
}

provider "aws" {
 region = var.aws_region
 
 default_tags {
 tags = {
 Environment = var.environment
 ManagedBy = "terraform"
 Team = var.team
 }
 }
}

Version Constraints#

Constraint	Meaning	Example
`= 5.0.0`	Exact version	Only 5.0.0
`>= 5.0`	Minimum version	5.0 or higher
`~> 5.0`	Pessimistic (minor)	≥ 5.0, < 6.0
`~> 5.0.0`	Pessimistic (patch)	≥ 5.0.0, < 5.1.0
`>= 5.0, < 6.0`	Range	Between 5.0 and 6.0

Best practice: Use ~> MAJOR.MINOR for providers (allows patch updates, blocks breaking changes).

Python: Object-Oriented Programming

Mon, 01 Jan 0001 00:00:00 +0000

Classes — The Fundamentals#

Anatomy of a Class#

class BankAccount:
 """A simple bank account with deposit/withdraw operations."""
 
 # Class variable — shared across ALL instances
 bank_name = "Python National Bank"
 _total_accounts = 0
 
 def __init__(self, owner: str, balance: float = 0.0):
 """Instance initializer (NOT a constructor — __new__ is the constructor)."""
 # Instance variables — unique to each instance
 self.owner = owner
 self._balance = balance # Convention: "protected"
 self.__id = id(self) # Name-mangled: becomes _BankAccount__id
 BankAccount._total_accounts += 1
 
 @property
 def balance(self) -> float:
 """Read-only property — controlled access to _balance."""
 return self._balance
 
 @balance.setter
 def balance(self, value: float):
 if value < 0:
 raise ValueError("Balance cannot be negative")
 self._balance = value
 
 def deposit(self, amount: float) -> float:
 """Instance method — operates on self."""
 if amount <= 0:
 raise ValueError("Deposit must be positive")
 self._balance += amount
 return self._balance
 
 def withdraw(self, amount: float) -> float:
 if amount > self._balance:
 raise InsufficientFundsError(self._balance, amount)
 self._balance -= amount
 return self._balance
 
 @classmethod
 def from_dict(cls, data: dict) -> "BankAccount":
 """Alternative constructor — creates instance from dictionary."""
 return cls(owner=data["owner"], balance=data.get("balance", 0))
 
 @classmethod
 def get_total_accounts(cls) -> int:
 """Access class-level state."""
 return cls._total_accounts
 
 @staticmethod
 def validate_amount(amount: float) -> bool:
 """Utility — no access to instance or class state."""
 return isinstance(amount, (int, float)) and amount > 0
 
 def __repr__(self) -> str:
 """Unambiguous representation (for developers)."""
 return f"BankAccount(owner={self.owner!r}, balance={self._balance})"
 
 def __str__(self) -> str:
 """Human-readable representation."""
 return f"{self.owner}'s account: ${self._balance:,.2f}"

Instance vs Class vs Static Methods#

flowchart TD
 A["Method Types"] --> B["Instance Method"]
 A --> C["Class Method"]
 A --> D["Static Method"]
 
 B --> B1["def method(self)"]
 B --> B2["Access instance + class state"]
 B --> B3["Most common"]
 
 C --> C1["@classmethod<br>def method(cls)"]
 C --> C2["Access class state only"]
 C --> C3["Alternative constructors"]
 
 D --> D1["@staticmethod<br>def method()"]
 D --> D2["No access to instance or class"]
 D --> D3["Utility functions"]

The Data Model — Dunder Methods#

Making Objects Behave Like Built-in Types#

class Vector:
 """2D vector with full operator support."""
 
 __slots__ = ('x', 'y') # Memory optimization — no __dict__
 
 def __init__(self, x: float, y: float):
 self.x = x
 self.y = y
 
 # Arithmetic operators
 def __add__(self, other: "Vector") -> "Vector":
 return Vector(self.x + other.x, self.y + other.y)
 
 def __sub__(self, other: "Vector") -> "Vector":
 return Vector(self.x - other.x, self.y - other.y)
 
 def __mul__(self, scalar: float) -> "Vector":
 return Vector(self.x * scalar, self.y * scalar)
 
 def __rmul__(self, scalar: float) -> "Vector":
 """Handles: 3 * vector (scalar on the left)."""
 return self.__mul__(scalar)
 
 def __abs__(self) -> float:
 """Magnitude: abs(vector)."""
 return (self.x ** 2 + self.y ** 2) ** 0.5
 
 # Comparison
 def __eq__(self, other: object) -> bool:
 if not isinstance(other, Vector):
 return NotImplemented
 return self.x == other.x and self.y == other.y
 
 def __lt__(self, other: "Vector") -> bool:
 return abs(self) < abs(other)
 
 # Container protocol
 def __len__(self) -> int:
 return 2
 
 def __getitem__(self, index: int) -> float:
 match index:
 case 0: return self.x
 case 1: return self.y
 case _: raise IndexError(f"Vector index {index} out of range")
 
 def __iter__(self):
 yield self.x
 yield self.y
 
 # Hashing (required for use in sets/dict keys)
 def __hash__(self) -> int:
 return hash((self.x, self.y))
 
 def __repr__(self) -> str:
 return f"Vector({self.x}, {self.y})"
 
 def __format__(self, spec: str) -> str:
 if spec == "polar":
 import math
 r = abs(self)
 theta = math.atan2(self.y, self.x)
 return f"({r:.4f}, {math.degrees(theta):.2f}°)"
 return f"({self.x}, {self.y})"


# Usage
v1 = Vector(3, 4)
v2 = Vector(1, 2)

v3 = v1 + v2 # Vector(4, 6)
v4 = v1 * 2 # Vector(6, 8)
v5 = 3 * v1 # Vector(9, 12) — uses __rmul__
magnitude = abs(v1) # 5.0
x, y = v1 # unpacking via __iter__
print(f"{v1:polar}") # (5.0000, 53.13°)

Key Dunder Methods Reference#

Category	Methods	Purpose
Creation	`__new__`, `__init__`, `__del__`	Object lifecycle
Representation	`__repr__`, `__str__`, `__format__`	String conversion
Comparison	`__eq__`, `__lt__`, `__le__`, `__gt__`, `__ge__`	Ordering
Arithmetic	`__add__`, `__sub__`, `__mul__`, `__truediv__`	Operators
Container	`__len__`, `__getitem__`, `__setitem__`, `__contains__`	Sequence/mapping
Callable	`__call__`	`obj()` syntax
Context	`__enter__`, `__exit__`	`with` statement
Attribute	`__getattr__`, `__setattr__`, `__delattr__`	Attribute access
Hashing	`__hash__`	Sets, dict keys

Inheritance and Composition#

Method Resolution Order (MRO)#

class A:
 def method(self):
 return "A"

class B(A):
 def method(self):
 return "B"

class C(A):
 def method(self):
 return "C"

class D(B, C):
 pass

# MRO: D → B → C → A → object
print(D.__mro__)
# (<class 'D'>, <class 'B'>, <class 'C'>, <class 'A'>, <class 'object'>)

D().method() # "B" — follows MRO left to right

graph BT
 D --> B
 D --> C
 B --> A
 C --> A
 A --> object
 
 style D fill:#e1f5fe
 style B fill:#fff3e0
 style C fill:#fff3e0
 style A fill:#f3e5f5

Cooperative Multiple Inheritance with super()#

class Serializable:
 def serialize(self) -> dict:
 return {"type": type(self).__name__}

class Timestamped:
 def __init__(self, **kwargs):
 super().__init__(**kwargs)
 from datetime import datetime
 self.created_at = datetime.now()
 
 def serialize(self) -> dict:
 data = super().serialize()
 data["created_at"] = self.created_at.isoformat()
 return data

class Identifiable:
 def __init__(self, **kwargs):
 super().__init__(**kwargs)
 import uuid
 self.id = str(uuid.uuid4())
 
 def serialize(self) -> dict:
 data = super().serialize()
 data["id"] = self.id
 return data

class Event(Identifiable, Timestamped, Serializable):
 def __init__(self, name: str, **kwargs):
 super().__init__(**kwargs)
 self.name = name
 
 def serialize(self) -> dict:
 data = super().serialize()
 data["name"] = self.name
 return data

event = Event("user_signup")
event.serialize()
# {"type": "Event", "created_at": "...", "id": "...", "name": "user_signup"}

Composition Over Inheritance#

# BAD: Deep inheritance hierarchy
class Animal:
 def eat(self): ...
class FlyingAnimal(Animal):
 def fly(self): ...
class SwimmingAnimal(Animal):
 def swim(self): ...
class Duck(FlyingAnimal, SwimmingAnimal): # Diamond problem
 pass

# GOOD: Composition with protocols
from typing import Protocol

class CanFly(Protocol):
 def fly(self) -> str: ...

class CanSwim(Protocol):
 def swim(self) -> str: ...

class Wings:
 def fly(self) -> str:
 return "Flapping wings"

class Fins:
 def swim(self) -> str:
 return "Using fins"

class Webbed:
 def swim(self) -> str:
 return "Paddling with webbed feet"

class Duck:
 def __init__(self):
 self._flying = Wings()
 self._swimming = Webbed()
 
 def fly(self) -> str:
 return self._flying.fly()
 
 def swim(self) -> str:
 return self._swimming.swim()

Abstract Base Classes#

from abc import ABC, abstractmethod

class Shape(ABC):
 """Abstract base class — cannot be instantiated directly."""
 
 @abstractmethod
 def area(self) -> float:
 """Subclasses MUST implement this."""
 ...
 
 @abstractmethod
 def perimeter(self) -> float:
 ...
 
 def describe(self) -> str:
 """Concrete method — inherited by all subclasses."""
 return f"{type(self).__name__}: area={self.area():.2f}, perimeter={self.perimeter():.2f}"

class Circle(Shape):
 def __init__(self, radius: float):
 self.radius = radius
 
 def area(self) -> float:
 import math
 return math.pi * self.radius ** 2
 
 def perimeter(self) -> float:
 import math
 return 2 * math.pi * self.radius

# Shape() # TypeError: Can't instantiate abstract class
c = Circle(5)
c.describe() # "Circle: area=78.54, perimeter=31.42"

Dataclasses — Modern Python OOP#

from dataclasses import dataclass, field, asdict, astuple
from typing import ClassVar

@dataclass(frozen=True, slots=True) # Immutable + memory efficient
class Coordinate:
 latitude: float
 longitude: float
 
 def distance_to(self, other: "Coordinate") -> float:
 """Haversine distance in km."""
 import math
 R = 6371 # Earth radius in km
 dlat = math.radians(other.latitude - self.latitude)
 dlon = math.radians(other.longitude - self.longitude)
 a = (math.sin(dlat/2)**2 + 
 math.cos(math.radians(self.latitude)) * 
 math.cos(math.radians(other.latitude)) * 
 math.sin(dlon/2)**2)
 return R * 2 * math.asin(math.sqrt(a))

@dataclass
class Order:
 id: str
 customer: str
 items: list[str] = field(default_factory=list) # Mutable default
 total: float = 0.0
 _internal: str = field(default="", repr=False, compare=False)
 
 # Class variable (not a field)
 MAX_ITEMS: ClassVar[int] = 100
 
 def __post_init__(self):
 """Validation after __init__."""
 if self.total < 0:
 raise ValueError("Total cannot be negative")
 if len(self.items) > self.MAX_ITEMS:
 raise ValueError(f"Max {self.MAX_ITEMS} items per order")

# Auto-generated: __init__, __repr__, __eq__, __hash__ (if frozen)
paris = Coordinate(48.8566, 2.3522)
london = Coordinate(51.5074, -0.1278)
paris.distance_to(london) # ~343.5 km

order = Order(id="ORD-001", customer="Alice", items=["Widget"])
asdict(order) # Convert to dictionary

Descriptors — The Mechanism Behind Properties#

class Validated:
 """Descriptor that validates values on assignment."""
 
 def __init__(self, validator, error_msg="Invalid value"):
 self.validator = validator
 self.error_msg = error_msg
 self.attr_name = None # Set by __set_name__
 
 def __set_name__(self, owner, name):
 self.attr_name = f"_{name}"
 
 def __get__(self, obj, objtype=None):
 if obj is None:
 return self
 return getattr(obj, self.attr_name, None)
 
 def __set__(self, obj, value):
 if not self.validator(value):
 raise ValueError(f"{self.error_msg}: {value!r}")
 setattr(obj, self.attr_name, value)

class PositiveNumber(Validated):
 def __init__(self):
 super().__init__(
 validator=lambda x: isinstance(x, (int, float)) and x > 0,
 error_msg="Must be a positive number"
 )

class NonEmptyString(Validated):
 def __init__(self):
 super().__init__(
 validator=lambda x: isinstance(x, str) and len(x.strip()) > 0,
 error_msg="Must be a non-empty string"
 )

class Product:
 name = NonEmptyString()
 price = PositiveNumber()
 quantity = PositiveNumber()
 
 def __init__(self, name: str, price: float, quantity: int):
 self.name = name # Triggers NonEmptyString.__set__
 self.price = price # Triggers PositiveNumber.__set__
 self.quantity = quantity

p = Product("Widget", 9.99, 100) # OK
# Product("", 9.99, 100) # ValueError: Must be a non-empty string
# Product("X", -1, 100) # ValueError: Must be a positive number

Metaclasses — Classes That Create Classes#

class SingletonMeta(type):
 """Metaclass that ensures only one instance of a class exists."""
 _instances = {}
 
 def __call__(cls, *args, **kwargs):
 if cls not in cls._instances:
 cls._instances[cls] = super().__call__(*args, **kwargs)
 return cls._instances[cls]

class DatabaseConnection(metaclass=SingletonMeta):
 def __init__(self, url: str = "localhost:5432"):
 self.url = url
 self.connected = False
 
 def connect(self):
 self.connected = True

# Always returns the same instance
db1 = DatabaseConnection("prod:5432")
db2 = DatabaseConnection("other:5432") # Ignored — returns existing
db1 is db2 # True

flowchart TD
 A["type (metaclass)"] -->|"creates"| B["MyClass (class)"]
 B -->|"creates"| C["instance (object)"]
 
 A -->|"type(MyClass) → type"| A
 B -->|"type(instance) → MyClass"| B
 C -->|"isinstance(instance, MyClass) → True"| B

Hypothetical Use Cases#

Use Case: Domain Model with Validation#

from dataclasses import dataclass, field
from datetime import datetime, date
from enum import Enum, auto

class OrderStatus(Enum):
 PENDING = auto()
 CONFIRMED = auto()
 SHIPPED = auto()
 DELIVERED = auto()
 CANCELLED = auto()

@dataclass
class OrderItem:
 product_id: str
 name: str
 quantity: int
 unit_price: float
 
 @property
 def subtotal(self) -> float:
 return self.quantity * self.unit_price

@dataclass
class Order:
 id: str
 customer_id: str
 items: list[OrderItem] = field(default_factory=list)
 status: OrderStatus = OrderStatus.PENDING
 created_at: datetime = field(default_factory=datetime.now)
 shipped_at: datetime | None = None
 
 @property
 def total(self) -> float:
 return sum(item.subtotal for item in self.items)
 
 def add_item(self, item: OrderItem) -> None:
 if self.status != OrderStatus.PENDING:
 raise InvalidOperationError(f"Cannot modify {self.status.name} order")
 self.items.append(item)
 
 def confirm(self) -> None:
 if not self.items:
 raise InvalidOperationError("Cannot confirm empty order")
 self._transition(OrderStatus.CONFIRMED)
 
 def ship(self) -> None:
 self._transition(OrderStatus.SHIPPED)
 self.shipped_at = datetime.now()
 
 def cancel(self) -> None:
 if self.status in (OrderStatus.SHIPPED, OrderStatus.DELIVERED):
 raise InvalidOperationError("Cannot cancel shipped/delivered order")
 self._transition(OrderStatus.CANCELLED)
 
 def _transition(self, new_status: OrderStatus) -> None:
 valid_transitions = {
 OrderStatus.PENDING: {OrderStatus.CONFIRMED, OrderStatus.CANCELLED},
 OrderStatus.CONFIRMED: {OrderStatus.SHIPPED, OrderStatus.CANCELLED},
 OrderStatus.SHIPPED: {OrderStatus.DELIVERED},
 }
 allowed = valid_transitions.get(self.status, set())
 if new_status not in allowed:
 raise InvalidOperationError(
 f"Cannot transition from {self.status.name} to {new_status.name}"
 )
 self.status = new_status

Use Case: Registry Pattern with Metaclass#

class PluginRegistry(type):
 """Auto-registers all subclasses of a base class."""
 _registry: dict[str, type] = {}
 
 def __init_subclass__(cls, **kwargs):
 super().__init_subclass__(**kwargs)
 if hasattr(cls, "name"):
 PluginRegistry._registry[cls.name] = cls
 
 @classmethod
 def get(mcs, name: str):
 return mcs._registry.get(name)
 
 @classmethod
 def list_plugins(mcs) -> list[str]:
 return list(mcs._registry.keys())

class Exporter(metaclass=PluginRegistry):
 """Base class — subclasses auto-register."""
 def export(self, data) -> bytes:
 raise NotImplementedError

class JSONExporter(Exporter):
 name = "json"
 def export(self, data) -> bytes:
 import json
 return json.dumps(data).encode()

class CSVExporter(Exporter):
 name = "csv"
 def export(self, data) -> bytes:
 # ... CSV export logic
 pass

# Usage
exporter_cls = PluginRegistry.get("json")
exporter = exporter_cls()
result = exporter.export({"key": "value"})

Key Takeaways#

Properties provide controlled attribute access without changing the API
__slots__ saves memory by eliminating per-instance __dict__
Dataclasses eliminate boilerplate for data-holding classes
Composition > inheritance — prefer has-a over is-a relationships
Descriptors are the mechanism behind property, classmethod, staticmethod
Metaclasses are rarely needed — use __init_subclass__ for most registration patterns
ABC enforces interface contracts at instantiation time, not at call time

Reflog & Recovery

Mon, 01 Jan 0001 00:00:00 +0000

Reflog & Recovery#

The reflog is Git’s safety net — a local log of every position your HEAD and branch refs have pointed to. Even after a destructive operation, the reflog usually holds the key to recovery.

What the Reflog Tracks#

The reflog records every time a ref (HEAD, branch, etc.) changes value. Each entry contains:

The old and new SHA of the ref
A timestamp
The action that caused the change
A message describing what happened

# View HEAD reflog
git reflog
# or equivalently:
git reflog show HEAD

# View a specific branch's reflog
git reflog show main

# View with timestamps
git reflog --date=iso

Example output:

SQL for Data Engineering

Mon, 01 Jan 0001 00:00:00 +0000

SQL as the Data Engineer’s Core Language#

SQL is the lingua franca of data engineering. Every warehouse, most processing engines (Spark SQL, Presto, Trino), and transformation tools (dbt) speak SQL. Mastering advanced SQL patterns is non-negotiable.

Common Table Expressions (CTEs)#

CTEs improve readability by breaking complex queries into named, logical steps. They are defined with the WITH keyword and referenced like tables.

WITH daily_revenue AS (
 SELECT
 date_key,
 SUM(total_amount) AS revenue,
 COUNT(DISTINCT customer_key) AS unique_customers
 FROM fact_sales
 GROUP BY date_key
),
revenue_with_avg AS (
 SELECT
 dr.*,
 AVG(revenue) OVER (ORDER BY date_key ROWS BETWEEN 6 PRECEDING AND CURRENT ROW)
 AS rolling_7d_avg
 FROM daily_revenue dr
)
SELECT
 d.full_date,
 r.revenue,
 r.unique_customers,
 ROUND(r.rolling_7d_avg, 2) AS rolling_7d_avg
FROM revenue_with_avg r
JOIN dim_date d ON r.date_key = d.date_key
WHERE d.calendar_year = 2025
ORDER BY d.full_date;

Recursive CTEs#

Useful for traversing hierarchical data (org charts, category trees).

SSH

Mon, 01 Jan 0001 00:00:00 +0000

What Is SSH?#

SSH (Secure Shell) is a cryptographic network protocol for secure communication between computers over an unsecured network. It provides encrypted remote login, command execution, file transfer, and port forwarding.

SSH replaced insecure predecessors like Telnet, rlogin, and FTP — which transmitted everything (including passwords) in plaintext.

What SSH Provides#

Feature	Description
Authentication	Verify identity (password or key-based)
Encryption	All traffic encrypted in transit (AES, ChaCha20)
Integrity	Detect tampering (HMAC)
Tunneling	Forward ports, proxy traffic through encrypted channel
File transfer	SCP and SFTP for secure file operations

How SSH Works#

sequenceDiagram
 participant Client
 participant Server
 
 Client->>Server: TCP connection (port 22)
 Client->>Server: Protocol version exchange
 Server->>Client: Protocol version exchange
 Note over Client,Server: Key exchange (Diffie-Hellman)
 Note over Client,Server: Session encryption established
 Client->>Server: Authentication (key or password)
 Server->>Client: Authentication success
 Note over Client,Server: Encrypted session active

Authentication Methods#

Method	Security	Use Case
Public key	Excellent	Default for servers, CI/CD, automation
Password	Moderate	Quick setup, less secure
Certificate	Excellent	Enterprise, large fleet management
GSSAPI/Kerberos	Excellent	Corporate environments with AD

Key-Based Authentication#

Generating Keys#

# Ed25519 (recommended — fast, secure, short keys)
ssh-keygen -t ed25519 -C "you@example.com"

# RSA (legacy compatibility — use 4096 bits minimum)
ssh-keygen -t rsa -b 4096 -C "you@example.com"

# With custom filename
ssh-keygen -t ed25519 -f ~/.ssh/id_work -C "work laptop"

This creates two files:

Strings & Arrays

Mon, 01 Jan 0001 00:00:00 +0000

Strings & Arrays#

C has no string type. A “string” is a char array terminated by a null byte ('\0'). This simplicity is elegant but requires constant vigilance against buffer overflows.

C Strings#

char greeting[] = "Hello"; // compiler adds '\0', size = 6
char *ptr = "World"; // points to string literal (read-only!)

// Memory layout of greeting:
// ['H','e','l','l','o','\0']
// [0] [1] [2] [3] [4] [5]

String Length vs Array Size#

char s[] = "Hello";
strlen(s); // 5 (characters, NOT including '\0')
sizeof(s); // 6 (bytes, INCLUDING '\0')

String Functions (`<string.h>`)#

Function	Purpose	Danger
`strlen(s)`	Length (without ‘\0’)	Scans until ‘\0’ — O(n)
`strcpy(dst, src)`	Copy string	No bounds checking — use `strncpy`
`strncpy(dst, src, n)`	Copy at most n chars	May not null-terminate!
`strcat(dst, src)`	Concatenate	No bounds checking — use `strncat`
`strcmp(a, b)`	Compare (0 = equal)	Returns int, not bool
`strncmp(a, b, n)`	Compare first n chars
`strchr(s, c)`	Find first occurrence of c	Returns NULL if not found
`strstr(s, sub)`	Find substring	Returns NULL if not found
`strtok(s, delim)`	Tokenise (destructive!)	Modifies the original string
`snprintf(buf, n, fmt, ...)`	Safe formatted print to buffer	Preferred — always bounds-checked

Safe String Handling#

// BAD — buffer overflow
char buf[10];
strcpy(buf, "This string is way too long for the buffer");

// GOOD — bounded
char buf[10];
snprintf(buf, sizeof(buf), "Hello %s", name);
// Always null-terminates, truncates if necessary

Rule: Always use snprintf instead of sprintf, strncpy instead of strcpy. Better yet, track buffer sizes explicitly.

Advanced Branching

Mon, 01 Jan 0001 00:00:00 +0000

Advanced Branching#

Beyond basic branching, Git offers powerful tools for parallel development, automated debugging, selective commit application, and efficient handling of large repositories.

Worktrees — Parallel Development#

A worktree is an additional working directory linked to the same repository. Instead of stashing or committing WIP to switch context, you open another worktree:

# Create a new worktree for a branch
git worktree add ../project-hotfix hotfix/urgent-fix

# Create a worktree with a new branch
git worktree add -b feature/new-thing ../project-feature main

# List all worktrees
git worktree list
# /home/user/project abc1234 [main]
# /home/user/project-hotfix def5678 [hotfix/urgent-fix]
# /home/user/project-feature 9ab0123 [feature/new-thing]

# Remove a worktree (after you're done)
git worktree remove ../project-hotfix

# Prune stale worktree references
git worktree prune

Worktree Rules#

Rule	Reason
Each worktree must be on a different branch	Git can’t have two worktrees on the same ref
All worktrees share the same `.git` objects	Storage efficient — no duplicate history
Commits in one worktree are immediately visible in others	Single object store
Don’t `git checkout` into a branch used by another worktree	Fails with an error

When to Use Worktrees#

Reviewing a PR while your current branch has uncommitted work
Running a long build on one branch while developing on another
Comparing behaviour between two branches simultaneously
Hotfixes that can’t wait for your current work to finish

Bisect — Binary Search for Bugs#

git bisect performs a binary search through history to find which commit introduced a bug:

Aggregations and Grouping

Mon, 01 Jan 0001 00:00:00 +0000

Aggregate Functions#

Aggregate functions compute a single value from a set of rows:

Function	Purpose	Example
`COUNT(*)`	Number of rows	Total employees
`COUNT(column)`	Non-NULL values in column	Employees with a phone
`COUNT(DISTINCT col)`	Unique non-NULL values	Number of departments
`SUM(column)`	Total	Total payroll
`AVG(column)`	Average	Average salary
`MIN(column)`	Smallest	Lowest salary
`MAX(column)`	Largest	Highest salary

SELECT
 COUNT(*) AS total_employees,
 COUNT(DISTINCT department_id) AS departments,
 SUM(salary) AS total_payroll,
 AVG(salary) AS avg_salary,
 MIN(salary) AS min_salary,
 MAX(salary) AS max_salary
FROM employees;

GROUP BY#

Groups rows sharing a value, then applies aggregate functions per group:

Apache HTTP Server

Mon, 01 Jan 0001 00:00:00 +0000

What Is Apache?#

Apache HTTP Server (commonly “Apache” or “httpd”) is the world’s most widely-used web server, maintained by the Apache Software Foundation since 1995. It serves static and dynamic content via a modular architecture with support for virtual hosts, URL rewriting, authentication, proxying, and more.

Apache vs Nginx#

Aspect	Apache	Nginx
Architecture	Process/thread per connection	Event-driven, async
Config style	`.htaccess` per-directory (flexible)	Centralized config (performant)
Dynamic content	`mod_php`, `mod_python` (in-process)	Proxies to external process (PHP-FPM)
Best for	Shared hosting, `.htaccess` flexibility	High concurrency, reverse proxying
Performance (static)	Good	Excellent
Performance (dynamic)	Excellent (mod_php)	Excellent (via proxy)

Installation#

# Debian/Ubuntu
sudo apt update && sudo apt install apache2

# RHEL/CentOS/Amazon Linux
sudo yum install httpd # or: sudo dnf install httpd

# macOS (built-in, or via Homebrew)
brew install httpd

# Start and enable
sudo systemctl start apache2 # Debian
sudo systemctl start httpd # RHEL
sudo systemctl enable apache2

Directory Structure#

Debian/Ubuntu#

/etc/apache2/
├── apache2.conf # Main config
├── ports.conf # Listen directives
├── sites-available/ # Virtual host configs (inactive)
├── sites-enabled/ # Symlinks to active vhosts
├── mods-available/ # Available modules
├── mods-enabled/ # Active modules (symlinks)
├── conf-available/ # Additional config fragments
└── conf-enabled/ # Active config fragments

RHEL/CentOS#

/etc/httpd/
├── conf/httpd.conf # Main config
├── conf.d/ # Additional configs (*.conf auto-loaded)
└── modules/ # Module .so files

Content and Logs#

/var/www/html/ # Default document root
/var/log/apache2/ # Debian logs
/var/log/httpd/ # RHEL logs

Virtual Hosts#

Virtual hosts let one Apache instance serve multiple websites:

Asynchronous JavaScript

Mon, 01 Jan 0001 00:00:00 +0000

Why Async Matters#

JavaScript is single-threaded — it has one call stack and can execute one thing at a time. Without asynchronous mechanisms, any I/O operation (network request, file read, timer) would freeze the entire program. The event loop solves this by enabling non-blocking concurrency on a single thread.

The Event Loop#

flowchart TD
 subgraph Runtime
 CS["Call Stack<br>(one frame at a time)"]
 WA["Web APIs / Node APIs<br>(setTimeout, fetch, fs)"]
 MQ["Microtask Queue<br>(Promise callbacks, queueMicrotask)"]
 TQ["Task Queue (Macrotask)<br>(setTimeout, setInterval, I/O)"]
 EL["Event Loop"]
 end
 
 CS -->|"Async call"| WA
 WA -->|"Callback ready"| TQ
 WA -->|"Promise resolved"| MQ
 EL -->|"Stack empty?"| MQ
 MQ -->|"Drain ALL microtasks"| CS
 EL -->|"Microtasks empty?"| TQ
 TQ -->|"ONE task"| CS

Execution Order Rules#

Synchronous code runs first (fills the call stack)
When the stack is empty, the event loop checks the microtask queue and drains it completely
After all microtasks are processed, one macrotask is dequeued and executed
After that macrotask, microtasks are drained again before the next macrotask

console.log("1 - sync");

setTimeout(() => console.log("2 - macrotask"), 0);

Promise.resolve().then(() => console.log("3 - microtask"));

queueMicrotask(() => console.log("4 - microtask"));

Promise.resolve().then(() => {
 console.log("5 - microtask");
 queueMicrotask(() => console.log("6 - nested microtask"));
});

console.log("7 - sync");

// Output: 1, 7, 3, 4, 5, 6, 2
// All sync first, then ALL microtasks (including nested), then macrotask

Microtasks vs Macrotasks#

Microtasks	Macrotasks
`Promise.then/catch/finally`	`setTimeout` / `setInterval`
`queueMicrotask()`	`setImmediate` (Node.js)
`MutationObserver`	I/O callbacks
`process.nextTick` (Node.js, even higher priority)	`requestAnimationFrame` (browser)

Callbacks#

The original async pattern — pass a function to be called when the operation completes.

Authentication

Mon, 01 Jan 0001 00:00:00 +0000

What Is Authentication?#

Authentication answers one question: who are you? It is the process of verifying that a claimed identity is genuine. Authentication is distinct from authorisation (what you can do), but a failure in authentication makes authorisation meaningless.

Password Storage#

The Golden Rule#

Never store passwords in plaintext. Never store them encrypted (encryption is reversible). Always store a one-way hash with a unique salt per password.

How Password Hashing Works#

User password: "correct horse battery staple"
 │
 ▼
┌───────────────────┐
│ Hashing function │ ← bcrypt, argon2, scrypt
│ + unique salt │
│ + cost factor │
└────────┬──────────┘
 ▼
Stored hash: "$2b$12$LJ3m4ys3Lg0Yf7KlZ8K0VePXdRkSEqGNmFCS.sVGBHby/RASY7Y/G"

Algorithm Comparison#

Algorithm	Type	Memory-Hard	Recommended	Notes
Argon2id	KDF	Yes	✅ Best choice	Winner of Password Hashing Competition (2015)
bcrypt	KDF	No	✅ Excellent	Battle-tested since 1999, 72-byte password limit
scrypt	KDF	Yes	✅ Good	Memory-hard, harder to tune than argon2
PBKDF2	KDF	No	⚠️ Acceptable	SHA-256 variant only; GPU-friendly
SHA-256	Hash	No	❌ Never	No salt, no cost factor, GPU-crackable
MD5	Hash	No	❌ Never	Broken, collisions found, GPU-crackable

Implementation#

Python (argon2):

Caching

Mon, 01 Jan 0001 00:00:00 +0000

Caching#

Caching stores copies of data in a faster storage layer so future requests can be served without hitting the slower origin. It is the single most impactful technique for reducing latency and database load in distributed systems.

The Cache Hierarchy#

Every request traverses multiple caching layers. Each layer trades freshness for speed.

flowchart TD
 Client[Client Request] --> Browser[Browser Cache]
 Browser -->|Miss| CDN[CDN Edge Cache]
 CDN -->|Miss| LB[Load Balancer]
 LB --> AppCache[Application Cache<br/>Redis / Memcached]
 AppCache -->|Miss| DB[Database Query Cache]
 DB -->|Miss| Disk[Disk / Storage]

 style Browser fill:#e8f5e1
 style CDN fill:#e8f5e1
 style AppCache fill:#e3f2fd
 style DB fill:#fff3e0

Layer	Latency	Scope	Controlled By
CPU cache (L1/L2/L3)	1–10 ns	Single process	Hardware
In-process cache	10–100 ns	Single instance	Application code
Browser cache	0 ms (local)	Single user	Cache-Control headers
CDN	5–50 ms	Geographic region	CDN config + headers
Application cache (Redis)	1–5 ms	Cluster-wide	Application code
Database query cache	1–10 ms	Database cluster	DB configuration
Database buffer pool	0.1–1 ms	Database instance	DB engine

Caching Strategies#

Cache-Aside (Lazy Loading)#

The application manages the cache explicitly.

Compute Patterns

Mon, 01 Jan 0001 00:00:00 +0000

Compute Patterns#

Choosing the right compute model is one of the highest-leverage architecture decisions you’ll make. It affects cost, operational complexity, scaling behaviour, and team velocity. This section provides frameworks for making that decision and patterns for getting the most out of each model.

Compute Decision Framework#

Every workload has different characteristics. Match the compute model to the workload, not the other way around.

What is the workload profile?
│
├── Stateless request/response, runs < 15 min
│ ├── < 1M requests/day → FaaS (Lambda / Cloud Functions)
│ └── > 1M requests/day → Containers (cost-optimised)
│
├── Long-running process, always-on
│ ├── Needs OS-level control → VMs
│ └── Doesn't need OS-level control → Containers
│
├── Batch processing, can tolerate interruption
│ └── Spot/preemptible instances (VMs or containers)
│
├── GPU/ML training or inference
│ ├── Training → GPU VMs or managed ML (SageMaker, Vertex AI)
│ └── Inference → GPU containers or serverless inference
│
└── Legacy app, can't be modified
 └── VMs (lift and shift)

Compute Model Comparison#

Factor	VMs	Containers	FaaS
Startup time	Minutes	Seconds	Milliseconds (warm) to seconds (cold)
Scaling granularity	Per VM	Per container	Per request
Min cost at idle	VM cost 24/7	Cluster cost 24/7	Zero (scale to zero)
Max control	Full OS access	User-space control	Code only
Operational burden	Patching, hardening, monitoring	Orchestration, image management	Near-zero ops
State handling	Local disk available	Ephemeral by design	Stateless
Cold start	N/A (always running)	N/A (always running)	100ms–10s (varies by runtime)
Cost at scale	Lower per-unit (reserved)	Lower per-unit (bin-packing)	Higher per-unit (per-invocation)

Auto-Scaling Patterns#

Auto-scaling adjusts compute capacity based on demand. The goal is to match capacity to load with minimal lag and minimal waste.

Consensus Algorithms

Mon, 01 Jan 0001 00:00:00 +0000

Consensus Algorithms#

Consensus is the problem of getting multiple nodes to agree on a single value, even when some nodes crash or messages are lost. It is the foundation of replicated state machines, leader election, distributed locks, and total order broadcast.

The FLP impossibility result (Section 1) proves that deterministic consensus is impossible in a purely asynchronous system. Practical algorithms sidestep this by assuming partial synchrony — the system is eventually timely enough for progress, even if delays occur.

Data Structures

Mon, 01 Jan 0001 00:00:00 +0000

Data structures organize data for efficient access and modification. The right structure can make the difference between an algorithm that runs in milliseconds and one that takes hours.

Choosing a Data Structure#

flowchart TD
 A["What operations do you need?"] --> B{"Ordered access?"}
 B -->|"By index"| C["Array / ArrayList"]
 B -->|"By key"| D["Hash Map"]
 B -->|"Sorted order"| E["Tree / Sorted Set"]
 B -->|"No"| F{"Access pattern?"}
 F -->|"FIFO"| G["Queue"]
 F -->|"LIFO"| H["Stack"]
 F -->|"Priority"| I["Heap / Priority Queue"]
 F -->|"Unique elements"| J["Set"]
 F -->|"Relationships"| K["Graph"]

Arrays and Dynamic Arrays#

Static Arrays#

Fixed-size, contiguous memory block. The most fundamental data structure.

Decision Trees & Ensembles

Mon, 01 Jan 0001 00:00:00 +0000

Decision Trees & Ensembles#

Decision trees are the foundation of the most powerful classical ML algorithms. A single tree is intuitive and interpretable but prone to overfitting. Ensemble methods — random forests and gradient boosting — combine many trees to achieve state-of-the-art performance on tabular data.

Decision Trees#

A decision tree recursively splits the feature space into regions, making predictions based on the majority class (classification) or mean value (regression) in each region.

Discrete Mathematics

Mon, 01 Jan 0001 00:00:00 +0000

Discrete Mathematics#

Discrete mathematics deals with countable, separated values — the natural habitat of computers, which process distinct symbols, not continuous quantities. Sets, relations, and combinatorics form the theoretical foundation of data structures, databases, and algorithms.

Sets#

A set is an unordered collection of distinct elements.

Notation#

A = {1, 2, 3, 4, 5} Roster notation
B = {x ∈ ℤ | x > 0 and x ≤ 5} Set-builder notation
∅ = {} Empty set

Set Operations#

Operation	Symbol	Definition	Example (A={1,2,3}, B={2,3,4})
Union	A ∪ B	Elements in A or B (or both)	{1, 2, 3, 4}
Intersection	A ∩ B	Elements in both A and B	{2, 3}
Difference	A \ B	Elements in A but not B	{1}
Symmetric difference	A △ B	Elements in exactly one of A or B	{1, 4}
Complement	Aᶜ	Elements not in A (relative to universe U)	U \ A
Cartesian product	A × B	All ordered pairs (a, b)	{(1,2),(1,3),(1,4),…}

Set Properties#

Property	Formula
Commutative	A ∪ B = B ∪ A, A ∩ B = B ∩ A
Associative	(A ∪ B) ∪ C = A ∪ (B ∪ C)
Distributive	A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C)
De Morgan’s	(A ∪ B)ᶜ = Aᶜ ∩ Bᶜ, (A ∩ B)ᶜ = Aᶜ ∪ Bᶜ
Idempotent	A ∪ A = A, A ∩ A = A
Identity	A ∪ ∅ = A, A ∩ U = A

In Programming#

A = {1, 2, 3}
B = {2, 3, 4}

A | B # {1, 2, 3, 4} union
A & B # {2, 3} intersection
A - B # {1} difference
A ^ B # {1, 4} symmetric difference
A <= B # False subset check

SQL SELECT with UNION, INTERSECT, EXCEPT maps directly to set operations.

Distributed Tracing

Mon, 01 Jan 0001 00:00:00 +0000

Distributed Tracing#

Distributed tracing follows individual requests as they propagate through multiple services. When a user hits “Place Order” and that request touches an API gateway, order service, inventory service, payment service, and notification service, a trace captures the entire journey — showing where time is spent, which calls failed, and how services depend on each other.

Core Concepts#

Traces and Spans#

A trace represents the end-to-end journey of a single request. It consists of one or more spans.

Domain-Driven Design

Mon, 01 Jan 0001 00:00:00 +0000

Domain-Driven Design#

Domain-Driven Design (DDD) is an approach to software development that centers the design on the core business domain. It provides strategic patterns for organizing large systems and tactical patterns for modeling complex business logic.

Strategic DDD#

Strategic DDD addresses the big picture — how to decompose a large system into manageable parts that can evolve independently.

Ubiquitous Language#

The most fundamental concept in DDD: a shared language between developers and domain experts that is used consistently in code, conversations, and documentation.

ETL & ELT Pipelines

Mon, 01 Jan 0001 00:00:00 +0000

ETL vs ELT#

The two dominant pipeline paradigms differ in where transformation happens relative to loading.

ETL (Extract, Transform, Load)#

Data is extracted from sources, transformed in a processing engine (Spark, Python, custom code), and then loaded into the target system.

Source ──▶ Extract ──▶ Transform (Spark/Python) ──▶ Load ──▶ Warehouse

When to use: transformations require compute-heavy logic, data cleansing before landing, legacy systems that can’t handle raw data, or when the warehouse lacks transformation power.

Functions

Mon, 01 Jan 0001 00:00:00 +0000

Functions#

Functions in Go are first-class values. They can be assigned to variables, passed as arguments, returned from other functions, and defined inline as closures. Go functions support multiple return values — the foundation of Go’s error handling model.

Basic Functions#

func add(a int, b int) int {
 return a + b
}

// Shortened: consecutive params of same type
func add(a, b int) int {
 return a + b
}

Multiple Return Values#

Go functions can return multiple values. The dominant pattern is (result, error):

Functions

Mon, 01 Jan 0001 00:00:00 +0000

Defining Functions#

# Standard syntax
greet() {
 local name="${1:?Error: name required}"
 local greeting="${2:-Hello}"
 echo "${greeting}, ${name}!"
}

# Alternative syntax (less common)
function greet {
 echo "Hello, $1"
}

The function keyword is optional in bash. Prefer the name() syntax — it’s POSIX-compatible.

Parameters and Return Values#

Positional Parameters#

create_user() {
 local username="$1"
 local email="$2"
 local role="${3:-viewer}" # default value
 
 echo "Creating $username ($email) with role: $role"
}

create_user "alice" "alice@example.com" "admin"
create_user "bob" "bob@example.com" # role defaults to "viewer"

Parameter Validation#

deploy() {
 local env="${1:?Error: environment required (test|pre|prod)}"
 local version="${2:?Error: version required}"
 
 [[ "$env" =~ ^(test|pre|prod)$ ]] || {
 echo "Invalid environment: $env" >&2
 return 1
 }
 
 echo "Deploying v$version to $env"
}

Return Values#

Functions return exit codes (0-255), not data. Use echo + command substitution for data:

GitLab CI

Mon, 01 Jan 0001 00:00:00 +0000

GitLab CI#

GitLab CI/CD is GitLab’s built-in continuous integration and delivery system. Configuration lives in a .gitlab-ci.yml file at the repository root, defining pipelines as code that execute on every push, merge request, or schedule.

.gitlab-ci.yml Structure#

A minimal pipeline:

stages:
 - build
 - test
 - deploy

build-app:
 stage: build
 image: node:20-alpine
 script:
 - npm ci
 - npm run build
 artifacts:
 paths:
 - dist/

test-app:
 stage: test
 image: node:20-alpine
 script:
 - npm ci
 - npm test
 coverage: '/Lines\s*:\s*(\d+\.?\d*)%/'

deploy-staging:
 stage: deploy
 script:
 - deploy-to-staging.sh
 environment:
 name: staging
 url: https://staging.example.com
 rules:
 - if: $CI_COMMIT_BRANCH == "main"

Top-Level Keywords#

Keyword	Purpose	Example
`stages`	Define pipeline stages and their order	`[lint, test, build, deploy]`
`variables`	Global environment variables	`NODE_ENV: production`
`default`	Default values for all jobs	`image`, `before_script`, `tags`
`include`	Import external CI configuration	Templates, shared pipelines
`workflow`	Control when entire pipeline runs	Filter by branch, variable

Stages, Jobs, and Artifacts#

Stages#

Stages run sequentially. All jobs within a stage run in parallel by default:

Integration Testing

Mon, 01 Jan 0001 00:00:00 +0000

Integration Testing#

Unit tests verify logic in isolation. Integration tests verify that components work together — your code with a real database, your service with a real HTTP API, your function with the real filesystem. This section covers strategies for testing these interactions reliably.

What Integration Tests Cover#

graph LR
 subgraph "Unit Test Scope"
 A[Your Code]
 end
 subgraph "Integration Test Scope"
 B[Your Code] --> C[Database]
 B --> D[External API]
 B --> E[Message Queue]
 B --> F[File System]
 end

Aspect	Unit Test	Integration Test
Dependencies	Mocked/faked	Real (or realistic containers)
Speed	Milliseconds	Seconds
What it catches	Logic errors	Wiring errors, serialization bugs, query bugs
Reliability	Deterministic	May flake if external systems are unstable
Setup complexity	Low	Medium–High

What Integration Tests Catch That Unit Tests Miss#

SQL syntax errors and wrong column names
Serialization/deserialization mismatches (JSON ↔ object)
HTTP header issues, content-type mismatches
Transaction isolation problems
Connection pool exhaustion
File permission issues
Environment variable misconfiguration

Database Testing#

Strategy 1: Testcontainers (Recommended)#

Testcontainers spins up real databases in Docker for your tests:

Java: Generics and Type Erasure

Mon, 01 Jan 0001 00:00:00 +0000

Why Generics#

// Without generics — no type safety, requires casting
List rawList = new ArrayList();
rawList.add("hello");
rawList.add(42); // No compile error!
String s = (String) rawList.get(1); // ClassCastException at RUNTIME

// With generics — compile-time type safety
List<String> typedList = new ArrayList<>();
typedList.add("hello");
// typedList.add(42); // Compile error!
String s = typedList.get(0); // No cast needed

Generic Classes#

public class Pair<A, B> {
 private final A first;
 private final B second;
 
 public Pair(A first, B second) {
 this.first = first;
 this.second = second;
 }
 
 public A first() { return first; }
 public B second() { return second; }
 
 public <C> Pair<C, B> mapFirst(Function<A, C> mapper) {
 return new Pair<>(mapper.apply(first), second);
 }
 
 @Override
 public String toString() {
 return "(%s, %s)".formatted(first, second);
 }
}

// Usage
Pair<String, Integer> nameAge = new Pair<>("Alice", 30);
Pair<Integer, Integer> coordinates = new Pair<>(10, 20);

Generic Interface with Multiple Implementations#

public interface Repository<T, ID> {
 Optional<T> findById(ID id);
 List<T> findAll();
 T save(T entity);
 void deleteById(ID id);
 boolean existsById(ID id);
}

public class InMemoryUserRepository implements Repository<User, String> {
 private final Map<String, User> store = new ConcurrentHashMap<>();
 
 @Override
 public Optional<User> findById(String id) {
 return Optional.ofNullable(store.get(id));
 }
 
 @Override
 public List<User> findAll() {
 return List.copyOf(store.values());
 }
 
 @Override
 public User save(User user) {
 store.put(user.id(), user);
 return user;
 }
 
 @Override
 public void deleteById(String id) {
 store.remove(id);
 }
 
 @Override
 public boolean existsById(String id) {
 return store.containsKey(id);
 }
}

Bounded Type Parameters#

// Upper bound: T must be Comparable
public static <T extends Comparable<T>> T max(T a, T b) {
 return a.compareTo(b) >= 0 ? a : b;
}

max(3, 7); // 7
max("apple", "banana"); // "banana"

// Multiple bounds: T must implement BOTH interfaces
public static <T extends Comparable<T> & Serializable> void process(T item) {
 // Can use both Comparable and Serializable methods
}

// Bounded in class definition
public class SortedList<T extends Comparable<T>> {
 private final List<T> items = new ArrayList<>();
 
 public void add(T item) {
 int index = Collections.binarySearch(items, item);
 if (index < 0) index = -(index + 1);
 items.add(index, item);
 }
 
 public List<T> asList() {
 return Collections.unmodifiableList(items);
 }
}

Wildcards#

graph TD
 A["Object"] --> B["Number"]
 B --> C["Integer"]
 B --> D["Double"]
 B --> E["Long"]
 
 style A fill:#e1f5fe
 style B fill:#fff3e0
 style C fill:#c8e6c9
 style D fill:#c8e6c9
 style E fill:#c8e6c9

// Unbounded wildcard: ? — read-only, any type
void printAll(List<?> list) {
 for (Object item : list) {
 System.out.println(item);
 }
 // list.add(???); // Can't add — don't know the type!
}

// Upper bounded: ? extends Number — "producer" (read from)
double sum(List<? extends Number> numbers) {
 double total = 0;
 for (Number n : numbers) {
 total += n.doubleValue(); // Can read as Number
 }
 // numbers.add(42); // Can't add — might be List<Double>!
 return total;
}

sum(List.of(1, 2, 3)); // Works with List<Integer>
sum(List.of(1.5, 2.5)); // Works with List<Double>

// Lower bounded: ? super Integer — "consumer" (write to)
void addNumbers(List<? super Integer> list) {
 list.add(1); // Can add Integer (or subtype)
 list.add(2);
 // Integer x = list.get(0); // Can't read as Integer — might be List<Object>!
 Object x = list.get(0); // Can only read as Object
}

addNumbers(new ArrayList<Integer>()); // OK
addNumbers(new ArrayList<Number>()); // OK
addNumbers(new ArrayList<Object>()); // OK

PECS: Producer Extends, Consumer Super#

// Copy from source (producer) to destination (consumer)
public static <T> void copy(List<? extends T> source, List<? super T> dest) {
 for (T item : source) {
 dest.add(item);
 }
}

List<Integer> ints = List.of(1, 2, 3);
List<Number> nums = new ArrayList<>();
copy(ints, nums); // Integer extends Number, Number super Integer

Type Erasure#

What Happens at Runtime#

// What you write:
List<String> strings = new ArrayList<>();
List<Integer> integers = new ArrayList<>();

// What the JVM sees (after erasure):
List strings = new ArrayList(); // Generic type info is GONE
List integers = new ArrayList();

// Consequence:
strings.getClass() == integers.getClass(); // true! Both are ArrayList

// You CANNOT do:
// if (obj instanceof List<String>) {} // Compile error
// new T() // Compile error
// new T[] // Compile error
// T.class // Compile error

Working Around Erasure#

// Problem: can't create generic arrays
// T[] array = new T[10]; // Won't compile

// Solution 1: pass Class<T> token
public class TypeSafeContainer<T> {
 private final Class<T> type;
 private final Object[] storage;
 private int size = 0;
 
 public TypeSafeContainer(Class<T> type, int capacity) {
 this.type = type;
 this.storage = new Object[capacity];
 }
 
 public void add(T item) {
 storage[size++] = item;
 }
 
 public T get(int index) {
 return type.cast(storage[index]); // Safe cast using Class token
 }
}

// Solution 2: use @SuppressWarnings with documented reason
@SuppressWarnings("unchecked") // Safe: array never exposed outside class
T[] createArray(int size) {
 return (T[]) new Object[size];
}

Erasure and Overloading#

// This WON'T compile — both erase to process(List)
// void process(List<String> strings) { }
// void process(List<Integer> integers) { }

// Fix: use different method names or a single generic method
<T> void process(List<T> items, Class<T> type) { }

Generic Methods#

public class Collections {
 
 // Generic method — type parameter declared before return type
 public static <T> List<T> singletonList(T item) {
 return List.of(item);
 }
 
 // Multiple type parameters
 public static <K, V> Map<K, V> mapOf(K key, V value) {
 return Map.of(key, value);
 }
 
 // Bounded generic method
 public static <T extends Comparable<T>> Optional<T> min(Collection<T> items) {
 return items.stream().min(Comparable::compareTo);
 }
 
 // Recursive type bound (self-referential)
 public static <T extends Comparable<T>> void sort(List<T> list) {
 list.sort(Comparable::compareTo);
 }
}

// Type inference — compiler figures out T
List<String> list = Collections.singletonList("hello"); // T = String
Optional<Integer> min = Collections.min(List.of(3, 1, 4)); // T = Integer

Hypothetical Use Case: Type-Safe Builder#

public class QueryBuilder<T> {
 private final Class<T> entityType;
 private final List<String> conditions = new ArrayList<>();
 private String orderBy;
 private int limit = -1;
 
 private QueryBuilder(Class<T> entityType) {
 this.entityType = entityType;
 }
 
 public static <T> QueryBuilder<T> from(Class<T> entityType) {
 return new QueryBuilder<>(entityType);
 }
 
 public QueryBuilder<T> where(String condition) {
 conditions.add(condition);
 return this;
 }
 
 public QueryBuilder<T> orderBy(String field) {
 this.orderBy = field;
 return this;
 }
 
 public QueryBuilder<T> limit(int limit) {
 this.limit = limit;
 return this;
 }
 
 public Query<T> build() {
 return new Query<>(entityType, conditions, orderBy, limit);
 }
}

// Usage — fully type-safe
Query<User> query = QueryBuilder.from(User.class)
 .where("age > 18")
 .where("active = true")
 .orderBy("name")
 .limit(50)
 .build();

List<User> users = database.execute(query); // Returns List<User>, not List<Object>

Key Takeaways#

Generics provide compile-time type safety — catch errors before runtime
Type erasure means generic info is gone at runtime — can’t do instanceof List<String>
PECS — Producer Extends, Consumer Super (read from extends, write to super)
Bounded types (<T extends X>) restrict what types can be used
Wildcards (?) are for flexibility in method parameters
Pass Class<T> when you need type info at runtime (erasure workaround)
Prefer generic methods over raw types — never use raw List, always List<T>

Linux Basics

Mon, 01 Jan 0001 00:00:00 +0000

Why Linux?#

Linux runs ~96% of cloud servers, all Android phones, most IoT devices, and the world’s top supercomputers. For software engineers, Linux fluency is non-negotiable.

Essential Commands#

pwd # print working directory
ls -la # list all files with details
cd /path/to/dir # change directory
cd ~ # go to home directory
cd - # go to previous directory

mkdir -p path/to/dir # create directories (with parents)
touch file.txt # create empty file
cp -r source/ dest/ # copy (recursive)
mv old new # move/rename
rm -r directory/ # remove (recursive)

Viewing Files#

cat file.txt # print entire file
less file.txt # paginated viewer (q to quit)
head -20 file.txt # first 20 lines
tail -20 file.txt # last 20 lines
tail -f /var/log/app.log # follow (live updates)
wc -l file.txt # count lines

Searching#

find . -name "*.py" # find files by name
grep -r "pattern" dir/ # search file contents
which command # find command location
locate filename # fast search (uses index)

Users and Permissions#

User Management#

whoami # current user
id # user ID, groups
sudo command # run as root
su - username # switch user

File Permissions#

-rwxr-xr-- 1 alice devs 4096 Jan 1 12:00 script.sh
 │││ │││ │││
 │││ │││ └── Others: read only
 │││ └──── Group: read + execute
 └────── Owner: read + write + execute

chmod 755 script.sh # rwxr-xr-x
chmod 644 file.txt # rw-r--r--
chown alice:devs file # change owner:group

Package Management#

Distro Family	Tool	Commands
Debian/Ubuntu	apt	`apt update`, `apt install pkg`, `apt remove pkg`
RHEL/CentOS/Amazon	yum/dnf	`yum install pkg`, `dnf update`
Alpine	apk	`apk add pkg`, `apk del pkg`

# Debian/Ubuntu
sudo apt update # refresh package index
sudo apt install nginx # install
sudo apt remove nginx # remove
sudo apt upgrade # upgrade all packages

# RHEL/Amazon Linux
sudo yum install nginx
sudo yum update

Service Management (systemd)#

# Service control
systemctl start nginx # start service
systemctl stop nginx # stop service
systemctl restart nginx # restart
systemctl status nginx # check status
systemctl enable nginx # start on boot
systemctl disable nginx # don't start on boot

# View logs
journalctl -u nginx # logs for a service
journalctl -u nginx --since today
journalctl -f # follow all logs

Process Management#

ps aux # all processes
ps aux | grep nginx # find specific process
top # real-time process monitor
htop # better process monitor

kill PID # graceful stop (SIGTERM)
kill -9 PID # force kill (SIGKILL)
pkill -f "pattern" # kill by name pattern

Networking Commands#

ip addr show # show IP addresses
ip route show # show routing table
ss -tlnp # listening TCP ports
curl -v http://localhost # HTTP request
ping host # test connectivity
dig domain.com # DNS lookup

Environment Variables#

echo $PATH # view variable
export MY_VAR="value" # set for current session
env # list all env vars

# Persistent (add to ~/.bashrc or ~/.profile)
echo 'export MY_VAR="value"' >> ~/.bashrc
source ~/.bashrc # reload

SSH (Secure Shell)#

# Connect to remote server
ssh user@hostname
ssh -i ~/.ssh/key.pem user@host # with private key

# Copy files
scp file.txt user@host:/path/ # local → remote
scp user@host:/path/file.txt . # remote → local

# SSH key generation
ssh-keygen -t ed25519 -C "email@example.com"
# Public key → ~/.ssh/id_ed25519.pub (share this)
# Private key → ~/.ssh/id_ed25519 (keep secret, chmod 600)

Key Takeaways#

Learn the core commands — ls, cd, grep, find, chmod, ps, systemctl
Everything is a file in Linux — config, devices, processes
systemd manages services — systemctl and journalctl are essential
SSH is how you access remote servers — use key-based auth, never passwords
Package managers vary by distro — apt (Debian), yum/dnf (RHEL), apk (Alpine)
Environment variables configure applications — export for the session, .bashrc for persistence

Object-Oriented Kotlin

Mon, 01 Jan 0001 00:00:00 +0000

Object-Oriented Kotlin#

Kotlin builds on Java’s object-oriented foundation but strips away ceremony and adds powerful features like data classes, sealed hierarchies, object declarations, and delegation. The result is a concise, expressive class system that favours composition over inheritance.

Classes and Constructors#

Kotlin classes have a primary constructor in the class header and can have secondary constructors in the body:

class User(val name: String, val email: String, var isActive: Boolean = true) {
 // Properties declared in the primary constructor
 // val = read-only, var = mutable

 init {
 require(name.isNotBlank()) { "Name must not be blank" }
 require(email.contains("@")) { "Invalid email" }
 }

 // Secondary constructor
 constructor(name: String) : this(name, "$name@example.com")
}

val user = User("Alice", "alice@example.com")
println(user.name) // Alice
println(user.isActive) // true

Properties and Accessors#

Properties can have custom getters and setters:

Object-Oriented PHP

Mon, 01 Jan 0001 00:00:00 +0000

Classes and Objects#

<?php
class User
{
 // Properties with visibility and types
 private string $name;
 private string $email;
 private readonly int $id; // readonly (PHP 8.1+) — set once, never changed

 // Constructor with promotion (PHP 8.0+)
 public function __construct(
 private string $name,
 private string $email,
 private readonly int $id = 0,
 ) {}

 // Methods
 public function getDisplayName(): string
 {
 return "{$this->name} <{$this->email}>";
 }

 // Static methods
 public static function createGuest(): self
 {
 return new self('Guest', 'guest@example.com');
 }
}

$user = new User(name: 'Alice', email: 'alice@example.com', id: 1);
$guest = User::createGuest();

Constructor Property Promotion (PHP 8.0+)#

Eliminates the boilerplate of declaring properties, assigning in constructor:

<?php
// Before PHP 8.0 (verbose)
class Point {
 private float $x;
 private float $y;
 
 public function __construct(float $x, float $y) {
 $this->x = $x;
 $this->y = $y;
 }
}

// PHP 8.0+ (promoted — property declaration, type, and assignment in one)
class Point {
 public function __construct(
 private float $x,
 private float $y,
 ) {}
}

Visibility#

Modifier	Access
`public`	Anywhere
`protected`	Class + subclasses
`private`	Class only

PHP 8.4 introduces asymmetric visibility:

Offline-First Architecture

Mon, 01 Jan 0001 00:00:00 +0000

The Offline-First Mindset#

Offline-first design treats the network as an enhancement rather than a requirement. Instead of building for connectivity and then patching in offline support, you design the application to work without a network from the start. The network, when available, synchronises local state with the server.

This approach matters because network reliability is a spectrum, not a binary. Users encounter:

Scenario	Reality
Underground trains, lifts, aeroplanes	Zero connectivity
Conference centres, stadiums	Congested, unreliable
Rural areas, emerging markets	Slow, intermittent
Office Wi-Fi switching to mobile data	Brief disconnection during handoff

An offline-first PWA handles all of these gracefully.

OpenAPI & Documentation

Mon, 01 Jan 0001 00:00:00 +0000

OpenAPI & Documentation#

A well-documented API is a usable API. The OpenAPI Specification (formerly Swagger) provides a machine-readable description of your API that powers documentation, code generation, testing, and validation.

What is OpenAPI?#

OpenAPI is a standard for describing REST APIs. It’s a YAML or JSON document that describes every endpoint, parameter, request body, response, and authentication mechanism your API offers.

OpenAPI Document Structure#

openapi: 3.1.0
info:
 title: Orders API
 description: API for managing customer orders
 version: 1.2.0
 contact:
 name: Platform Team
 email: platform@example.com

servers:
 - url: https://api.example.com/v1
 description: Production
 - url: https://staging-api.example.com/v1
 description: Staging

paths:
 /orders:
 get: ...
 post: ...
 /orders/{id}:
 get: ...
 put: ...
 delete: ...

components:
 schemas: ...
 securitySchemes: ...

security:
 - bearerAuth: []

Key Sections#

Section	Purpose	Contains
`info`	API metadata	Title, version, description, contact
`servers`	Base URLs	Production, staging, dev environments
`paths`	Endpoints	All routes with operations
`components`	Reusable definitions	Schemas, parameters, security, responses
`security`	Global auth requirements	Default security schemes
`tags`	Grouping	Logical grouping of operations

Schema Definition#

Schemas define the shape of request and response bodies:

Python: The Standard Library

Mon, 01 Jan 0001 00:00:00 +0000

File I/O and Pathlib#

pathlib — Object-Oriented Filesystem Paths#

from pathlib import Path

# Creating paths (OS-independent)
home = Path.home() # /Users/username
project = Path.cwd() / "src" / "main.py" # Operator / joins paths
config = Path("/etc/nginx/nginx.conf")

# Path inspection
project.name # "main.py"
project.stem # "main"
project.suffix # ".py"
project.parent # PosixPath('/Users/.../src')
project.parts # ('/', 'Users', '...', 'src', 'main.py')
project.is_absolute() # True

# File operations
path = Path("data/output.json")
path.parent.mkdir(parents=True, exist_ok=True) # Create directories
path.write_text('{"key": "value"}', encoding="utf-8")
content = path.read_text(encoding="utf-8")
path.write_bytes(b'\x00\x01\x02')

# Globbing
for py_file in Path("src").rglob("*.py"): # Recursive
 print(py_file)

# Temporary files
import tempfile
with tempfile.NamedTemporaryFile(suffix=".json", delete=False) as f:
 f.write(b'{"temp": true}')
 temp_path = Path(f.name)

File I/O Patterns#

# Reading large files line by line (memory efficient)
def count_errors(log_path: Path) -> int:
 count = 0
 with open(log_path, "r", encoding="utf-8") as f:
 for line in f: # Lazy iteration — one line in memory at a time
 if "ERROR" in line:
 count += 1
 return count

# Binary file handling
def copy_file(src: Path, dst: Path, chunk_size: int = 8192) -> None:
 with open(src, "rb") as fsrc, open(dst, "wb") as fdst:
 while chunk := fsrc.read(chunk_size):
 fdst.write(chunk)

Collections Module#

from collections import (
 defaultdict, Counter, deque, OrderedDict, ChainMap, namedtuple
)

# defaultdict — auto-creates missing keys
word_index = defaultdict(list)
for i, word in enumerate(words):
 word_index[word].append(i) # No KeyError if word is new

graph = defaultdict(set)
graph["A"].add("B")
graph["A"].add("C")

# Counter — frequency counting
text = "the quick brown fox jumps over the lazy dog"
word_counts = Counter(text.split())
word_counts.most_common(3) # [('the', 2), ('quick', 1), ('brown', 1)]

# Arithmetic on Counters
inventory = Counter(apples=5, oranges=3)
sold = Counter(apples=2, oranges=1)
remaining = inventory - sold # Counter({'apples': 3, 'oranges': 2})

# deque — O(1) append/pop from both ends
from collections import deque
buffer = deque(maxlen=100) # Fixed-size circular buffer
buffer.append("event_1")
buffer.appendleft("priority_event")
buffer.popleft() # O(1) vs list.pop(0) which is O(n)

# ChainMap — layered dictionaries (first match wins)
defaults = {"color": "blue", "size": "medium", "debug": False}
user_prefs = {"color": "red"}
env_config = {"debug": True}
config = ChainMap(env_config, user_prefs, defaults)
config["color"] # "red" (from user_prefs)
config["debug"] # True (from env_config)
config["size"] # "medium" (from defaults)

itertools — Efficient Iteration#

import itertools

# Infinite iterators
counter = itertools.count(start=1, step=2) # 1, 3, 5, 7, ...
cycler = itertools.cycle(["red", "green", "blue"]) # repeats forever
repeater = itertools.repeat("hello", times=3) # "hello" x3

# Combinatorics
list(itertools.permutations([1, 2, 3], r=2))
# [(1,2), (1,3), (2,1), (2,3), (3,1), (3,2)]

list(itertools.combinations([1, 2, 3, 4], r=2))
# [(1,2), (1,3), (1,4), (2,3), (2,4), (3,4)]

list(itertools.product("AB", "12"))
# [('A','1'), ('A','2'), ('B','1'), ('B','2')]

# Grouping
data = [
 {"dept": "eng", "name": "Alice"},
 {"dept": "eng", "name": "Bob"},
 {"dept": "sales", "name": "Charlie"},
 {"dept": "sales", "name": "Diana"},
]
# MUST be sorted by key first!
data.sort(key=lambda x: x["dept"])
for dept, members in itertools.groupby(data, key=lambda x: x["dept"]):
 print(f"{dept}: {[m['name'] for m in members]}")

# Chain — flatten iterables
all_items = list(itertools.chain([1, 2], [3, 4], [5, 6])) # [1,2,3,4,5,6]
all_items = list(itertools.chain.from_iterable(list_of_lists))

# Slicing iterators
first_10 = list(itertools.islice(infinite_gen(), 10))

# Accumulate (running totals)
list(itertools.accumulate([1, 2, 3, 4, 5])) # [1, 3, 6, 10, 15]
list(itertools.accumulate([3, 1, 4, 1, 5], max)) # [3, 3, 4, 4, 5]

functools — Higher-Order Functions#

from functools import lru_cache, cache, reduce, wraps, total_ordering

# Caching expensive computations
@lru_cache(maxsize=128)
def fibonacci(n: int) -> int:
 if n < 2:
 return n
 return fibonacci(n - 1) + fibonacci(n - 2)

fibonacci(100) # Instant — without cache this would take forever
fibonacci.cache_info() # CacheInfo(hits=98, misses=101, maxsize=128, currsize=101)

# @cache — unlimited cache (Python 3.9+)
@cache
def expensive_lookup(key: str) -> dict:
 # Simulates slow database/API call
 return fetch_from_database(key)

# reduce — fold a sequence into a single value
from functools import reduce
product = reduce(lambda a, b: a * b, [1, 2, 3, 4, 5]) # 120

# total_ordering — generate comparison methods from __eq__ and one other
@total_ordering
class Version:
 def __init__(self, major, minor, patch):
 self.major, self.minor, self.patch = major, minor, patch
 
 def __eq__(self, other):
 return (self.major, self.minor, self.patch) == (other.major, other.minor, other.patch)
 
 def __lt__(self, other):
 return (self.major, self.minor, self.patch) < (other.major, other.minor, other.patch)

# Now Version also has __le__, __gt__, __ge__ automatically

datetime and Time#

from datetime import datetime, date, timedelta, timezone
from zoneinfo import ZoneInfo # Python 3.9+

# Current time
now = datetime.now() # Naive (no timezone)
now_utc = datetime.now(timezone.utc) # Aware (UTC)
now_madrid = datetime.now(ZoneInfo("Europe/Madrid"))

# Parsing and formatting
dt = datetime.fromisoformat("2026-05-29T12:00:00+02:00")
dt.strftime("%Y-%m-%d %H:%M:%S %Z") # "2026-05-29 12:00:00 CEST"

# Arithmetic
tomorrow = now + timedelta(days=1)
next_week = now + timedelta(weeks=1)
duration = datetime(2026, 12, 31) - datetime(2026, 1, 1)
duration.days # 364

# Timezone conversion
utc_time = datetime(2026, 5, 29, 10, 0, tzinfo=timezone.utc)
tokyo_time = utc_time.astimezone(ZoneInfo("Asia/Tokyo"))
# 2026-05-29 19:00:00+09:00

JSON, CSV, and Data Serialization#

import json
import csv
from dataclasses import dataclass, asdict

# JSON with custom serialization
class CustomEncoder(json.JSONEncoder):
 def default(self, obj):
 if isinstance(obj, datetime):
 return obj.isoformat()
 if isinstance(obj, set):
 return list(obj)
 if hasattr(obj, "__dict__"):
 return obj.__dict__
 return super().default(obj)

data = {"timestamp": datetime.now(), "tags": {"python", "tutorial"}}
json.dumps(data, cls=CustomEncoder, indent=2)

# CSV reading/writing
def read_csv_as_dicts(path: Path) -> list[dict]:
 with open(path, newline="", encoding="utf-8") as f:
 reader = csv.DictReader(f)
 return list(reader)

def write_csv(path: Path, rows: list[dict]) -> None:
 if not rows:
 return
 with open(path, "w", newline="", encoding="utf-8") as f:
 writer = csv.DictWriter(f, fieldnames=rows[0].keys())
 writer.writeheader()
 writer.writerows(rows)

subprocess — Running External Commands#

import subprocess

# Simple command execution
result = subprocess.run(
 ["git", "status", "--short"],
 capture_output=True,
 text=True,
 check=True, # Raises CalledProcessError on non-zero exit
 timeout=30,
)
print(result.stdout)

# Piping commands
ps = subprocess.run(["ps", "aux"], capture_output=True, text=True)
grep = subprocess.run(
 ["grep", "python"],
 input=ps.stdout,
 capture_output=True,
 text=True,
)

# Streaming output
process = subprocess.Popen(
 ["tail", "-f", "/var/log/app.log"],
 stdout=subprocess.PIPE,
 text=True,
)
for line in process.stdout:
 if "ERROR" in line:
 alert(line)

logging — Structured Logging#

import logging
import logging.handlers
import json

# Basic configuration
logging.basicConfig(
 level=logging.INFO,
 format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
 datefmt="%Y-%m-%d %H:%M:%S",
)

# Module-level logger (best practice)
logger = logging.getLogger(__name__)

# Structured JSON logging
class JSONFormatter(logging.Formatter):
 def format(self, record):
 log_data = {
 "timestamp": self.formatTime(record),
 "level": record.levelname,
 "logger": record.name,
 "message": record.getMessage(),
 }
 if record.exc_info:
 log_data["exception"] = self.formatException(record.exc_info)
 if hasattr(record, "extra_data"):
 log_data.update(record.extra_data)
 return json.dumps(log_data)

# Rotating file handler
handler = logging.handlers.RotatingFileHandler(
 "app.log", maxBytes=10_000_000, backupCount=5
)
handler.setFormatter(JSONFormatter())
logger.addHandler(handler)

# Usage with context
logger.info("Order processed", extra={"extra_data": {"order_id": "ORD-123", "total": 99.99}})

re — Regular Expressions#

import re

# Common patterns
email_pattern = re.compile(r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}")
ip_pattern = re.compile(r"\b(?:\d{1,3}\.){3}\d{1,3}\b")
url_pattern = re.compile(r"https?://[^\s<>\"]+")

# Named groups
log_pattern = re.compile(
 r"(?P<timestamp>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})"
 r" \[(?P<level>\w+)\]"
 r" (?P<message>.*)"
)

line = "2026-05-29 12:00:00 [ERROR] Connection timeout"
match = log_pattern.match(line)
if match:
 match.group("level") # "ERROR"
 match.group("message") # "Connection timeout"
 match.groupdict() # {"timestamp": "...", "level": "ERROR", "message": "..."}

# Substitution
cleaned = re.sub(r"\s+", " ", messy_text) # Collapse whitespace
redacted = re.sub(r"\b\d{4}-\d{4}-\d{4}-\d{4}\b", "XXXX-XXXX-XXXX-XXXX", text)

# findall vs finditer
emails = re.findall(email_pattern, document) # List of all matches
for match in re.finditer(email_pattern, document): # Iterator of Match objects
 print(f"Found {match.group()} at position {match.start()}")

Hypothetical Use Cases#

Use Case: Log Analyzer#

from pathlib import Path
from collections import Counter, defaultdict
from datetime import datetime
import re

def analyze_logs(log_path: Path) -> dict:
 """Parse and analyze application logs."""
 pattern = re.compile(
 r"(?P<ts>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2})"
 r" (?P<level>DEBUG|INFO|WARN|ERROR|FATAL)"
 r" \[(?P<service>[\w-]+)\]"
 r" (?P<msg>.*)"
 )
 
 level_counts = Counter()
 errors_by_service = defaultdict(list)
 hourly_distribution = Counter()
 
 with open(log_path, encoding="utf-8") as f:
 for line in f:
 if m := pattern.match(line):
 level = m.group("level")
 service = m.group("service")
 ts = datetime.fromisoformat(m.group("ts"))
 
 level_counts[level] += 1
 hourly_distribution[ts.hour] += 1
 
 if level in ("ERROR", "FATAL"):
 errors_by_service[service].append({
 "timestamp": ts,
 "message": m.group("msg"),
 })
 
 return {
 "total_lines": sum(level_counts.values()),
 "level_distribution": dict(level_counts),
 "peak_hour": hourly_distribution.most_common(1)[0],
 "error_services": {
 svc: len(errs) for svc, errs in errors_by_service.items()
 },
 }

Key Takeaways#

pathlib replaces os.path — use / operator for path joining
defaultdict eliminates key-existence checks; Counter handles frequency counting
itertools provides memory-efficient iteration — never build full lists when you can iterate
lru_cache gives free memoization for pure functions
Always use encoding="utf-8" explicitly in file operations
logging over print — configure once at application entry point, use getLogger(__name__) everywhere
subprocess.run with check=True and capture_output=True is the modern way to run commands

Responsive Design

Mon, 01 Jan 0001 00:00:00 +0000

The Responsive Mindset#

Responsive design means building interfaces that adapt to any screen size, orientation, and input method. The modern approach is mobile-first — start with the smallest screen and progressively enhance.

flowchart LR
 M["Mobile<br>(320-480px)"] -->|"Add complexity"| T["Tablet<br>(768-1024px)"]
 T -->|"Add complexity"| D["Desktop<br>(1024px+)"]
 
 style M fill:#c8e6c9
 style T fill:#fff3e0
 style D fill:#e1f5fe

The Viewport Meta Tag#

Without this, mobile browsers render pages at desktop width and zoom out:

<meta name="viewport" content="width=device-width, initial-scale=1.0">

Property	Effect
`width=device-width`	Match viewport to device width
`initial-scale=1.0`	No zoom on load
`user-scalable=no`	NEVER use — breaks accessibility

Media Queries#

Mobile-First Breakpoints#

/* Base styles — mobile (no media query needed) */
.container {
 padding: 1rem;
}

/* Tablet and up */
@media (min-width: 768px) {
 .container {
 padding: 2rem;
 max-width: 720px;
 margin: 0 auto;
 }
}

/* Desktop and up */
@media (min-width: 1024px) {
 .container {
 max-width: 960px;
 }
}

/* Large desktop */
@media (min-width: 1280px) {
 .container {
 max-width: 1200px;
 }
}

Common Breakpoints#

Name	Width	Targets
`sm`	640px	Large phones (landscape)
`md`	768px	Tablets
`lg`	1024px	Small laptops
`xl`	1280px	Desktops
`2xl`	1536px	Large screens

Important: These are guidelines, not rules. Set breakpoints where your design breaks, not at device widths.

Routing

Mon, 01 Jan 0001 00:00:00 +0000

What is Routing?#

Routing is the process of selecting a path for network traffic across one or more networks. Routers examine the destination IP address of each packet and decide where to forward it next.

flowchart LR
 A["Source<br/>192.168.1.10"] --> R1["Router 1"]
 R1 --> R2["Router 2"]
 R2 --> R3["Router 3"]
 R3 --> B["Destination<br/>203.0.113.50"]

Each router makes an independent forwarding decision based on its routing table — it doesn’t know the full path, only the next hop.

Stacks & Queues

Mon, 01 Jan 0001 00:00:00 +0000

Stacks & Queues#

Stacks (LIFO) and queues (FIFO) appear in problems involving ordering, nesting, and level-by-level processing. The monotonic stack pattern alone covers a large class of interview questions.

Pattern Summary#

Pattern	Time	Space	When to Use
Monotonic Stack	O(n)	O(n)	Next greater/smaller element, histogram
Valid Parentheses	O(n)	O(n)	Matching brackets, nesting validation
Stack as History	O(n)	O(n)	Undo operations, path simplification
Queue with Max/Min	O(n)	O(n)	Sliding window max/min
BFS with Queue	O(V+E)	O(V)	Shortest path, level-order traversal
Deque Patterns	O(n)	O(k)	Sliding window, double-ended access

Monotonic Stack#

A stack that maintains elements in sorted order (increasing or decreasing). As you push elements, you pop anything that violates the monotonic property.

State

Mon, 01 Jan 0001 00:00:00 +0000

What is State?#

State is Terraform’s memory. It’s a JSON file that maps your configuration to real-world infrastructure. Without state, Terraform wouldn’t know which cloud resources it created, what their current attributes are, or what needs to change.

flowchart TD
 Config["Configuration (.tf files)<br>What you WANT"] 
 State["State File<br>What Terraform KNOWS"]
 Reality["Real Infrastructure<br>What ACTUALLY EXISTS"]
 
 Config --> |"terraform plan"| Compare["Compare all three"]
 State --> Compare
 Reality --> Compare
 Compare --> Plan["Execution Plan<br>(create, update, destroy)"]

Why State Exists#

Terraform needs state because:

Storage

Mon, 01 Jan 0001 00:00:00 +0000

Storage Types#

AWS offers storage at different layers — block, file, and object — each optimized for different access patterns:

flowchart TD
 Storage["AWS Storage"]
 Storage --> Block["Block Storage<br>(EBS, Instance Store)"]
 Storage --> Object["Object Storage<br>(S3, S3 Glacier)"]
 Storage --> File["File Storage<br>(EFS, FSx)"]
 
 Block --> |"Attached to EC2"| Disk["Like a hard drive"]
 Object --> |"HTTP API"| Bucket["Like a file server with infinite capacity"]
 File --> |"NFS/SMB"| Share["Like a network share"]

S3 (Simple Storage Service)#

Object storage with virtually unlimited capacity. The most-used AWS service.

Storage

Mon, 01 Jan 0001 00:00:00 +0000

Storage#

Managing persistent and ephemeral data in Kubernetes — from temporary scratch space to durable database volumes, plus configuration and secrets injection.

Volume Types Overview#

Volume Type	Persistence	Use Case
`emptyDir`	Pod lifetime only	Scratch space, inter-container sharing
`hostPath`	Node lifetime	Node-level logs, dev/testing only
`configMap`	Cluster lifetime	Configuration files mounted as volumes
`secret`	Cluster lifetime	Sensitive data mounted as files
`persistentVolumeClaim`	Independent of pod	Databases, stateful workloads
`nfs`	External NFS server	Shared read-write across pods
`csi`	Depends on driver	Any storage backend via CSI standard

Volumes (Ephemeral)#

emptyDir#

Created when a pod is assigned to a node. Deleted when the pod is removed.

Structs & Enums

Mon, 01 Jan 0001 00:00:00 +0000

Structs & Enums#

Structs group related data. Enums represent values that can be one of several variants. Together with pattern matching, they form Rust’s primary data modelling toolkit — replacing classes, inheritance, and null in other languages.

Structs#

struct User {
 id: u64,
 name: String,
 email: String,
 active: bool,
}

let user = User {
 id: 1,
 name: String::from("Alice"),
 email: String::from("alice@example.com"),
 active: true,
};

println!("{}", user.name);

Field Init Shorthand#

fn new_user(name: String, email: String) -> User {
 User {
 id: generate_id(),
 name, // shorthand: field name matches variable name
 email, // same
 active: true,
 }
}

Struct Update Syntax#

let user2 = User {
 email: String::from("bob@example.com"),
 ..user // take remaining fields from user (MOVES String fields)
};

Tuple Structs#

struct Color(u8, u8, u8);
struct Point(f64, f64);

let red = Color(255, 0, 0);
let origin = Point(0.0, 0.0);
println!("r={}", red.0);

Unit Structs#

struct AlwaysEqual; // no fields, used as markers or trait implementations

Methods#

Defined in impl blocks:

Structs & Unions

Mon, 01 Jan 0001 00:00:00 +0000

Structs & Unions#

Structs group related data into a single type. Unions overlay multiple fields in the same memory. Together with enums and bitfields, they let you model any data layout — from database records to hardware registers.

Structs#

struct Point {
 double x;
 double y;
};

struct Point origin = {0.0, 0.0}; // positional init
struct Point p = {.x = 3.0, .y = 4.0}; // designated init (C99)

printf("(%f, %f)\n", p.x, p.y);

// Pointer access
struct Point *pp = &p;
printf("(%f, %f)\n", pp->x, pp->y); // -> dereferences + accesses

Typedef#

typedef struct {
 int id;
 char name[64];
 double salary;
} Employee;

Employee e = {1, "Alice", 95000.0};

Structs with Pointers#

typedef struct Node {
 int value;
 struct Node *next; // self-referential — must use full struct name
} Node;

Node a = {1, NULL};
Node b = {2, &a};
// b -> a -> NULL

Memory Layout and Alignment#

The compiler inserts padding to align struct fields to their natural boundaries:

Structs and Classes

Mon, 01 Jan 0001 00:00:00 +0000

Structs and Classes#

Swift provides two mechanisms for defining custom types: structs (value types) and classes (reference types). Understanding their differences is fundamental to writing correct Swift code.

Value Types vs Reference Types#

graph TD
 subgraph "Value Type (Struct)"
 A[var a = Point&lpar;x: 1, y: 2&rpar;] --> B[Memory: x=1, y=2]
 C["var b = a (copy)"] --> D[Memory: x=1, y=2]
 E["b.x = 5"] --> F[a unchanged: x=1, y=2]
 end

 subgraph "Reference Type (Class)"
 G[var c = Person&lpar;name: &quot;Alice&quot;&rpar;] --> H[Heap: name=&quot;Alice&quot;]
 I["var d = c (shared reference)"] --> H
 J["d.name = &quot;Bob&quot;"] --> K["c.name is also &quot;Bob&quot;"]
 end

Characteristic	Struct (Value Type)	Class (Reference Type)
Storage	Stack (usually)	Heap
Assignment	Copies the value	Copies the reference
Identity	No identity — equality by value	Has identity (`===` operator)
Mutability	`mutating` keyword required	Mutable through any reference
Inheritance	❌	✅
Deinitializers	❌	✅ (`deinit`)
Reference counting	❌	✅ (ARC)

When to Use Which#

Use structs (the default choice) when:

Time Management

Mon, 01 Jan 0001 00:00:00 +0000

Time Management#

Engineers face a unique time management challenge: their highest-value work requires deep focus, but their environment is full of interruptions — Slack messages, code reviews, meetings, incidents, and “quick questions.” Managing time is not about doing more — it’s about protecting the space for work that matters.

Prioritisation Frameworks#

The Eisenhower Matrix#

Categorise tasks by urgency and importance to decide what to do, schedule, delegate, or drop.

quadrantChart
 title Eisenhower Matrix
 x-axis "Not Urgent" --> "Urgent"
 y-axis "Not Important" --> "Important"
 quadrant-1 "DO: Schedule focused time"
 quadrant-2 "DO FIRST: Handle now"
 quadrant-3 "ELIMINATE: Stop doing"
 quadrant-4 "DELEGATE: Hand off"

Quadrant	Examples in engineering	Action
Important + Urgent	Production incident, blocking bug, security vulnerability	Do it now
Important + Not Urgent	Architecture improvements, documentation, learning, testing	Schedule protected time — this is where growth happens
Not Important + Urgent	Most Slack pings, routine emails, someone else’s meeting	Delegate or batch-process
Not Important + Not Urgent	Bike-shedding, unnecessary refactors, rabbit holes	Eliminate

The trap: spending all time in Quadrant 1 (urgent) while Quadrant 2 (important but not urgent) atrophies. Invest in Q2 to reduce future Q1 emergencies.

Training and Fine-Tuning Language Models

Mon, 01 Jan 0001 00:00:00 +0000

This guide covers the full lifecycle of making a language model useful — from pre-training on raw text to aligning it with human preferences. You’ll understand when to fine-tune, when to prompt, and when to use RAG instead.

Prerequisites#

Transformer architecture (attention, decoder-only models)
Neural network training (loss functions, optimizers, backpropagation)
Basic understanding of what language models do (next-token prediction)

The Three Stages of LLM Development#

Stage 1: Pre-training → "Learn language and world knowledge"
Stage 2: Supervised Fine-tuning → "Learn to follow instructions"
Stage 3: Alignment (RLHF/DPO) → "Learn human preferences"

graph LR
 PT[Pre-training<br/>Trillions of tokens<br/>Next token prediction] --> SFT[Supervised Fine-tuning<br/>Thousands of examples<br/>Instruction-response pairs]
 SFT --> ALIGN[Alignment<br/>RLHF or DPO<br/>Human preferences]
 ALIGN --> DEPLOY[Deployed Model<br/>Helpful, harmless<br/>honest]

Stage 1: Pre-training#

Pre-training creates the foundation model — a model that understands language, facts, reasoning patterns, and code, but doesn’t yet know how to be a helpful assistant.

Virtual Memory

Mon, 01 Jan 0001 00:00:00 +0000

Virtual Memory#

Virtual memory extends the basic paging model into a complete system for managing address spaces larger than physical RAM. It encompasses the page table structures, swap management, memory overcommit policies, and hardware-aware optimizations like NUMA and huge pages.

Virtual vs Physical Addresses#

Aspect	Virtual Address	Physical Address
Generated by	CPU / program	MMU (after translation)
Space size	Full architecture width (e.g., 48-bit on x86-64)	Limited by installed RAM
Per-process	Yes — each process has its own	Shared — single physical space
Contiguous?	Appears contiguous to process	Scattered frames
Bounds	0 to 2^48 - 1 (256 TB on x86-64)	0 to installed_ram - 1

Process A sees: Physical RAM: Process B sees:
┌──────────┐ ┌──────────┐ ┌──────────┐
│ 0x0000 │────────┐ │ Frame 0 │ ┌─────│ 0x0000 │
│ 0x1000 │──┐ └───▶│ Frame 1 │ │ │ 0x1000 │
│ 0x2000 │ │ │ Frame 2 │◀────┘ │ 0x2000 │
└──────────┘ └─────────▶│ Frame 3 │ └──────────┘
 │ Frame 4 │◀──── Process B page 1
 └──────────┘

Page Table Structures#

A flat page table for a 48-bit address space with 4KB pages would require 2^36 entries × 8 bytes = 512 GB. Clearly impractical. Real systems use hierarchical or inverted page tables.

Web Application Security

Mon, 01 Jan 0001 00:00:00 +0000

Why This Matters#

Web applications are the most common attack surface. They’re internet-facing, complex, and handle sensitive data. Most breaches involve exploiting web application vulnerabilities.

OWASP Top 10 (2021)#

The Open Web Application Security Project’s definitive list of critical web risks:

#	Category	Root Cause
A01	Broken Access Control	Missing or bypassed authorization checks
A02	Cryptographic Failures	Weak crypto, plaintext data, missing encryption
A03	Injection	Untrusted data interpreted as code/commands
A04	Insecure Design	Flawed architecture, missing threat model
A05	Security Misconfiguration	Default configs, unnecessary features enabled
A06	Vulnerable Components	Outdated libraries with known CVEs
A07	Identification & Auth Failures	Weak passwords, broken session management
A08	Software & Data Integrity Failures	Unsigned updates, insecure deserialization
A09	Logging & Monitoring Failures	Attacks go undetected
A10	Server-Side Request Forgery	Server makes requests to unintended targets

Injection Attacks#

SQL Injection#

The server treats user input as SQL code:

Write-Ahead Logging

Mon, 01 Jan 0001 00:00:00 +0000

The Durability Problem#

A database crash can happen at any moment — power failure, kernel panic, hardware fault. If modified pages exist only in the buffer pool (memory), they are lost. Writing every modified page to disk immediately would be safe but catastrophically slow (random I/O for every change).

Write-Ahead Logging (WAL) solves this: before modifying any data page, write a log record describing the change to a sequential log file. If the system crashes, replay the log to reconstruct any modifications that were not yet flushed to the data files.

Algorithms

Mon, 01 Jan 0001 00:00:00 +0000

An algorithm is a finite, well-defined sequence of steps that solves a problem. This section covers the essential algorithms every programmer should know — sorting, searching, and graph traversal — with implementations and analysis.

Sorting Algorithms#

Why Sorting Matters#

Sorting is a prerequisite for many other algorithms (binary search, merge operations, duplicate detection). Understanding sorting algorithms teaches fundamental algorithmic techniques: divide-and-conquer, incremental construction, and heap properties.

Bubble Sort — The Teaching Algorithm#

Repeatedly swap adjacent elements if they’re in the wrong order:

Apache Spark

Mon, 01 Jan 0001 00:00:00 +0000

What Is Apache Spark?#

Apache Spark is a distributed computing engine for large-scale data processing. It runs batch and streaming workloads across clusters of machines, processing data in memory for speed. Spark is the de facto standard for big data processing.

Spark Architecture#

┌─────────────┐
│ Driver │ ← Your application code
│ (SparkContext) │
└──────┬──────┘
 │
 ┌────┴────┐
 │ Cluster │ ← YARN, Kubernetes, Standalone, Mesos
 │ Manager │
 └────┬────┘
 │
┌──────┴──────┬──────────────┐
│ Executor 1 │ Executor 2 │ Executor N │
│ (Tasks) │ (Tasks) │ (Tasks) │
│ [Cache] │ [Cache] │ [Cache] │
└─────────────┴──────────────┘

Driver — the main process that creates the SparkSession, defines transformations, and coordinates execution. Executors — worker processes that execute tasks and cache data in memory. Cluster Manager — allocates resources (YARN, Kubernetes, or Spark Standalone).

Authorisation

Mon, 01 Jan 0001 00:00:00 +0000

Authentication vs Authorisation#

Authentication proves who you are. Authorisation determines what you can do. A system can have perfect authentication and still be completely broken if authorisation is missing or bypassable.

Broken access control was the #1 risk in the OWASP Top 10 (2021), overtaking injection for the first time.

Access Control Models#

Role-Based Access Control (RBAC)#

Permissions are assigned to roles, and users are assigned to roles. Simple, widely understood, and sufficient for most applications.

Buffer Pool Management

Mon, 01 Jan 0001 00:00:00 +0000

The Memory–Disk Gap#

Disk I/O (even on SSDs) is orders of magnitude slower than memory access. The buffer pool bridges this gap by caching frequently accessed disk pages in memory. A well-tuned buffer pool means most reads never touch disk.

Access latency comparison:
 L1 cache: ~1 ns
 DRAM: ~100 ns
 SSD random: ~50,000 ns (500× slower than RAM)
 HDD random: ~10,000,000 ns (100,000× slower than RAM)

The buffer pool is the single most important performance structure in a database — it determines the cache hit ratio, which directly controls query latency.

Configuration & Scaling

Mon, 01 Jan 0001 00:00:00 +0000

Configuration & Scaling#

Resource management, autoscaling strategies, and fine-grained pod scheduling — controlling where pods run and how much of the cluster they consume.

Resource Requests and Limits#

Every container should declare how much CPU and memory it needs (requests) and the maximum it may consume (limits).

spec:
 containers:
 - name: api
 image: myapp:1.0
 resources:
 requests:
 cpu: "250m" # 0.25 CPU cores (guaranteed minimum)
 memory: "256Mi" # 256 MiB (guaranteed minimum)
 limits:
 cpu: "1000m" # 1 CPU core (throttled beyond this)
 memory: "512Mi" # 512 MiB (OOM-killed beyond this)

Concept	CPU Behavior	Memory Behavior
Request	Guaranteed allocation; used for scheduling	Guaranteed allocation; used for scheduling
Limit	Throttled (not killed) when exceeded	OOM-killed when exceeded
No request set	Best-effort scheduling	Best-effort scheduling
No limit set	Can consume all available CPU	Can consume all available memory

QoS Classes#

Kubernetes assigns a Quality of Service class based on resource declarations:

Coroutines and Concurrency

Mon, 01 Jan 0001 00:00:00 +0000

Coroutines and Concurrency#

Kotlin coroutines provide a structured, lightweight model for asynchronous and concurrent programming. Unlike threads, coroutines are cheap to create (thousands can run simultaneously), and structured concurrency ensures that work is properly scoped and cancelled.

Suspend Functions#

A suspend function can be paused and resumed without blocking a thread. Suspension is the fundamental building block of coroutines:

suspend fun fetchUser(id: Long): User {
 // This call suspends the coroutine, freeing the thread
 val response = httpClient.get("https://api.example.com/users/$id")
 return response.body()
}

suspend fun fetchUserWithPosts(id: Long): UserWithPosts {
 val user = fetchUser(id) // Sequential
 val posts = fetchPosts(user.id) // Sequential — waits for user first
 return UserWithPosts(user, posts)
}

Suspend functions can only be called from other suspend functions or from a coroutine builder.

CSS Architecture

Mon, 01 Jan 0001 00:00:00 +0000

The Scaling Problem#

CSS is global by default. Every selector can affect any element on the page. As projects grow, this leads to specificity wars, unintended side effects, and fear of changing styles. CSS architecture methodologies solve this by providing naming conventions and organizational patterns.

BEM (Block Element Modifier)#

The most widely adopted naming convention. It makes relationships between components explicit in class names.

Naming Pattern#

.block {}
.block__element {}
.block--modifier {}

Part	Meaning	Example
Block	Standalone component	`.card`, `.nav`, `.form`
Element	Part of a block (can’t exist alone)	`.card__title`, `.card__image`
Modifier	Variation of block or element	`.card--featured`, `.card__title--large`

Example#

<article class="card card--featured">
 <img class="card__image" src="..." alt="...">
 <div class="card__body">
 <h2 class="card__title">Title</h2>
 <p class="card__description">Description text</p>
 <a class="card__link card__link--primary" href="#">Read more</a>
 </div>
</article>

/* Block */
.card {
 border-radius: 8px;
 overflow: hidden;
 box-shadow: 0 2px 4px rgba(0, 0, 0, 0.1);
}

/* Modifier on block */
.card--featured {
 border: 2px solid var(--color-primary);
}

/* Elements */
.card__image {
 width: 100%;
 aspect-ratio: 16 / 9;
 object-fit: cover;
}

.card__title {
 font-size: 1.25rem;
 margin-bottom: 0.5rem;
}

.card__link {
 text-decoration: none;
 color: var(--color-primary);
}

/* Modifier on element */
.card__link--primary {
 font-weight: bold;
}

BEM Rules#

Never nest selectors — .card .card__title is wrong; just .card__title
No element of element — .card__body__title is wrong; flatten to .card__title
Modifiers don’t exist alone — always pair: class="card card--featured"
Blocks are independent — a block should work anywhere on the page

ITCSS (Inverted Triangle CSS)#

ITCSS organizes CSS by specificity, from generic to specific:

Data Architecture Patterns

Mon, 01 Jan 0001 00:00:00 +0000

Data Architecture Patterns#

Data outlives applications. The services you build today will be rewritten, but the data they produce will still matter in ten years. Cloud data architecture is about choosing the right storage for each workload, placing data where it needs to be, and ensuring it’s protected, governed, and accessible.

Polyglot Persistence#

Using a single database for everything is an anti-pattern. Different data access patterns require different storage engines.

Data Definition and Constraints

Mon, 01 Jan 0001 00:00:00 +0000

Creating Tables#

CREATE TABLE employees (
 id SERIAL PRIMARY KEY, -- Auto-increment PK (PostgreSQL)
 name VARCHAR(100) NOT NULL,
 email VARCHAR(255) UNIQUE NOT NULL,
 department_id INTEGER REFERENCES departments(id),
 salary NUMERIC(10, 2) DEFAULT 0.00,
 level VARCHAR(20) CHECK (level IN ('junior', 'mid', 'senior', 'lead')),
 is_active BOOLEAN DEFAULT TRUE,
 hire_date DATE NOT NULL DEFAULT CURRENT_DATE,
 created_at TIMESTAMP DEFAULT NOW()
);

-- MySQL equivalent
CREATE TABLE employees (
 id INT AUTO_INCREMENT PRIMARY KEY,
 name VARCHAR(100) NOT NULL,
 email VARCHAR(255) UNIQUE NOT NULL,
 department_id INT,
 salary DECIMAL(10, 2) DEFAULT 0.00,
 level ENUM('junior', 'mid', 'senior', 'lead'),
 is_active BOOLEAN DEFAULT TRUE,
 hire_date DATE NOT NULL DEFAULT (CURRENT_DATE),
 created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
 FOREIGN KEY (department_id) REFERENCES departments(id)
);

Constraints#

Constraints enforce data integrity rules at the database level — data that violates them is rejected.

Databases

Mon, 01 Jan 0001 00:00:00 +0000

Database Options#

AWS offers purpose-built databases for different data models and access patterns:

flowchart TD
 DB["AWS Databases"]
 DB --> Relational["Relational (SQL)"]
 DB --> KV["Key-Value"]
 DB --> Document["Document"]
 DB --> Cache["In-Memory Cache"]
 DB --> Graph["Graph"]
 DB --> TimeSeries["Time Series"]
 
 Relational --> RDS["RDS / Aurora"]
 KV --> DDB["DynamoDB"]
 Document --> DocDB["DocumentDB"]
 Cache --> EC["ElastiCache"]
 Graph --> Neptune["Neptune"]
 TimeSeries --> TS["Timestream"]

RDS (Relational Database Service)#

Managed relational databases — AWS handles patching, backups, failover, and replication.

Databases at Scale

Mon, 01 Jan 0001 00:00:00 +0000

Databases at Scale#

A single database server eventually becomes a bottleneck — whether for read throughput, write throughput, storage capacity, or availability. Scaling databases requires understanding replication, sharding, and the trade-offs each introduces.

Replication#

Replication maintains copies of data across multiple database nodes for availability, durability, and read scaling.

Synchronous vs Asynchronous Replication#

sequenceDiagram
 participant Client
 participant Primary
 participant Replica1
 participant Replica2

 Note over Client,Replica2: Synchronous Replication
 Client->>Primary: INSERT INTO orders...
 Primary->>Replica1: Replicate (wait for ACK)
 Replica1-->>Primary: ACK
 Primary-->>Client: Commit confirmed

 Note over Client,Replica2: Asynchronous Replication
 Client->>Primary: INSERT INTO orders...
 Primary-->>Client: Commit confirmed (immediate)
 Primary->>Replica2: Replicate (eventual)

Aspect	Synchronous	Asynchronous
Write latency	Higher (wait for replica ACK)	Lower (immediate commit)
Data safety	Zero data loss on primary failure	Potential data loss (replication lag)
Availability	Slower (blocked if replica is down)	Higher (primary independent)
Consistency	Strong (replicas always current)	Eventual (replicas may lag)
Throughput	Lower (bounded by slowest replica)	Higher (primary writes freely)
Use case	Financial transactions, inventory	Analytics, read-heavy workloads

Semi-Synchronous#

A hybrid: the primary waits for at least one replica to acknowledge before confirming to the client. Other replicas receive updates asynchronously.

End-to-End Testing

Mon, 01 Jan 0001 00:00:00 +0000

End-to-End Testing#

End-to-end (E2E) tests verify complete user workflows through the full application stack — from the browser or API client, through your backend, to the database and back. They provide the highest confidence but are the slowest and most maintenance-intensive.

What E2E Tests Cover#

graph LR
 subgraph "E2E Test Scope"
 USER[User/Client] --> UI[Browser/API]
 UI --> SERVER[Backend Server]
 SERVER --> DB[(Database)]
 SERVER --> EXT[External Services]
 end

Characteristic	E2E Tests
Scope	Full system, multiple components
Speed	Seconds to minutes per test
Confidence	Highest — tests real user experience
Maintenance	High — brittle to UI/flow changes
Debugging	Hard — failures could be anywhere in the stack
Count	Few — cover critical paths only

What to E2E Test#

Critical user journeys — signup, login, checkout, payment
Cross-system flows — data that passes through multiple services
Smoke tests — “does the app start and load?” after deployment
Regression tests — for bugs that slipped through other test layers

What NOT to E2E Test#

Edge cases (use unit tests)
Every possible form validation (use unit/integration tests)
Internal API responses (use API-level integration tests)
Visual pixel-perfection (use visual regression tests separately)

Browser Testing#

Playwright (Recommended)#

// tests/e2e/checkout.spec.js
import { test, expect } from '@playwright/test';

test('user can complete checkout flow', async ({ page }) => {
 // Navigate and add item to cart
 await page.goto('/products');
 await page.click('[data-testid="product-widget"]');
 await page.click('button:text("Add to Cart")');

 // Go to cart and proceed to checkout
 await page.click('[data-testid="cart-icon"]');
 await expect(page.locator('.cart-item')).toHaveCount(1);
 await page.click('button:text("Checkout")');

 // Fill shipping information
 await page.fill('[name="address"]', '123 Test Street');
 await page.fill('[name="city"]', 'Barcelona');
 await page.fill('[name="zip"]', '08001');

 // Submit order
 await page.click('button:text("Place Order")');

 // Verify confirmation
 await expect(page.locator('.order-confirmation')).toBeVisible();
 await expect(page.locator('.order-id')).toContainText(/ORD-/);
});

Cypress#

// cypress/e2e/login.cy.js
describe('Login Flow', () => {
 beforeEach(() => {
 cy.visit('/login');
 });

 it('logs in with valid credentials', () => {
 cy.get('[data-testid="email-input"]').type('alice@example.com');
 cy.get('[data-testid="password-input"]').type('secure123');
 cy.get('[data-testid="login-button"]').click();

 cy.url().should('include', '/dashboard');
 cy.get('[data-testid="welcome-message"]')
 .should('contain', 'Welcome, Alice');
 });

 it('shows error for invalid credentials', () => {
 cy.get('[data-testid="email-input"]').type('alice@example.com');
 cy.get('[data-testid="password-input"]').type('wrong');
 cy.get('[data-testid="login-button"]').click();

 cy.get('[data-testid="error-message"]')
 .should('be.visible')
 .and('contain', 'Invalid credentials');
 cy.url().should('include', '/login');
 });
});

Playwright vs Cypress#

Feature	Playwright	Cypress
Languages	JS, Python, Java, .NET	JavaScript only
Browsers	Chromium, Firefox, WebKit	Chromium, Firefox, WebKit
Parallelism	Built-in	Requires paid Cloud
Multi-tab/window	Supported	Not supported
Network interception	Full control	Good (cy.intercept)
Auto-waiting	Built-in	Built-in
iframes	Full support	Limited
Best for	Cross-browser, multi-page apps	Single-page apps, rapid prototyping

API End-to-End Testing#

Test complete API workflows without a browser:

Error Handling

Mon, 01 Jan 0001 00:00:00 +0000

Error Handling#

Rust uses Result<T, E> for recoverable errors and panic! for unrecoverable ones. There are no exceptions, no try/catch — errors are values in the type system, and the compiler forces you to handle them.

Result<T, E>#

enum Result<T, E> {
 Ok(T), // success with value
 Err(E), // failure with error
}

use std::fs;

fn read_config(path: &str) -> Result<String, std::io::Error> {
 fs::read_to_string(path)
}

match read_config("config.toml") {
 Ok(contents) => println!("{contents}"),
 Err(e) => eprintln!("Failed to read config: {e}"),
}

The `?` Operator#

? propagates errors — if the Result is Err, return it immediately; if Ok, unwrap the value:

Error Handling and Exceptions

Mon, 01 Jan 0001 00:00:00 +0000

PHP’s Error Model#

PHP has two parallel error systems — legacy errors and modern exceptions. Understanding both is essential.

The Error Hierarchy (PHP 7+)#

graph TD
 T["Throwable (interface)"] --> Error["Error"]
 T --> Exception["Exception"]
 Error --> TypeError
 Error --> ValueError
 Error --> ArithmeticError
 Error --> DivisionByZeroError
 Error --> ParseError
 Exception --> RuntimeException
 Exception --> LogicException
 Exception --> InvalidArgumentException
 RuntimeException --> OverflowException
 RuntimeException --> UnexpectedValueException
 LogicException --> DomainException
 LogicException --> LengthException

Key distinction:

Error — programming mistakes that should not be caught in normal flow (type errors, parse errors)
Exception — recoverable conditions that business logic should handle

Try / Catch / Finally#

<?php
try {
 $result = riskyOperation();
} catch (SpecificException $e) {
 // Handle specific exception type
 logger()->error('Operation failed', ['error' => $e->getMessage()]);
 $result = fallbackValue();
} catch (AnotherException | YetAnotherException $e) {
 // Catch multiple types (PHP 7.1+)
 handleMultiple($e);
} catch (\Throwable $e) {
 // Last resort — catches both Error and Exception
 logger()->critical('Unexpected error', ['exception' => $e]);
 throw $e; // Re-throw if you can't handle it
} finally {
 // Always runs — cleanup, close resources
 cleanup();
}

Non-Capturing Catches (PHP 8.0+)#

When you don’t need the exception object:

Event-Driven Architecture

Mon, 01 Jan 0001 00:00:00 +0000

Event-Driven Architecture#

Event-Driven Architecture (EDA) is a paradigm where the production, detection, consumption, and reaction to events drives the system’s behavior. Instead of direct calls between services, components communicate through events — facts about things that happened.

Event Types#

Not all events are equal. Understanding the type helps you choose the right infrastructure and guarantees.

Type	Purpose	Content	Example
Domain Event	Records what happened in the domain	Full state at time of event	`BookingConfirmed { booking_id, guest, dates, total }`
Integration Event	Communicates across bounded contexts	Minimal data + references	`BookingConfirmed { booking_id, guest_id }`
Notification Event	Signals something happened (no data)	Event type + aggregate ID only	`BookingConfirmed { booking_id }`

Domain Events#

Rich events used within a bounded context. They contain enough information for any handler within the same context to act.

File Systems

Mon, 01 Jan 0001 00:00:00 +0000

File Systems#

A file system organizes persistent data on storage devices into a logical structure of files and directories. It translates high-level operations (open, read, write) into low-level block I/O while maintaining data integrity, supporting concurrent access, and managing metadata.

File System Concepts#

Concept	Description
File	Named collection of related data stored on secondary storage
Directory	Container that maps file names to file metadata
Path	Hierarchical location identifier (`/home/user/docs/file.txt`)
Mount point	Location where a file system attaches to the directory tree
Superblock	File system metadata (size, block count, free space)
Block	Minimum allocation unit (typically 4 KB)

File Metadata (stored separately from data)#

┌─────────────────────────────────────────┐
│ File type │ Regular, directory, symlink, device, pipe │
│ Permissions │ rwxr-xr-x (owner/group/other) │
│ Owner/Group │ UID, GID │
│ Size │ Bytes of data │
│ Timestamps │ atime, mtime, ctime │
│ Link count │ Number of hard links │
│ Block pointers │ Locations of data blocks on disk │
└─────────────────────────────────────────┘

Inodes#

An inode (index node) stores all metadata about a file except its name:

GitHub Actions

Mon, 01 Jan 0001 00:00:00 +0000

GitHub Actions#

GitHub Actions is GitHub’s native CI/CD platform. Workflows are defined in YAML files under .github/workflows/ and triggered by repository events (push, pull request, schedule, and more). Its marketplace of reusable actions and tight GitHub integration make it one of the most popular CI/CD systems.

Workflow Files#

Every workflow lives in .github/workflows/:

# .github/workflows/ci.yml
name: CI Pipeline

on:
 push:
 branches: [main]
 pull_request:
 branches: [main]

permissions:
 contents: read
 checks: write

jobs:
 lint:
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4
 - uses: actions/setup-node@v4
 with:
 node-version: 20
 cache: 'npm'
 - run: npm ci
 - run: npm run lint

 test:
 runs-on: ubuntu-latest
 needs: lint
 steps:
 - uses: actions/checkout@v4
 - uses: actions/setup-node@v4
 with:
 node-version: 20
 cache: 'npm'
 - run: npm ci
 - run: npm test -- --coverage
 - uses: actions/upload-artifact@v4
 with:
 name: coverage
 path: coverage/

Workflow Structure#

graph TD
 A[Workflow File] --> B[Events/Triggers]
 A --> C[Permissions]
 A --> D[Environment Variables]
 A --> E[Jobs]
 E --> F[Job 1]
 E --> G[Job 2]
 F --> H[Steps]
 H --> I[Action Step]
 H --> J[Run Step]

Events and Triggers#

Event	Triggers On	Common Use
`push`	Commit pushed to branch/tag	CI on every commit
`pull_request`	PR opened, updated, reopened	Gate before merge
`schedule`	Cron expression	Nightly builds, dependency checks
`workflow_dispatch`	Manual trigger (UI or API)	Ad-hoc runs, prod deploys
`release`	Release published	Build release artifacts
`repository_dispatch`	External webhook	Cross-repo triggers
`workflow_call`	Called by another workflow	Reusable workflows
`merge_group`	Merge queue	Validate before auto-merge

Event Filters#

on:
 push:
 branches:
 - main
 - 'release/**'
 tags:
 - 'v*'
 paths:
 - 'src/**'
 - 'package.json'
 paths-ignore:
 - 'docs/**'
 - '**.md'

 pull_request:
 types: [opened, synchronize, reopened]
 branches: [main]

 schedule:
 - cron: '0 2 * * 1-5' # Weekdays at 2 AM UTC

 workflow_dispatch:
 inputs:
 environment:
 description: 'Deploy target'
 required: true
 type: choice
 options: [staging, production]
 dry_run:
 description: 'Dry run only'
 type: boolean
 default: true

Jobs and Steps#

Job Configuration#

jobs:
 build:
 name: "Build Application"
 runs-on: ubuntu-latest  # Runner environment
 timeout-minutes: 15 # Maximum duration
 continue-on-error: false # Fail workflow on job failure
 concurrency: # Prevent parallel runs
 group: build-${{ github.ref }}
 cancel-in-progress: true
 
 env: # Job-level env vars
 NODE_ENV: production
 
 services: # Sidecar containers
 postgres:
 image: postgres:16
 env:
 POSTGRES_PASSWORD: test
 ports:
 - 5432:5432
 options: >-
 --health-cmd pg_isready
 --health-interval 10s
 --health-timeout 5s
 --health-retries 5
 
 steps:
 - uses: actions/checkout@v4
 - run: echo "Building..."

Step Types#

Type	Syntax	Purpose
Action	`uses: owner/repo@ref`	Run a reusable action
Run	`run: command`	Execute shell commands
Composite	Action with multiple steps	Bundle common step patterns

Step Features#

steps:
 - name: "Conditional step"
 if: github.event_name == 'push'
 run: echo "This is a push event"

 - name: "Step with outputs"
 id: version
 run: echo "tag=v1.2.3" >> $GITHUB_OUTPUT

 - name: "Use output from previous step"
 run: echo "Version is ${{ steps.version.outputs.tag }}"

 - name: "Continue on failure"
 continue-on-error: true
 run: npm run optional-check

 - name: "Set environment variable"
 run: echo "BUILD_DATE=$(date -u +%Y%m%d)" >> $GITHUB_ENV

 - name: "Multi-line script"
 run: |
 echo "Line 1"
 echo "Line 2"
 if [ "${{ github.ref }}" = "refs/heads/main" ]; then
 echo "On main branch"
 fi
 shell: bash

Actions Marketplace#

The GitHub Marketplace provides thousands of reusable actions:

Identity & Access Management

Mon, 01 Jan 0001 00:00:00 +0000

Why This Matters#

Identity is the new perimeter. In cloud and remote-work environments, controlling WHO can access WHAT is the primary security control. IAM failures are behind most breaches — stolen credentials, excessive permissions, and broken authentication.

Core Concepts#

flowchart LR
 Subject["Subject<br/>(user, service, device)"]
 Subject -->|"AuthN: Who are you?"| Identity["Identity<br/>Verification"]
 Identity -->|"AuthZ: What can you do?"| Access["Access<br/>Decision"]
 Access -->|"Allow/Deny"| Resource["Protected<br/>Resource"]

Concept	Question	Mechanism
Identification	Who claims to be?	Username, email, certificate CN
Authentication (AuthN)	Prove it	Password, MFA, certificate, biometric
Authorization (AuthZ)	What’s allowed?	Roles, policies, attributes
Accounting/Auditing	What did they do?	Logs, audit trail

Authentication Protocols#

SAML 2.0 (Security Assertion Markup Language)#

XML-based SSO protocol, common in enterprise:

Instrumentation

Mon, 01 Jan 0001 00:00:00 +0000

Instrumentation#

Instrumentation is the process of adding telemetry code to your applications so they emit metrics, logs, and traces. OpenTelemetry provides a unified API and SDK for all three signals. This section covers auto vs manual instrumentation, language-specific setup, and best practices.

Auto vs Manual Instrumentation#

Auto-Instrumentation#

Auto-instrumentation injects telemetry into your application without code changes. It hooks into frameworks and libraries (HTTP clients, database drivers, messaging systems) to automatically create spans, record metrics, and correlate logs.

Java: Collections Framework

Mon, 01 Jan 0001 00:00:00 +0000

The Collections Hierarchy#

graph TD
 I["Iterable"] --> C["Collection"]
 C --> L["List"]
 C --> S["Set"]
 C --> Q["Queue"]
 
 L --> AL["ArrayList"]
 L --> LL["LinkedList"]
 L --> CL["CopyOnWriteArrayList"]
 
 S --> HS["HashSet"]
 S --> TS["TreeSet"]
 S --> LHS["LinkedHashSet"]
 
 Q --> PQ["PriorityQueue"]
 Q --> D["Deque"]
 D --> AD["ArrayDeque"]
 
 M["Map"] --> HM["HashMap"]
 M --> TM["TreeMap"]
 M --> LHM["LinkedHashMap"]
 M --> CHM["ConcurrentHashMap"]

List Implementations#

ArrayList — Dynamic Array#

// O(1) random access, O(1) amortized append, O(n) insert/remove at index
List<String> names = new ArrayList<>();
names.add("Alice"); // O(1) amortized
names.add(0, "Bob"); // O(n) — shifts elements
names.get(0); // O(1) — direct index access
names.remove(0); // O(n) — shifts elements
names.contains("Alice"); // O(n) — linear scan

// Pre-size when you know the capacity
List<Integer> large = new ArrayList<>(10_000);

// Immutable list (Java 9+)
List<String> immutable = List.of("a", "b", "c");
// immutable.add("d"); // UnsupportedOperationException

LinkedList — Doubly-Linked List#

// O(1) add/remove at head/tail, O(n) random access
LinkedList<String> queue = new LinkedList<>();
queue.addFirst("first"); // O(1)
queue.addLast("last"); // O(1)
queue.removeFirst(); // O(1)
queue.get(5); // O(n) — must traverse

// Use as Deque (double-ended queue)
Deque<String> deque = new LinkedList<>();
deque.offerFirst("front");
deque.offerLast("back");
deque.pollFirst(); // Remove from front
deque.pollLast(); // Remove from back

When to Use Which#

Operation	ArrayList	LinkedList
Random access (`get(i)`)	O(1) ✓	O(n) ✗
Append (`add(e)`)	O(1) amortized ✓	O(1) ✓
Insert at index	O(n)	O(1) if at iterator position
Memory overhead	Low (contiguous)	High (node objects + pointers)
Default choice	Yes	Rarely needed

Set Implementations#

// HashSet — O(1) add/remove/contains, unordered
Set<String> tags = new HashSet<>();
tags.add("java");
tags.add("java"); // Duplicate ignored
tags.contains("java"); // O(1)

// TreeSet — O(log n) operations, sorted order
Set<Integer> sorted = new TreeSet<>();
sorted.addAll(List.of(5, 2, 8, 1, 9));
// Iteration order: 1, 2, 5, 8, 9

NavigableSet<Integer> nav = new TreeSet<>(sorted);
nav.headSet(5); // {1, 2} — elements < 5
nav.tailSet(5); // {5, 8, 9} — elements >= 5
nav.subSet(2, 8); // {2, 5} — elements in [2, 8)

// LinkedHashSet — O(1) operations, insertion order preserved
Set<String> ordered = new LinkedHashSet<>();
ordered.add("first");
ordered.add("second");
ordered.add("third");
// Iteration order: first, second, third (insertion order)

// Set operations
Set<Integer> a = Set.of(1, 2, 3, 4);
Set<Integer> b = Set.of(3, 4, 5, 6);

// Union
Set<Integer> union = new HashSet<>(a);
union.addAll(b); // {1, 2, 3, 4, 5, 6}

// Intersection
Set<Integer> intersection = new HashSet<>(a);
intersection.retainAll(b); // {3, 4}

// Difference
Set<Integer> difference = new HashSet<>(a);
difference.removeAll(b); // {1, 2}

Map Implementations#

// HashMap — O(1) average, unordered
Map<String, Integer> wordCount = new HashMap<>();
wordCount.put("hello", 1);
wordCount.get("hello"); // 1
wordCount.getOrDefault("world", 0); // 0

// Compute patterns (Java 8+)
wordCount.merge("hello", 1, Integer::sum); // Increment or set to 1
wordCount.computeIfAbsent("key", k -> expensiveComputation(k));
wordCount.computeIfPresent("hello", (k, v) -> v + 1);

// TreeMap — O(log n), sorted by key
Map<String, Integer> sorted = new TreeMap<>(wordCount);
// Keys in alphabetical order

NavigableMap<LocalDate, Event> timeline = new TreeMap<>();
timeline.floorEntry(today); // Latest entry <= today
timeline.ceilingEntry(today); // Earliest entry >= today
timeline.subMap(startDate, endDate); // Range query

// LinkedHashMap — insertion order OR access order
// Access-order mode: LRU cache
Map<String, String> lruCache = new LinkedHashMap<>(16, 0.75f, true) {
 @Override
 protected boolean removeEldestEntry(Map.Entry<String, String> eldest) {
 return size() > 100; // Evict when cache exceeds 100 entries
 }
};

// Map.of (immutable, Java 9+)
Map<String, Integer> config = Map.of(
 "timeout", 30,
 "retries", 3,
 "port", 8080
);

Queue and Deque#

// PriorityQueue — min-heap by default
Queue<Integer> minHeap = new PriorityQueue<>();
minHeap.offer(5);
minHeap.offer(1);
minHeap.offer(3);
minHeap.poll(); // 1 (smallest)
minHeap.poll(); // 3
minHeap.poll(); // 5

// Max-heap
Queue<Integer> maxHeap = new PriorityQueue<>(Comparator.reverseOrder());

// Custom priority
record Task(String name, int priority) {}
Queue<Task> taskQueue = new PriorityQueue<>(
 Comparator.comparingInt(Task::priority).reversed()
);

// ArrayDeque — faster than LinkedList for stack/queue use
Deque<String> stack = new ArrayDeque<>();
stack.push("first"); // addFirst
stack.push("second");
stack.pop(); // "second" (LIFO)

Deque<String> queue = new ArrayDeque<>();
queue.offer("first"); // addLast
queue.offer("second");
queue.poll(); // "first" (FIFO)

Collections Utility Class#

import java.util.Collections;

// Sorting
List<String> names = new ArrayList<>(List.of("Charlie", "Alice", "Bob"));
Collections.sort(names); // Natural order
names.sort(Comparator.reverseOrder()); // Reverse
names.sort(Comparator.comparingInt(String::length)); // By length

// Multi-key sorting
List<Employee> employees = getEmployees();
employees.sort(
 Comparator.comparing(Employee::department)
 .thenComparing(Employee::salary, Comparator.reverseOrder())
 .thenComparing(Employee::name)
);

// Unmodifiable wrappers
List<String> readOnly = Collections.unmodifiableList(names);
// readOnly.add("x"); // UnsupportedOperationException

// Thread-safe wrappers (prefer ConcurrentHashMap for maps)
List<String> syncList = Collections.synchronizedList(new ArrayList<>());

// Searching (list must be sorted)
int index = Collections.binarySearch(sortedList, "target");

// Frequency and disjoint
int count = Collections.frequency(list, "hello");
boolean noCommon = Collections.disjoint(set1, set2);

Concurrent Collections#

import java.util.concurrent.*;

// ConcurrentHashMap — thread-safe, high-performance map
ConcurrentMap<String, AtomicLong> counters = new ConcurrentHashMap<>();

// Atomic compute operations (no external synchronization needed)
counters.computeIfAbsent("requests", k -> new AtomicLong()).incrementAndGet();

// CopyOnWriteArrayList — thread-safe list for read-heavy workloads
List<EventListener> listeners = new CopyOnWriteArrayList<>();
// Writes copy the entire array (expensive)
// Reads never block (fast, no locking)

// BlockingQueue — producer-consumer pattern
BlockingQueue<Task> workQueue = new LinkedBlockingQueue<>(1000);

// Producer
workQueue.put(new Task("process")); // Blocks if full

// Consumer
Task task = workQueue.take(); // Blocks if empty

// ConcurrentSkipListMap — sorted, concurrent map
ConcurrentNavigableMap<String, Integer> sortedConcurrent = new ConcurrentSkipListMap<>();

Hypothetical Use Case: In-Memory Index#

public class InvertedIndex {
 // word → set of document IDs containing that word
 private final Map<String, Set<String>> index = new HashMap<>();
 
 public void indexDocument(String docId, String content) {
 String[] words = content.toLowerCase().split("\\W+");
 for (String word : words) {
 index.computeIfAbsent(word, k -> new HashSet<>()).add(docId);
 }
 }
 
 public Set<String> search(String query) {
 String[] terms = query.toLowerCase().split("\\W+");
 
 // AND search: documents must contain ALL terms
 return Arrays.stream(terms)
 .map(term -> index.getOrDefault(term, Set.of()))
 .reduce((a, b) -> {
 Set<String> intersection = new HashSet<>(a);
 intersection.retainAll(b);
 return intersection;
 })
 .orElse(Set.of());
 }
 
 public List<Map.Entry<String, Integer>> topTerms(int n) {
 return index.entrySet().stream()
 .map(e -> Map.entry(e.getKey(), e.getValue().size()))
 .sorted(Map.Entry.<String, Integer>comparingByValue().reversed())
 .limit(n)
 .toList();
 }
}

Key Takeaways#

ArrayList is the default List — use LinkedList only for frequent head/tail operations
HashSet/HashMap for O(1) lookups — TreeSet/TreeMap when you need sorted order
Use Map.computeIfAbsent instead of check-then-put patterns
ArrayDeque > LinkedList for stack and queue use cases
Immutable collections (List.of, Map.of) for constants and return values
ConcurrentHashMap for thread-safe maps — never Collections.synchronizedMap
Choose the right collection based on access patterns, not familiarity

Leadership

Mon, 01 Jan 0001 00:00:00 +0000

Leadership#

Leadership in engineering is not about a title. It’s about influence, direction, and making others more effective. You can lead from any position — as a senior engineer, a tech lead, or even a mid-level developer who steps up during a crisis.

Technical Leadership Without Authority#

Most engineering leadership happens without direct reports. You influence through expertise, communication, and consistent behaviour — not by telling people what to do.

Logic & Proofs

Mon, 01 Jan 0001 00:00:00 +0000

Logic & Proofs#

Logic is the science of valid reasoning — the same principles that govern circuit design, type systems, query optimisation, and formal verification. Proof techniques are how you demonstrate that algorithms are correct and that security properties hold.

Propositional Logic#

Propositional logic deals with statements (propositions) that are either true or false, connected by logical operators.

Logical Connectives#

Connective	Symbol	Name	English	Example
NOT	¬	Negation	“not p”	¬(it is raining)
AND	∧	Conjunction	“p and q”	(sunny) ∧ (warm)
OR	∨	Disjunction	“p or q” (inclusive)	(admin) ∨ (owner)
XOR	⊕	Exclusive or	“p or q but not both”	(left) ⊕ (right)
IMPLIES	→	Implication	“if p then q”	(logged_in) → (has_session)
IFF	↔	Biconditional	“p if and only if q”	(x>0) ↔ (x is positive)

Truth Tables#

AND:

Nginx

Mon, 01 Jan 0001 00:00:00 +0000

What Is Nginx?#

Nginx (pronounced “engine-x”) is a high-performance web server, reverse proxy, and load balancer. Created by Igor Sysoev in 2004 to solve the C10K problem (handling 10,000+ concurrent connections), it uses an event-driven, asynchronous architecture that excels at serving static content and proxying to application backends.

Architecture#

flowchart TB
 CLIENTS["Clients<br/>(thousands of connections)"]
 MASTER["Master Process<br/>(reads config, manages workers)"]
 W1["Worker Process 1<br/>(event loop)"]
 W2["Worker Process 2<br/>(event loop)"]
 W3["Worker Process N<br/>(event loop)"]
 
 CLIENTS --> MASTER
 MASTER --> W1
 MASTER --> W2
 MASTER --> W3

Each worker process handles thousands of connections in a single thread using non-blocking I/O (epoll/kqueue). No thread-per-connection overhead.

Prompt Engineering

Mon, 01 Jan 0001 00:00:00 +0000

Prompt engineering is the discipline of crafting inputs to large language models to elicit desired outputs. It ranges from simple instruction writing to sophisticated multi-step reasoning strategies. Mastering prompt engineering is the highest-leverage skill for working with LLMs — it requires no training data, no compute, and produces immediate results.

Prerequisites#

Understanding of how language models generate text (autoregressive, token-by-token)
Familiarity with the transformer architecture and attention mechanism
Basic understanding of tokenization

Prompting Fundamentals#

A prompt is the text input provided to an LLM. The model generates a completion conditioned on this input. The quality, specificity, and structure of the prompt directly determine the quality of the output.

Protocols and Generics

Mon, 01 Jan 0001 00:00:00 +0000

Protocols and Generics#

Swift is a protocol-oriented language. Protocols define contracts that types must fulfill, while generics enable code reuse across types. Together, they form the backbone of Swift’s abstraction system.

Protocols#

A protocol defines a blueprint of methods, properties, and requirements:

protocol Drawable {
 var boundingBox: CGRect { get }
 func draw(on canvas: Canvas)
}

protocol Resizable {
 mutating func resize(by factor: Double)
}

Property Requirements#

Protocols specify whether properties must be gettable or gettable-and-settable:

protocol Named {
 var name: String { get } // Read-only requirement
 var nickname: String { get set } // Read-write requirement
}

// Conforming with stored property (satisfies both get and get/set)
struct User: Named {
 var name: String // var satisfies { get }
 var nickname: String // var satisfies { get set }
}

// Conforming with computed property
struct AnonymousUser: Named {
 var name: String { "Anonymous" } // Computed, read-only
 var nickname: String { // Computed, read-write
 get { "Anon" }
 set { /* ignore */ }
 }
}

Method Requirements#

protocol DataSource {
 func numberOfItems() -> Int
 func item(at index: Int) -> String
 mutating func addItem(_ item: String) // mutating for value types
}

Protocol Conformance#

struct Circle: Drawable, Resizable {
 var center: CGPoint
 var radius: Double

 var boundingBox: CGRect {
 CGRect(x: center.x - radius, y: center.y - radius,
 width: radius * 2, height: radius * 2)
 }

 func draw(on canvas: Canvas) {
 canvas.drawCircle(center: center, radius: radius)
 }

 mutating func resize(by factor: Double) {
 radius *= factor
 }
}

Protocol Inheritance#

Protocols can inherit from other protocols:

Push Notifications

Mon, 01 Jan 0001 00:00:00 +0000

The Push Architecture#

Push notifications on the web involve three actors and two separate APIs:

┌──────────────┐ ┌──────────────────┐ ┌──────────────┐
│ Your Server │ ──► │ Push Service │ ──► │ Browser + │
│ (backend) │ │ (vendor-operated)│ │ Service Worker│
└──────────────┘ └──────────────────┘ └──────────────┘
 │ │ │
 Web Push Protocol Vendor endpoint push event
 (RFC 8030) (FCM, Mozilla, etc.) in SW

Component	Role
Notification API	Displays notifications to the user (can be used without push)
Push API	Allows the server to send messages to the browser, even when the page is closed
Push Service	Vendor-operated server (FCM for Chrome, autopush for Firefox) that queues and delivers messages
VAPID	Voluntary Application Server Identification — authenticates your server to the push service

The Notification API#

The Notification API works independently of push — you can show notifications from page JavaScript without involving a server:

Python: Error Handling and Exceptions

Mon, 01 Jan 0001 00:00:00 +0000

The Exception Hierarchy#

graph TD
 BE["BaseException"] --> SE["SystemExit"]
 BE --> KI["KeyboardInterrupt"]
 BE --> GE["GeneratorExit"]
 BE --> EX["Exception"]
 
 EX --> AE["ArithmeticError"]
 EX --> LE["LookupError"]
 EX --> OSE["OSError"]
 EX --> VE["ValueError"]
 EX --> TE["TypeError"]
 EX --> RE["RuntimeError"]
 EX --> ATE["AttributeError"]
 
 AE --> ZDE["ZeroDivisionError"]
 AE --> OFE["OverflowError"]
 
 LE --> IE["IndexError"]
 LE --> KE["KeyError"]
 
 OSE --> FNF["FileNotFoundError"]
 OSE --> PE["PermissionError"]
 OSE --> CE["ConnectionError"]
 
 CE --> CRE["ConnectionRefusedError"]
 CE --> CTO["TimeoutError"]

Rule: Never catch BaseException — it swallows SystemExit and KeyboardInterrupt.

Replication

Mon, 01 Jan 0001 00:00:00 +0000

Replication#

Replication — maintaining copies of data on multiple nodes — serves three goals: fault tolerance (surviving node failures), lower latency (serving reads from nearby replicas), and higher throughput (distributing load). The challenge is keeping replicas consistent while maximising performance and availability.

This section covers the three fundamental replication topologies, the consistency problems each introduces, and the strategies for resolving conflicts.

Single-Leader Replication#

In single-leader (also called primary-backup or master-slave) replication, one node is designated the leader. All writes go to the leader, which replicates changes to followers.

Structs & Methods

Mon, 01 Jan 0001 00:00:00 +0000

Structs & Methods#

Go has no classes. Structs are the primary way to define custom data types, and methods are functions with a receiver parameter that give structs behaviour.

Defining Structs#

type User struct {
 ID int
 FirstName string
 LastName string
 Email string
 CreatedAt time.Time
}

Creating Instances#

// Named fields (preferred — order-independent, self-documenting)
u := User{
 ID: 1,
 FirstName: "Alice",
 LastName: "Chen",
 Email: "alice@example.com",
 CreatedAt: time.Now(),
}

// Positional (fragile — breaks if fields are reordered)
u := User{1, "Alice", "Chen", "alice@example.com", time.Now()}

// Zero value
var u User // all fields are zero values

// Pointer
u := &User{ID: 1, FirstName: "Alice"}

Accessing Fields#

fmt.Println(u.FirstName)
u.Email = "new@example.com"

// Pointer access — Go auto-dereferences
p := &u
fmt.Println(p.FirstName) // same as (*p).FirstName

Methods#

A method is a function with a receiver — it binds the function to a type:

Submodules & Subtrees

Mon, 01 Jan 0001 00:00:00 +0000

Submodules & Subtrees#

When a project depends on code from another repository, Git offers two main strategies: submodules (linked pointer to an external repo) and subtrees (merged copy of an external repo). Each has distinct trade-offs for dependency management, monorepo architecture, and team workflows.

Submodules#

A submodule is a reference to a specific commit in another repository. The parent repo stores only the URL and commit SHA — not the actual files.

Support Vector Machines

Mon, 01 Jan 0001 00:00:00 +0000

Support Vector Machines#

Support Vector Machines (SVMs) find the optimal separating hyperplane between classes by maximising the margin — the distance between the decision boundary and the nearest data points. With the kernel trick, SVMs can learn non-linear boundaries while working in high-dimensional space efficiently.

Linear SVM#

The Maximum Margin Classifier#

Given two linearly separable classes, infinitely many hyperplanes can separate them. SVM chooses the one with the largest margin — the greatest distance to the nearest training points from either class.

Text Processing

Mon, 01 Jan 0001 00:00:00 +0000

The Big Three: grep, sed, awk#

These three tools handle 90% of text processing needs on any Unix system:

Tool	Purpose	Think of it as
`grep`	Find lines matching a pattern	Filter
`sed`	Transform text (find & replace)	Editor
`awk`	Process structured/columnar data	Spreadsheet

flowchart LR
 Input["Input Text"] --> grep["grep: FIND"]
 Input --> sed["sed: TRANSFORM"]
 Input --> awk["awk: PROCESS"]
 grep --> Lines["Matching lines"]
 sed --> Modified["Modified text"]
 awk --> Computed["Computed results"]

grep — Search for Patterns#

Basic Usage#

grep "error" logfile.txt # lines containing "error"
grep -i "error" logfile.txt # case-insensitive
grep -n "error" logfile.txt # show line numbers
grep -c "error" logfile.txt # count matches
grep -l "TODO" src/*.py # list files with matches
grep -L "test" src/*.py # list files WITHOUT matches
grep -v "debug" logfile.txt # invert (exclude matches)
grep -r "TODO" src/ # recursive search
grep -w "error" logfile.txt # whole word only

Extended Regex (`-E`)#

grep -E "error|warning|critical" log # alternation (OR)
grep -E "^[0-9]{4}-[0-9]{2}" log # lines starting with date
grep -E "HTTP/[12]\.[01]\" [45]" log # 4xx/5xx responses
grep -oE "[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+" log # extract IPs

Common Patterns#

# Find function definitions in Python
grep -rn "^def " src/

# Find TODO/FIXME comments
grep -rn "TODO\|FIXME\|HACK" --include="*.py" .

# Count errors per log file
grep -c "ERROR" /var/log/*.log

# Lines between two patterns
grep -A5 "Exception" log # 5 lines After match
grep -B3 "Exception" log # 3 lines Before match
grep -C2 "Exception" log # 2 lines Context (before + after)

sed — Stream Editor#

Substitution (Most Common Use)#

# Replace first occurrence per line
sed 's/old/new/' file

# Replace ALL occurrences per line
sed 's/old/new/g' file

# Case-insensitive replace
sed 's/error/ERROR/gi' file

# In-place editing
sed -i 's/old/new/g' file # Linux
sed -i '' 's/old/new/g' file # macOS (requires empty string)

Line Operations#

# Delete lines
sed '/^#/d' file # delete comment lines
sed '/^$/d' file # delete empty lines
sed '1,5d' file # delete lines 1-5

# Print specific lines
sed -n '10,20p' file # print lines 10-20
sed -n '/START/,/END/p' file # print between patterns

# Insert/append
sed '1i\Header line' file # insert before line 1
sed '$a\Footer line' file # append after last line

Practical Examples#

# Remove trailing whitespace
sed 's/[[:space:]]*$//' file

# Replace environment variable placeholders
sed "s/\${DB_HOST}/$DB_HOST/g" template.conf > config.conf

# Comment out a line
sed -i '/dangerous_setting/s/^/# /' config.ini

# Extract value from key=value
sed -n 's/^version=//p' config.ini

Multiple Operations#

# Chain with -e
sed -e 's/foo/bar/g' -e 's/baz/qux/g' file

# Or use semicolons
sed 's/foo/bar/g; s/baz/qux/g' file

awk — Pattern Scanning and Processing#

Column Extraction#

# Print specific columns (whitespace-delimited)
awk '{print $1, $3}' file

# Custom delimiter
awk -F',' '{print $2}' data.csv # CSV
awk -F':' '{print $1}' /etc/passwd # colon-separated

Filtering#

# Where clause (column conditions)
awk '$3 > 100 {print $1, $3}' file
awk '$2 == "ERROR" {print}' log
awk 'NF > 3' file # lines with more than 3 fields
awk 'length > 80' file # lines longer than 80 chars

Built-in Variables#

Variable	Meaning
`$0`	Entire line
`$1`-`$N`	Field N
`NR`	Current line number
`NF`	Number of fields in current line
`FS`	Field separator (input)
`OFS`	Output field separator

# Number lines
awk '{print NR": "$0}' file

# Print last field
awk '{print $NF}' file

# Print second-to-last field
awk '{print $(NF-1)}' file

Aggregation#

# Sum a column
awk '{sum += $2} END {print "Total:", sum}' sales.txt

# Average
awk '{sum += $2; n++} END {print "Avg:", sum/n}' data.txt

# Count by category
awk '{count[$1]++} END {for (k in count) print k, count[k]}' log

# Max value
awk 'BEGIN{max=0} $3>max{max=$3} END{print "Max:", max}' data.txt

BEGIN and END Blocks#

awk '
BEGIN { FS=","; OFS="\t"; print "Name\tAge\tCity" }
NR > 1 { print $1, $2, $3 }
END { print "---"; print NR-1, "records processed" }
' data.csv

cut, sort, uniq — Simple but Powerful#

cut — Extract Columns#

cut -d',' -f1,3 data.csv # fields 1 and 3, comma-delimited
cut -d':' -f1 /etc/passwd # usernames
cut -c1-10 file # characters 1-10

sort#

sort file # alphabetical
sort -n file # numeric
sort -k2 -t',' file # by field 2, comma-delimited
sort -r file # reverse
sort -u file # unique (deduplicate)
sort -h file # human-readable numbers (1K, 2M)

uniq (requires sorted input)#

sort file | uniq # remove duplicates
sort file | uniq -c # count occurrences
sort file | uniq -d # show only duplicates
sort file | uniq -c | sort -rn # frequency ranking

Combining Tools#

# Top 10 IP addresses in access log
awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -10

# Count HTTP status codes
awk '{print $9}' access.log | sort | uniq -c | sort -rn

# Find largest files
find . -type f -exec du -h {} \; | sort -rh | head -20

# Extract unique error messages
grep "ERROR" app.log | sed 's/.*ERROR //' | sort -u

Key Takeaways#

grep finds, sed transforms, awk processes — know which to reach for
grep -E for extended regex (alternation, quantifiers)
sed ’s/old/new/g’ for find-and-replace (add -i for in-place)
awk ‘{print $N}’ for column extraction
sort | uniq -c | sort -rn is the frequency analysis pattern
Combine tools with pipes — each does one thing well

The Preprocessor & Compilation

Mon, 01 Jan 0001 00:00:00 +0000

The Preprocessor & Compilation#

The C preprocessor is a text substitution engine that runs before compilation. It handles file inclusion, macro expansion, and conditional compilation. Understanding it is essential because it explains header files, include guards, and half the cryptic compiler errors you will encounter.

#include#

#include <stdio.h> // search system include paths
#include "myheader.h" // search current directory first, then system paths

#include literally copies the file contents into your source. A header file is just a C file that gets pasted in.

Transport Layer: TCP vs UDP

Mon, 01 Jan 0001 00:00:00 +0000

Role of the Transport Layer#

The transport layer provides end-to-end communication between applications on different hosts. It multiplexes connections using ports and (in TCP’s case) ensures reliable delivery.

flowchart TD
 App1["Web Browser (port 443)"] --> TL["Transport Layer"]
 App2["SSH Client (port 22)"] --> TL
 App3["DNS Client (port 53)"] --> TL
 TL --> NL["Network Layer (IP)"]

TCP (Transmission Control Protocol)#

Reliable, ordered, connection-oriented. Used when every byte must arrive correctly.

Three-Way Handshake#

sequenceDiagram
 participant C as Client
 participant S as Server
 
 C->>S: SYN (seq=100)
 S->>C: SYN-ACK (seq=300, ack=101)
 C->>S: ACK (seq=101, ack=301)
 Note over C,S: Connection established

TCP Features#

Feature	Mechanism
Reliability	Acknowledgments + retransmission
Ordering	Sequence numbers
Flow control	Sliding window (receiver advertises capacity)
Congestion control	Slow start, congestion avoidance, fast retransmit
Connection state	SYN → ESTABLISHED → FIN → CLOSED

TCP Connection Teardown#

sequenceDiagram
 participant C as Client
 participant S as Server
 
 C->>S: FIN (I'm done sending)
 S->>C: ACK
 S->>C: FIN (I'm done too)
 C->>S: ACK
 Note over C: TIME_WAIT (2×MSL)

TIME_WAIT: Client waits ~60s to ensure the final ACK arrives. This is why you see many TIME_WAIT connections on busy servers.

Trees

Mon, 01 Jan 0001 00:00:00 +0000

Trees#

Trees are the most frequently tested data structure in interviews. Most tree problems reduce to choosing the right traversal order and deciding what information to pass up or down the recursion.

Pattern Summary#

Pattern	Time	Space	When to Use
DFS (Recursive)	O(n)	O(h)	Most tree problems — default choice
BFS (Level-order)	O(n)	O(w)	Level-by-level processing, shortest path
BST Operations	O(h)	O(h)	Search, insert, delete, validate
Construction	O(n)	O(n)	Build tree from traversal arrays
Path Problems	O(n)	O(h)	Sum paths, max path, diameter

h = height (log n balanced, n worst case), w = max width

TypeScript: The Type System

Mon, 01 Jan 0001 00:00:00 +0000

What TypeScript Is#

TypeScript is a structural type system layered on top of JavaScript. Types exist only at compile time — they are completely erased before execution. The output is plain JavaScript.

flowchart LR
 TS[".ts source"] --> TSC["tsc (compiler)"]
 TSC --> TC["Type Checking<br>(compile-time only)"]
 TSC --> JS[".js output<br>(no types)"]
 TC -->|"Errors"| Dev["Developer"]
 JS --> Runtime["V8 / Node.js"]

Structural vs Nominal Typing#

// TypeScript is STRUCTURAL — shape matters, not name
interface Point { x: number; y: number }
interface Coordinate { x: number; y: number }

const p: Point = { x: 1, y: 2 };
const c: Coordinate = p; // OK! Same shape = compatible

// In Java/C# (nominal), this would fail — different types despite same shape

Basic Types#

// Primitives
const name: string = "Alice";
const age: number = 30;
const active: boolean = true;
const big: bigint = 9007199254740993n;
const id: symbol = Symbol("id");

// Special types
const nothing: null = null;
const missing: undefined = undefined;
const anything: any = "escape hatch"; // disables type checking
const safe: unknown = getUserInput(); // type-safe alternative to any

// void — function returns nothing
function log(msg: string): void { console.log(msg); }

// never — function never returns (throws or infinite loop)
function fail(msg: string): never { throw new Error(msg); }

Arrays and Tuples#

// Arrays (homogeneous)
const numbers: number[] = [1, 2, 3];
const names: Array<string> = ["a", "b"]; // generic syntax

// Tuples (fixed-length, heterogeneous)
const pair: [string, number] = ["age", 30];
const triple: [number, number, number] = [41.38, 2.17, 12];

// Named tuples (documentation only)
type Coordinate = [lat: number, lng: number, alt?: number];

// Rest elements in tuples
type StringAndNumbers = [string, ...number[]];
const data: StringAndNumbers = ["header", 1, 2, 3, 4];

Interfaces and Type Aliases#

Interfaces#

interface User {
 readonly id: string; // immutable after creation
 name: string;
 email: string;
 age?: number; // optional
 role: "admin" | "user"; // literal type
}

// Extension
interface Admin extends User {
 permissions: string[];
 department: string;
}

// Declaration merging (interfaces can be extended across files)
interface Window {
 myApp: { version: string }; // adds to existing Window interface
}

// Index signatures
interface Dictionary {
 [key: string]: string; // any string key → string value
}

// Call signatures
interface Formatter {
 (value: unknown): string;
 locale: string;
}

Type Aliases#

// Unions (OR)
type Status = "loading" | "success" | "error";
type ID = string | number;

// Intersections (AND)
type Admin = User & { permissions: string[] };

// Mapped types, conditional types, template literals — only with type
type Readonly<T> = { readonly [K in keyof T]: T[K] };

// Function types
type Handler = (event: Event) => void;
type AsyncFn<T> = () => Promise<T>;

When to Use Which#

Use `interface` when…	Use `type` when…
Defining object shapes	Unions or intersections
You want declaration merging	Mapped/conditional types
Extending other interfaces	Aliasing primitives or tuples
Public API contracts	Complex type computations

Generics#

Generics parameterize types, enabling reusable code that works with any type while preserving type safety.

Variables, Outputs, and Locals

Mon, 01 Jan 0001 00:00:00 +0000

Input Variables#

Variables parameterize your Terraform configuration — the same code works across environments by changing variable values.

Declaration#

variable "environment" {
 description = "Deployment environment (dev, staging, prod)"
 type = string
 default = "dev"
}

variable "instance_count" {
 description = "Number of EC2 instances to create"
 type = number
 # No default — Terraform will prompt or require a value
}

variable "enable_monitoring" {
 description = "Whether to enable detailed monitoring"
 type = bool
 default = false
}

Type Constraints#

# Simple types
variable "name" { type = string }
variable "port" { type = number }
variable "enabled" { type = bool }

# Collections
variable "availability_zones" {
 type = list(string)
}

variable "tags" {
 type = map(string)
}

variable "allowed_ports" {
 type = set(number)
}

# Complex objects
variable "database_config" {
 type = object({
 engine = string
 instance_class = string
 storage_gb = number
 multi_az = bool
 backup_window = optional(string, "03:00-04:00")
 })
}

# List of objects
variable "services" {
 type = list(object({
 name = string
 port = number
 cpu = number
 memory = number
 replicas = optional(number, 1)
 }))
}

Validation#

variable "environment" {
 type = string
 
 validation {
 condition = contains(["dev", "staging", "prod"], var.environment)
 error_message = "Environment must be dev, staging, or prod."
 }
}

variable "cidr_block" {
 type = string
 
 validation {
 condition = can(cidrhost(var.cidr_block, 0))
 error_message = "Must be a valid CIDR block (e.g., 10.0.0.0/16)."
 }
}

variable "instance_type" {
 type = string
 
 validation {
 condition = startswith(var.instance_type, "t3.") || startswith(var.instance_type, "t4g.")
 error_message = "Only t3 and t4g instance families are allowed."
 }
}

Sensitive Variables#

variable "db_password" {
 description = "Database master password"
 type = string
 sensitive = true # hidden in plan/apply output
}

Setting Variable Values#

flowchart TD
 A["How does Terraform get variable values?"]
 A --> B["1: -var flag (highest priority)"]
 A --> C["2: -var-file flag"]
 A --> D["3: *.auto.tfvars (auto-loaded)"]
 A --> E["4: terraform.tfvars (auto-loaded)"]
 A --> F["5: TF_VAR_* env variables"]
 A --> G["6: default value in variable block"]
 A --> H["7: Interactive prompt (lowest)"]

tfvars Files#

# config/variables/prod.tfvars
environment = "prod"
instance_count = 3
instance_type = "t3.large"
enable_monitoring = true

database_config = {
 engine = "postgres"
 instance_class = "db.r6g.large"
 storage_gb = 100
 multi_az = true
}

terraform plan -var-file="config/variables/prod.tfvars"

Environment Variables#

export TF_VAR_environment="prod"
export TF_VAR_db_password="secret123"
terraform plan

Auto-loaded Files#

Files matching these patterns are loaded automatically:

Versioning & Evolution

Mon, 01 Jan 0001 00:00:00 +0000

Versioning & Evolution#

APIs evolve. New features arrive, old patterns prove inadequate, business requirements shift. The challenge is evolving your API without breaking existing consumers. This section covers versioning strategies, change classification, and lifecycle management.

Why Version?#

Once an API has consumers, you can’t freely change it. Versioning gives you a mechanism to:

Introduce breaking changes without disrupting existing clients
Communicate the stability of different API surfaces
Allow consumers to migrate at their own pace
Maintain multiple versions during transition periods

Versioning Strategies#

URL Path Versioning#

The most common and visible approach. The version is part of the URL:

Virtualization and Containers

Mon, 01 Jan 0001 00:00:00 +0000

The Problem: “Works on My Machine”#

Applications depend on specific OS versions, libraries, and configurations. Virtualization and containers solve this by packaging the environment with the application.

Virtualization (Virtual Machines)#

A hypervisor creates virtual hardware — each VM runs its own complete OS:

flowchart TD
 subgraph Physical["Physical Server"]
 HV["Hypervisor"]
 subgraph VM1["VM 1"]
 OS1["Guest OS (Ubuntu)"]
 App1["App A"]
 end
 subgraph VM2["VM 2"]
 OS2["Guest OS (RHEL)"]
 App2["App B"]
 end
 end

Hypervisor Types#

Type	Description	Examples
Type 1 (bare-metal)	Runs directly on hardware	VMware ESXi, KVM, Xen, Hyper-V
Type 2 (hosted)	Runs on top of a host OS	VirtualBox, VMware Workstation

VM Characteristics#

Aspect	Value
Isolation	Full (separate kernel)
Boot time	Minutes
Size	Gigabytes (full OS)
Overhead	High (duplicate OS per VM)
Use case	Different OS requirements, strong isolation

Containers#

Containers share the host kernel but isolate the application’s filesystem, processes, and network:

Advanced Web APIs

Mon, 01 Jan 0001 00:00:00 +0000

Project Fugu and the Capabilities Gap#

Project Fugu is a cross-company effort (Google, Microsoft, Intel, Samsung) to bring native-level capabilities to the web platform while preserving security and user privacy. Named after the Japanese puffer fish — delicious if prepared correctly, lethal if not — the project acknowledges that exposing powerful APIs must be done carefully.

The Fugu tracker at fugu-tracker.web.app catalogues the status of dozens of APIs. This section covers the most impactful ones for PWA development.

Artifact Management

Mon, 01 Jan 0001 00:00:00 +0000

Artifact Management#

Artifacts — container images, packages, binaries — are the output of your CI pipeline and the input to your CD pipeline. Managing them well means consistent tagging, secure storage, vulnerability scanning, and retention policies.

Container Registries#

A container registry stores and distributes Docker/OCI images. Choose based on your ecosystem:

Registry	Best For	Authentication	Key Features
Amazon ECR	AWS-native workloads	IAM roles	Lifecycle policies, image scanning, cross-region replication
Docker Hub	Open-source, public images	Username/token	Official images, automated builds, rate limits on free tier
GitLab Container Registry	GitLab CI users	CI_JOB_TOKEN	Built-in, zero config, per-project isolation
GitHub Container Registry (ghcr.io)	GitHub Actions users	GITHUB_TOKEN	Free for public images, org-level packages
Google Artifact Registry	GCP workloads	Service accounts	Multi-format (Docker, npm, Maven, Python)
Azure Container Registry	Azure/AKS users	Managed Identity	Geo-replication, ACR Tasks for in-registry builds

ECR Lifecycle Policy Example#

Automatically clean old images to control storage costs:

Big-O Notation and Complexity Analysis

Mon, 01 Jan 0001 00:00:00 +0000

What Big-O Measures#

Big-O notation describes how an algorithm’s resource usage (time or space) scales as input size grows. It captures the upper bound of growth rate, ignoring constants and lower-order terms.

O(n) means: "grows linearly with input size"
O(n²) means: "grows quadratically with input size"
O(1) means: "constant — doesn't grow with input size"

Why Constants Don’t Matter#

# Both are O(n) — the constant factor (3 vs 1) is irrelevant at scale
def triple_loop(arr): # 3n operations
 for x in arr: pass
 for x in arr: pass
 for x in arr: pass

def single_loop(arr): # n operations
 for x in arr: pass

At n = 1,000,000: 3n = 3,000,000 vs n = 1,000,000. Both are “millions.” But O(n) vs O(n²): n = 1,000,000 vs n² = 1,000,000,000,000. That’s the difference that matters.

Browser Rendering Pipeline

Mon, 01 Jan 0001 00:00:00 +0000

From URL to Pixels#

When you navigate to a page, the browser performs a complex pipeline to turn HTML, CSS, and JavaScript into pixels on screen. Understanding this pipeline is essential for performance optimization.

flowchart LR
 HTML["Parse HTML"] --> DOM["DOM Tree"]
 CSS["Parse CSS"] --> CSSOM["CSSOM Tree"]
 DOM --> RT["Render Tree"]
 CSSOM --> RT
 RT --> Layout["Layout<br>(geometry)"]
 Layout --> Paint["Paint<br>(pixels)"]
 Paint --> Comp["Composite<br>(layers)"]
 Comp --> Screen["Display"]

Step 1: Parsing#

HTML → DOM#

The browser parses HTML into a tree of nodes (the DOM):

Cloud Migration Strategies

Mon, 01 Jan 0001 00:00:00 +0000

Cloud Migration Strategies#

Migration is not a technical project — it’s a business transformation with technical components. Most migration failures are caused by poor planning, not poor execution. This section covers the strategic patterns for moving workloads to the cloud and the tactical patterns for cutting over with minimal risk.

The 6 Rs of Migration#

Every workload in your portfolio maps to one of six strategies. The right strategy depends on the workload’s business value, technical complexity, and cloud readiness.

Collections & Iterators

Mon, 01 Jan 0001 00:00:00 +0000

Collections & Iterators#

Rust’s collections are heap-allocated, growable data structures. Its iterator system is the primary way to process collections — zero-cost abstractions that compile to code as fast as hand-written loops.

Vec — Dynamic Array#

// Create
let mut v: Vec<i32> = Vec::new();
let v = vec![1, 2, 3, 4, 5]; // macro shorthand
let v = vec![0; 10]; // 10 zeros
let v = Vec::with_capacity(100); // pre-allocate

// Access
let first = v[0]; // panics if out of bounds
let first = v.get(0); // returns Option<&T>
let last = v.last(); // Option<&T>

// Modify
v.push(6);
v.pop(); // returns Option<T>
v.insert(0, 10); // insert at index
v.remove(2); // remove at index
v.retain(|x| x % 2 == 0); // keep only evens
v.sort();
v.dedup(); // remove consecutive duplicates

// Iteration
for item in &v { ... } // borrow
for item in &mut v { ... } // mutable borrow
for item in v { ... } // consume (v is moved)

// Slicing
let slice: &[i32] = &v[1..4];

Vec Internals#

A Vec<T> is three values on the stack: pointer, length, capacity.

Dashboards & Visualisation

Mon, 01 Jan 0001 00:00:00 +0000

Dashboards & Visualisation#

Dashboards transform raw telemetry into actionable insights. A well-designed dashboard answers the question “is everything OK?” in under five seconds and guides investigation when it isn’t. This section covers Grafana, dashboard design principles, and the established methodologies for choosing what to visualise.

Methodologies: What to Measure#

The Four Golden Signals#

Defined in the Google SRE book, the four golden signals are the minimum set of metrics for any user-facing service:

DNS (Domain Name System)

Mon, 01 Jan 0001 00:00:00 +0000

What is DNS?#

DNS translates human-readable domain names into IP addresses. It’s a distributed, hierarchical database — the “phone book” of the internet.

flowchart LR
 Browser["www.example.com"] -->|DNS Query| Resolver["DNS Resolver"]
 Resolver -->|"93.184.216.34"| Browser

Without DNS, you’d need to memorize IP addresses for every website.

DNS Hierarchy#

flowchart TD
 Root[". (Root)"]
 Root --> COM[".com"]
 Root --> ORG[".org"]
 Root --> NET[".net"]
 COM --> Example["example.com"]
 COM --> Google["google.com"]
 Example --> WWW["www.example.com"]
 Example --> API["api.example.com"]

Level	Example	Managed By
Root	`.`	ICANN (13 root server clusters)
TLD	`.com`, `.org`, `.io`	Registry operators
Domain	`example.com`	Domain owner
Subdomain	`www.example.com`	Domain owner

DNS Resolution Process#

sequenceDiagram
 participant B as Browser
 participant R as Recursive Resolver (ISP/8.8.8.8)
 participant Root as Root Server
 participant TLD as .com TLD Server
 participant Auth as example.com Authoritative

 B->>R: What is www.example.com?
 R->>Root: What is www.example.com?
 Root->>R: I don't know, ask .com TLD (ns1.verisign.com)
 R->>TLD: What is www.example.com?
 TLD->>R: Ask example.com nameservers (ns1.example.com)
 R->>Auth: What is www.example.com?
 Auth->>R: A record: 93.184.216.34 (TTL: 3600)
 R->>B: 93.184.216.34 (cached for 1 hour)

Resolver Types#

Type	Role	Example
Stub resolver	Client’s local DNS (asks recursive)	Your OS
Recursive resolver	Does the full lookup	8.8.8.8, 1.1.1.1, ISP DNS
Authoritative	Holds the actual records	ns1.example.com

DNS Record Types#

Type	Purpose	Example
A	IPv4 address	`example.com → 93.184.216.34`
AAAA	IPv6 address	`example.com → 2606:2800:220:1:...`
CNAME	Alias to another name	`www.example.com → example.com`
MX	Mail server	`example.com → mail.example.com (pri 10)`
TXT	Arbitrary text	SPF, DKIM, domain verification
NS	Nameserver delegation	`example.com → ns1.example.com`
SOA	Zone authority metadata	Serial, refresh, retry, expire
SRV	Service location	`_http._tcp.example.com → web:8080`
PTR	Reverse DNS (IP → name)	`34.216.184.93 → example.com`
CAA	Certificate authority authorization	Which CAs can issue certs

CNAME Rules#

Cannot coexist with other records at the same name
Cannot be used at the zone apex (example.com) — only subdomains
AWS Route 53 “Alias” records solve the apex limitation

TTL (Time To Live)#

How long resolvers cache a record (in seconds):

Error Handling and Concurrency

Mon, 01 Jan 0001 00:00:00 +0000

Error Handling and Concurrency#

Swift provides structured error handling with throws/try/catch and modern concurrency with async/await, actors, and structured task hierarchies.

Error Handling#

Defining Errors#

Errors conform to the Error protocol (usually implemented with enums):

enum NetworkError: Error {
 case invalidURL
 case noConnection
 case timeout(seconds: Int)
 case httpError(statusCode: Int, message: String)
}

enum ValidationError: Error, LocalizedError {
 case tooShort(minimum: Int)
 case invalidFormat(expected: String)

 var errorDescription: String? {
 switch self {
 case .tooShort(let min):
 return "Input must be at least \(min) characters"
 case .invalidFormat(let expected):
 return "Expected format: \(expected)"
 }
 }
}

Throwing and Catching#

func fetchUser(id: Int) throws -> User {
 guard id > 0 else {
 throw ValidationError.invalidFormat(expected: "positive integer")
 }

 let response = try makeRequest(url: "/users/\(id)")

 guard response.statusCode == 200 else {
 throw NetworkError.httpError(
 statusCode: response.statusCode,
 message: response.body
 )
 }

 return try decode(response.body)
}

// Calling throwing functions
do {
 let user = try fetchUser(id: 42)
 print("Got user: \(user.name)")
} catch NetworkError.httpError(404, _) {
 print("User not found")
} catch NetworkError.timeout(let seconds) {
 print("Request timed out after \(seconds)s")
} catch {
 print("Unexpected error: \(error)") // 'error' is implicit
}

Try Variants#

Syntax	Behavior	Use When
`try`	Propagates error to caller	Inside a `do-catch` or throwing function
`try?`	Returns `nil` on error	You don’t care about the specific error
`try!`	Crashes on error	You’re certain it won’t fail (rare)

// try? — converts to optional
let user = try? fetchUser(id: 42) // User? — nil if any error

// Combine with nil coalescing
let name = (try? fetchUser(id: 1))?.name ?? "Unknown"

// try! — force (crashes on error)
let config = try! loadConfig() // Only for truly impossible failures

Rethrowing Functions#

Functions that only throw if their closure argument throws:

File I/O & System Calls

Mon, 01 Jan 0001 00:00:00 +0000

File I/O & System Calls#

C provides two layers of file I/O: the standard library (<stdio.h>) for portable, buffered I/O, and POSIX system calls for low-level, unbuffered access. Understanding both is essential for systems programming.

Standard I/O (`<stdio.h>`)#

Opening and Closing Files#

FILE *fp = fopen("data.txt", "r");
if (!fp) {
 perror("fopen"); // prints: "fopen: No such file or directory"
 return 1;
}

// ... use fp ...

fclose(fp);

Mode	Meaning
`"r"`	Read (file must exist)
`"w"`	Write (truncates or creates)
`"a"`	Append (creates if needed)
`"r+"`	Read + write (file must exist)
`"w+"`	Read + write (truncates or creates)
`"rb"`	Read binary
`"wb"`	Write binary

Reading#

// Read character by character
int ch;
while ((ch = fgetc(fp)) != EOF) {
 putchar(ch);
}

// Read line by line
char line[256];
while (fgets(line, sizeof(line), fp)) {
 printf("%s", line); // line includes '\n'
}

// Read formatted
int id;
char name[64];
fscanf(fp, "%d %63s", &id, name);

// Read binary
size_t n = fread(buffer, sizeof(int), count, fp);

Writing#

fprintf(fp, "User %d: %s\n", id, name);
fputs("hello\n", fp);
fputc('A', fp);
fwrite(data, sizeof(int), count, fp);

Standard Streams#

Stream	File Descriptor	Purpose
`stdin`	0	Standard input
`stdout`	1	Standard output (line-buffered)
`stderr`	2	Standard error (unbuffered)

POSIX I/O (Low-Level)#

Direct system calls — no buffering, no formatting, works with file descriptors (integers):

Graph Theory

Mon, 01 Jan 0001 00:00:00 +0000

Graph Theory#

Graphs model relationships between things — networks, dependencies, routes, social connections, state machines. Graph theory provides the language and algorithms for navigating, searching, and optimising these structures.

Definitions#

A graph G = (V, E) consists of:

V — a set of vertices (nodes)
E — a set of edges (connections between vertices)

Types of Graphs#

Type	Description	Example
Undirected	Edges have no direction	Facebook friendships
Directed (digraph)	Edges have direction (u → v)	Twitter follows, dependencies
Weighted	Edges have numeric weights	Road distances, latencies
Unweighted	All edges are equal	Simple connectivity
Simple	No self-loops or parallel edges	Most theoretical graphs
Multigraph	Allows parallel edges	Flight routes between cities
Complete (Kₙ)	Edge between every pair of vertices	Fully connected network
Bipartite	Vertices split into two sets, edges only between sets	Job assignments, matching
DAG	Directed acyclic graph	Task dependencies, Git commits

Terminology#

Term	Definition
Adjacent	Two vertices connected by an edge
Degree	Number of edges incident to a vertex
In-degree / Out-degree	Edges coming in / going out (directed graphs)
Path	Sequence of vertices where each consecutive pair is connected
Cycle	Path that starts and ends at the same vertex
Connected	Every vertex is reachable from every other (undirected)
Strongly connected	Every vertex reachable from every other following edge directions (directed)
Component	Maximal connected subgraph

The Handshaking Lemma#

In any undirected graph, the sum of all vertex degrees equals twice the number of edges:

Heaps & Priority Queues

Mon, 01 Jan 0001 00:00:00 +0000

Heaps & Priority Queues#

A heap gives you O(1) access to the min (or max) element and O(log n) insertion/extraction. This makes it the right choice when you repeatedly need the “best” element from a dynamic collection.

Pattern Summary#

Pattern	Time	Space	When to Use
Top-K Elements	O(n log k)	O(k)	K largest/smallest, K most frequent
Merge K Sorted	O(N log k)	O(k)	Combine k sorted sources
Two Heaps (Median)	O(n log n)	O(n)	Running median, stream statistics
Heap + Lazy Delete	O(n log n)	O(n)	Sliding window median, delayed removal
Task Scheduling	O(n log n)	O(n)	Cooldown, interval scheduling

N = total elements, k = number of sources or size constraint

Helm & Packaging

Mon, 01 Jan 0001 00:00:00 +0000

Helm & Packaging#

Packaging, templating, and managing Kubernetes applications — from Helm charts to Kustomize overlays, and when to use each approach.

Why Packaging Matters#

Raw Kubernetes manifests work for simple deployments, but real applications need:

Parameterisation — different values per environment (dev, staging, prod)
Reusability — deploy the same app pattern across teams/clusters
Versioning — track which version of your manifests are deployed
Dependencies — install prerequisites (databases, message queues) together

Helm Overview#

Helm is the package manager for Kubernetes. It packages manifests into charts, parameterises them with values, and tracks releases.

Hooks & Automation

Mon, 01 Jan 0001 00:00:00 +0000

Hooks & Automation#

Git hooks are scripts that run automatically at specific points in the Git workflow. They enable quality gates, enforced conventions, and automated workflows — all without requiring developers to remember manual steps.

How Hooks Work#

Hooks live in .git/hooks/ and execute at predefined trigger points. They are:

Executable scripts (any language — bash, Python, Node.js, etc.)
Named after their trigger (e.g., pre-commit, commit-msg)
Local by default — not committed to the repo (solved by frameworks like Husky)
Blocking — a non-zero exit code aborts the operation (for pre-* hooks)

# List available hook samples
ls .git/hooks/
# applypatch-msg.sample pre-commit.sample prepare-commit-msg.sample ...

# Enable a hook by removing .sample and making executable
cp .git/hooks/pre-commit.sample .git/hooks/pre-commit
chmod +x .git/hooks/pre-commit

Client-Side Hooks#

Hook Execution Timeline#

graph LR
 A[git commit] --> B[pre-commit]
 B --> C[prepare-commit-msg]
 C --> D[commit-msg]
 D --> E[post-commit]
 
 F[git push] --> G[pre-push]
 G --> H[post-push]
 
 style B fill:#e3f2fd
 style C fill:#e3f2fd
 style D fill:#e3f2fd
 style G fill:#e3f2fd

pre-commit#

Runs before the commit is created. Use for linting, formatting, and fast checks.

I/O Systems

Mon, 01 Jan 0001 00:00:00 +0000

I/O Systems#

Input/Output systems bridge the gap between the CPU and the external world — disks, networks, keyboards, GPUs. Understanding how the OS manages I/O is critical for writing performant software and diagnosing bottlenecks.

I/O Hardware Fundamentals#

Every I/O device connects to the system through a controller that exposes registers to the CPU:

Register	Purpose
Status	Indicates device state (busy, ready, error)
Command	CPU writes operations (read, write, seek)
Data-in	CPU reads data from device
Data-out	CPU writes data to device

Port-Mapped vs Memory-Mapped I/O#

Approach	Mechanism	Example
Port-mapped	Special CPU instructions (`in`/`out`) access device registers	x86 legacy devices
Memory-mapped	Device registers mapped to memory addresses	Modern GPUs, NVMe

Memory-mapped I/O is dominant today — it allows the same load/store instructions to access both RAM and devices.

Interfaces

Mon, 01 Jan 0001 00:00:00 +0000

Interfaces#

Interfaces are Go’s primary abstraction mechanism. Unlike Java or C#, Go interfaces are satisfied implicitly — a type implements an interface simply by having the right methods. No implements keyword. No declaration of intent.

Defining Interfaces#

type Reader interface {
 Read(p []byte) (n int, err error)
}

type Writer interface {
 Write(p []byte) (n int, err error)
}

type ReadWriter interface {
 Reader
 Writer
}

An interface is a set of method signatures. Any type that has those methods satisfies the interface.

Java: Exception Handling

Mon, 01 Jan 0001 00:00:00 +0000

Exception Hierarchy#

graph TD
 T["Throwable"] --> E["Exception"]
 T --> Er["Error"]
 
 E --> RE["RuntimeException<br>(unchecked)"]
 E --> CE["Checked Exceptions"]
 
 RE --> NPE["NullPointerException"]
 RE --> IAE["IllegalArgumentException"]
 RE --> ISE["IllegalStateException"]
 RE --> IOOBE["IndexOutOfBoundsException"]
 RE --> CCE["ClassCastException"]
 RE --> UOE["UnsupportedOperationException"]
 
 CE --> IOE["IOException"]
 CE --> SQLE["SQLException"]
 CE --> FNFE["FileNotFoundException"]
 
 Er --> OOM["OutOfMemoryError"]
 Er --> SOE["StackOverflowError"]
 
 style RE fill:#fff3e0
 style CE fill:#e1f5fe
 style Er fill:#ffcdd2

Checked vs Unchecked#

	Checked	Unchecked (Runtime)	Error
Must handle?	Yes (catch or declare)	No	No
Represents	Recoverable conditions	Programming bugs	JVM failures
Examples	IOException, SQLException	NullPointerException, IllegalArgumentException	OutOfMemoryError
Declare with	`throws` in signature	Nothing	Nothing

Try-With-Resources#

// Automatic resource management (Java 7+)
// Resources implementing AutoCloseable are closed automatically
public String readFile(Path path) throws IOException {
 try (var reader = Files.newBufferedReader(path);
 var lines = reader.lines()) {
 return lines.collect(Collectors.joining("\n"));
 }
 // reader.close() called automatically, even on exception
}

// Multiple resources — closed in reverse order
public void copyFile(Path source, Path dest) throws IOException {
 try (var in = new BufferedInputStream(Files.newInputStream(source));
 var out = new BufferedOutputStream(Files.newOutputStream(dest))) {
 in.transferTo(out);
 }
}

// Custom AutoCloseable
public class DatabaseConnection implements AutoCloseable {
 private final Connection conn;
 
 public DatabaseConnection(String url) throws SQLException {
 this.conn = DriverManager.getConnection(url);
 }
 
 @Override
 public void close() throws SQLException {
 if (conn != null && !conn.isClosed()) {
 conn.close();
 }
 }
}

// Usage
try (var db = new DatabaseConnection("jdbc:postgresql://localhost/mydb")) {
 db.query("SELECT * FROM users");
} // Automatically closed

Custom Exception Design#

// Base application exception
public class AppException extends RuntimeException {
 private final String errorCode;
 private final Map<String, Object> context;
 
 public AppException(String message, String errorCode) {
 this(message, errorCode, null, Map.of());
 }
 
 public AppException(String message, String errorCode, Throwable cause, Map<String, Object> context) {
 super(message, cause);
 this.errorCode = errorCode;
 this.context = Map.copyOf(context);
 }
 
 public String errorCode() { return errorCode; }
 public Map<String, Object> context() { return context; }
}

// Specific exceptions
public class EntityNotFoundException extends AppException {
 public EntityNotFoundException(String entityType, Object id) {
 super(
 "%s with id '%s' not found".formatted(entityType, id),
 "NOT_FOUND",
 null,
 Map.of("entityType", entityType, "id", id.toString())
 );
 }
}

public class ValidationException extends AppException {
 private final List<FieldError> fieldErrors;
 
 public ValidationException(List<FieldError> errors) {
 super("Validation failed: " + errors.size() + " error(s)", "VALIDATION_ERROR");
 this.fieldErrors = List.copyOf(errors);
 }
 
 public List<FieldError> fieldErrors() { return fieldErrors; }
 
 public record FieldError(String field, String message, Object rejectedValue) {}
}

public class ConflictException extends AppException {
 public ConflictException(String resource, String conflictReason) {
 super(
 "Conflict on %s: %s".formatted(resource, conflictReason),
 "CONFLICT",
 null,
 Map.of("resource", resource, "reason", conflictReason)
 );
 }
}

Exception Handling Patterns#

Translate Low-Level to Domain Exceptions#

public class UserRepository {
 public User findById(String id) {
 try {
 return jdbcTemplate.queryForObject(
 "SELECT * FROM users WHERE id = ?",
 userRowMapper, id
 );
 } catch (EmptyResultDataAccessException e) {
 throw new EntityNotFoundException("User", id);
 } catch (DataAccessException e) {
 throw new AppException(
 "Database error while fetching user",
 "DB_ERROR", e, Map.of("userId", id)
 );
 }
 }
}

Multi-Catch and Rethrow#

public void processFile(Path path) {
 try {
 var content = Files.readString(path);
 var data = parseJson(content);
 validate(data);
 save(data);
 } catch (IOException | JsonParseException e) {
 // Multi-catch: handle multiple exception types the same way
 throw new ProcessingException("Failed to process " + path, e);
 } catch (ValidationException e) {
 // Handle differently
 log.warn("Validation failed for {}: {}", path, e.fieldErrors());
 throw e; // Rethrow as-is
 }
}

Finally vs Try-With-Resources#

// OLD way (pre-Java 7) — error-prone
Connection conn = null;
try {
 conn = getConnection();
 // use connection
} finally {
 if (conn != null) {
 try {
 conn.close(); // close() can also throw!
 } catch (SQLException e) {
 log.error("Failed to close connection", e);
 }
 }
}

// MODERN way — clean and correct
try (var conn = getConnection()) {
 // use connection
}
// Suppressed exceptions from close() are attached to the primary exception

Optional — Avoiding NullPointerException#

import java.util.Optional;

// Creating Optionals
Optional<String> present = Optional.of("hello"); // Must be non-null
Optional<String> empty = Optional.empty();
Optional<String> nullable = Optional.ofNullable(null); // May be null

// Using Optionals (NEVER call .get() without checking)
public Optional<User> findUser(String id) {
 User user = database.query(id);
 return Optional.ofNullable(user);
}

// Consuming
findUser("123")
 .map(User::email)
 .filter(email -> email.contains("@"))
 .ifPresentOrElse(
 email -> sendNotification(email),
 () -> log.warn("User not found or invalid email")
 );

// Chaining
String displayName = findUser("123")
 .map(User::displayName)
 .orElse("Anonymous");

// orElseThrow (preferred over .get())
User user = findUser("123")
 .orElseThrow(() -> new EntityNotFoundException("User", "123"));

// Flattening nested Optionals
Optional<String> city = findUser("123")
 .flatMap(User::address) // address() returns Optional<Address>
 .map(Address::city);

// Stream integration (Java 9+)
List<String> emails = userIds.stream()
 .map(this::findUser)
 .flatMap(Optional::stream) // Filters out empty Optionals
 .map(User::email)
 .toList();

Optional Anti-Patterns#

// BAD: Optional as method parameter
void process(Optional<String> name) { } // Don't do this

// GOOD: Use overloading or nullable parameter
void process(String name) { }
void process() { process(null); }

// BAD: Optional for fields
class User {
 Optional<String> middleName; // Don't do this
}

// GOOD: Nullable field, Optional return
class User {
 private String middleName; // May be null
 public Optional<String> middleName() { return Optional.ofNullable(middleName); }
}

// BAD: isPresent() + get()
if (optional.isPresent()) {
 doSomething(optional.get());
}

// GOOD: ifPresent or map
optional.ifPresent(this::doSomething);
optional.map(this::transform).orElse(defaultValue);

Hypothetical Use Case: Resilient Service Layer#

public class OrderService {
 private final OrderRepository orderRepo;
 private final PaymentClient paymentClient;
 private final InventoryClient inventoryClient;
 
 public Order placeOrder(OrderRequest request) {
 // Validate
 var errors = validate(request);
 if (!errors.isEmpty()) {
 throw new ValidationException(errors);
 }
 
 // Check inventory
 try {
 inventoryClient.reserve(request.items());
 } catch (InsufficientStockException e) {
 throw new ConflictException("inventory", 
 "Items out of stock: " + e.unavailableItems());
 }
 
 // Process payment
 PaymentResult payment;
 try {
 payment = paymentClient.charge(request.paymentMethod(), request.total());
 } catch (PaymentDeclinedException e) {
 inventoryClient.release(request.items()); // Compensate
 throw new AppException("Payment declined", "PAYMENT_DECLINED", e,
 Map.of("reason", e.declineReason()));
 } catch (Exception e) {
 inventoryClient.release(request.items()); // Compensate
 throw new AppException("Payment processing failed", "PAYMENT_ERROR", e, Map.of());
 }
 
 // Create order
 var order = Order.create(request, payment.transactionId());
 return orderRepo.save(order);
 }
}

Key Takeaways#

Use unchecked exceptions for programming errors (IllegalArgumentException, IllegalStateException)
Use checked exceptions only when the caller can meaningfully recover
Try-with-resources for ALL closeable resources — never manual finally blocks
Translate exceptions at layer boundaries (SQL → domain exception)
Optional for return types that may be absent — never for fields or parameters
Never catch Exception or Throwable broadly — catch specific types
Include context in exceptions — what operation failed, with what inputs
Suppressed exceptions from try-with-resources are accessible via getSuppressed()

Kotlin Patterns and Ecosystem

Mon, 01 Jan 0001 00:00:00 +0000

Kotlin Patterns and Ecosystem#

Beyond the core language, Kotlin offers powerful patterns for building domain-specific languages, handling errors gracefully, and sharing code across platforms. This section covers DSLs, type-safe builders, the Result type, value classes, Kotlin Multiplatform, and kotlinx.serialization.

Domain-Specific Languages (DSLs)#

Kotlin’s combination of extension functions, lambdas with receivers, and operator overloading makes it ideal for building DSLs — small, specialised languages embedded in Kotlin:

// A simple HTML DSL
fun html(block: HtmlBuilder.() -> Unit): String {
 val builder = HtmlBuilder()
 builder.block()
 return builder.build()
}

class HtmlBuilder {
 private val elements = mutableListOf<String>()

 fun head(block: HeadBuilder.() -> Unit) {
 val head = HeadBuilder()
 head.block()
 elements.add("<head>${head.build()}</head>")
 }

 fun body(block: BodyBuilder.() -> Unit) {
 val body = BodyBuilder()
 body.block()
 elements.add("<body>${body.build()}</body>")
 }

 fun build() = "<html>${elements.joinToString("")}</html>"
}

Usage reads like a declarative specification:

Message Queues & Event-Driven Architecture

Mon, 01 Jan 0001 00:00:00 +0000

Message Queues & Event-Driven Architecture#

Asynchronous messaging is the backbone of scalable distributed systems. Instead of services waiting for each other synchronously, message queues decouple producers from consumers, enabling independent scaling, fault tolerance, and temporal decoupling.

Why Asynchronous Processing?#

Synchronous communication (request-response) creates tight coupling between services. If the downstream service is slow or down, the upstream service is directly impacted.

Concern	Synchronous	Asynchronous
Coupling	Tight — caller waits for response	Loose — producer doesn’t know consumer
Latency	Cumulative across call chain	Producer returns immediately
Failure handling	Cascading failures	Failures isolated to consumer
Scaling	Both sides must scale together	Producer and consumer scale independently
Ordering	Implicit (request-response)	Must be explicitly managed
Complexity	Simpler mental model	Requires queue infrastructure

When to Use Async#

Task takes > 500ms — don’t make the user wait for email sends, PDF generation, or analytics processing
Spiky traffic — buffer requests during peaks and process at a steady rate
Cross-service coordination — notify multiple services without coupling them
Retry-heavy operations — let the queue handle retries rather than the caller

Message Queue vs Event Stream#

These are fundamentally different communication models despite superficial similarities.

Modules

Mon, 01 Jan 0001 00:00:00 +0000

What is a Module?#

A module is a container for multiple resources that are used together. Every Terraform configuration is a module — the root module. When you call another module, it becomes a child module.

flowchart TD
 Root["Root Module<br>(environments/prod/main.tf)"]
 Root --> VPC["module.vpc<br>VPC + Subnets + IGW + NAT"]
 Root --> ECS["module.ecs_cluster<br>Cluster + Capacity Providers"]
 Root --> DB["module.database<br>RDS + Subnet Group + SG"]
 
 ECS --> SVC["module.api_service<br>Service + Task Def + ALB Rule"]
 
 VPC -.->|"vpc_id, subnet_ids"| ECS
 VPC -.->|"vpc_id, subnet_ids"| DB
 DB -.->|"connection_string"| SVC

Modules provide:

Monolith to Microservices

Mon, 01 Jan 0001 00:00:00 +0000

Monolith to Microservices#

Migrating from a monolith to microservices is one of the most impactful — and risky — architectural transitions a team can undertake. Success requires a deliberate, incremental strategy rather than a big-bang rewrite.

Why Migrate?#

Driver	Monolith Pain	Microservices Solution
Deployment speed	Entire app redeployed for any change	Deploy services independently
Team scaling	Merge conflicts, coordination overhead	Teams own independent services
Technology lock-in	One stack for everything	Polyglot — best tool per service
Fault isolation	One bug crashes everything	Failures contained per service
Scalability	Scale entire app for one hot path	Scale individual services

When NOT to Migrate#

Small team (< 10 engineers)
Well-understood domain with stable requirements
No deployment bottleneck
You cannot invest in platform infrastructure (CI/CD, observability, service mesh)

The Modular Monolith — First Step#

Before extracting services, modularize within the monolith:

MVCC Implementation

Mon, 01 Jan 0001 00:00:00 +0000

Why MVCC?#

Without concurrency control, simultaneous readers and writers would see inconsistent data — partial updates, phantom rows, or torn reads. Lock-based approaches (shared/exclusive locks on every row) are correct but cause readers to block writers and vice versa.

Multi-Version Concurrency Control (MVCC) solves this by keeping multiple versions of each row. Readers see a consistent snapshot without acquiring locks; writers create new versions instead of overwriting in place. The result: readers never block writers, and writers never block readers.

Networking: VPC

Mon, 01 Jan 0001 00:00:00 +0000

What is a VPC?#

A Virtual Private Cloud (VPC) is your isolated network within AWS. It’s a logically isolated section of the AWS cloud where you launch resources in a virtual network that you define.

flowchart TD
 subgraph VPC["VPC: 10.0.0.0/16"]
 subgraph AZ1["AZ 1a"]
 PubSub1["Public Subnet<br>10.0.1.0/24"]
 PrivSub1["Private Subnet<br>10.0.10.0/24"]
 end
 subgraph AZ2["AZ 1b"]
 PubSub2["Public Subnet<br>10.0.2.0/24"]
 PrivSub2["Private Subnet<br>10.0.20.0/24"]
 end
 
 IGW["Internet Gateway"]
 NAT["NAT Gateway"]
 end
 
 Internet["Internet"] --> IGW
 IGW --> PubSub1
 IGW --> PubSub2
 PubSub1 --> NAT
 NAT --> PrivSub1
 NAT --> PrivSub2

VPC Components#

CIDR Blocks#

CIDR (Classless Inter-Domain Routing) defines the IP address range for your VPC:

Node.js Runtime and Architecture

Mon, 01 Jan 0001 00:00:00 +0000

What Node.js Is#

Node.js is a JavaScript runtime built on Chrome’s V8 engine. It takes JavaScript out of the browser and into servers, CLIs, and system tools. Its key innovation: non-blocking I/O on a single thread using an event-driven architecture.

flowchart TD
 subgraph Node["Node.js Runtime"]
 V8["V8 Engine<br>(JavaScript execution)"]
 Bindings["C++ Bindings<br>(bridge JS ↔ system)"]
 UV["libuv<br>(async I/O, event loop)"]
 SL["Standard Library<br>(fs, http, crypto, stream...)"]
 end
 
 UV --> TP["Thread Pool<br>(4 threads default)<br>DNS, fs, crypto, zlib"]
 UV --> OS["OS Async Primitives<br>epoll (Linux)<br>kqueue (macOS)<br>IOCP (Windows)"]
 
 App["Your JavaScript"] --> SL
 SL --> Bindings
 Bindings --> UV

The Node.js Event Loop (Detailed)#

Unlike the simplified browser event loop, Node.js has phases:

Normalization and Schema Design

Mon, 01 Jan 0001 00:00:00 +0000

What Is Normalization?#

Normalization is the process of organizing database tables to reduce redundancy and prevent anomalies (insertion, update, and deletion anomalies). Each normal form builds on the previous one.

Why Normalize?#

Problem	Without Normalization	With Normalization
Update anomaly	Change department name in 500 employee rows	Change it in 1 row in departments table
Insertion anomaly	Can’t add a department with no employees	Department exists independently
Deletion anomaly	Deleting last employee removes department info	Department persists regardless
Storage	Repeated data wastes space	Each fact stored once
Consistency	Same data might disagree across rows	Single source of truth

Normal Forms#

First Normal Form (1NF)#

Rule: Every column contains only atomic (indivisible) values. No repeating groups or arrays.

OAuth 2.0 & OpenID Connect

Mon, 01 Jan 0001 00:00:00 +0000

Overview#

OAuth 2.0 is an authorisation framework that allows third-party applications to obtain limited access to a user’s resources without exposing their credentials. OpenID Connect (OIDC) is an identity layer built on top of OAuth 2.0 that adds authentication.

Protocol	Purpose	Result
OAuth 2.0	Authorisation (what can the app do?)	Access Token
OpenID Connect	Authentication (who is the user?)	ID Token + Access Token

OAuth 2.0 Roles#

Role	Description
Resource Owner	The user who owns the data
Client	The application requesting access
Authorization Server	Issues tokens (e.g., Keycloak, Auth0, Okta)
Resource Server	The API that holds protected resources

Grant Types#

Authorization Code (with PKCE) — Recommended for All Clients#

This is the only recommended grant for browser-based and mobile applications. PKCE (Proof Key for Code Exchange) prevents authorisation code interception.

Operating System Security

Mon, 01 Jan 0001 00:00:00 +0000

Why This Matters#

The operating system is the foundation of every workload. If the OS is compromised, everything running on it — applications, data, credentials — is compromised. OS hardening reduces the attack surface and limits what an attacker can do even after gaining initial access.

Hardening Principles#

Principle	Actions
Minimize attack surface	Remove unused packages, disable unnecessary services
Least privilege	Non-root execution, restricted sudo, capability-based access
Patch continuously	Automated updates, vulnerability scanning
Secure defaults	Disable root login, enforce strong auth
Defense in depth	Multiple layers (firewall + MAC + audit)
Immutability	Read-only filesystems, rebuild instead of patch

Linux Security Architecture#

flowchart TD
 User["User Space"]
 User --> DAC["DAC<br/>(file permissions)"]
 User --> MAC["MAC<br/>(SELinux/AppArmor)"]
 User --> Caps["Capabilities<br/>(fine-grained root)"]
 
 Kernel["Kernel Space"]
 DAC --> Kernel
 MAC --> Kernel
 Caps --> Kernel
 
 Kernel --> Seccomp["seccomp<br/>(syscall filter)"]
 Kernel --> Namespaces["Namespaces<br/>(isolation)"]
 Kernel --> Cgroups["cgroups<br/>(resource limits)"]
 Kernel --> Audit["auditd<br/>(logging)"]

Discretionary Access Control (DAC)#

Traditional Unix permissions:

Package Management

Mon, 01 Jan 0001 00:00:00 +0000

What Is Package Management?#

A package manager automates the process of installing, upgrading, configuring, and removing software. Instead of downloading binaries from websites and managing dependencies manually, you use a command-line tool that handles everything — fetching packages from repositories, resolving dependency trees, and tracking what’s installed.

Why Package Managers Matter#

Without	With
Download from random websites	Install from trusted repositories
Manual dependency hunting	Automatic dependency resolution
No easy way to update all software	Single command to update everything
Unknown security status	Signed packages, vulnerability tracking
Difficult to reproduce environments	Scriptable, reproducible installs

Chocolatey (Windows)#

Chocolatey is the leading package manager for Windows, bringing Linux-style package management to the Windows ecosystem.

Partitioning & Sharding

Mon, 01 Jan 0001 00:00:00 +0000

Partitioning & Sharding#

When a dataset grows beyond what a single node can store or serve, it must be split across multiple nodes. Partitioning (also called sharding) divides data so each node is responsible for a subset. Combined with replication (Section 5), partitioning enables systems to scale both storage and throughput horizontally.

This section covers the strategies for partitioning data, the mechanisms for distributing partitions across nodes, and the problems that arise when you get it wrong.

Pipes, Redirection, and Process Substitution

Mon, 01 Jan 0001 00:00:00 +0000

The Unix Philosophy#

Write programs that do one thing and do it well. Write programs to work together.

Pipes are how you connect small tools into powerful workflows.

flowchart LR
 A["cat log"] -->|pipe| B["grep ERROR"]
 B -->|pipe| C["awk '{print $4}'"]
 C -->|pipe| D["sort"]
 D -->|pipe| E["uniq -c"]
 E -->|pipe| F["sort -rn"]
 F -->|pipe| G["head -10"]

Pipes (`|`)#

Connect stdout of one command to stdin of the next:

# Top 10 error sources from a log
cat access.log | grep "ERROR" | awk '{print $4}' | sort | uniq -c | sort -rn | head -10

# Count unique users currently logged in
who | awk '{print $1}' | sort -u | wc -l

# Find largest directories
du -sh */ | sort -rh | head -5

Pipe Behavior#

Each command in a pipe runs as a separate process
They run concurrently (not sequentially)
Data flows as a stream — no temp files needed
If any command in the pipe exits, the others receive SIGPIPE

File Descriptors#

Every process has three standard streams:

Problem Solving

Mon, 01 Jan 0001 00:00:00 +0000

Problem Solving#

Engineering is problem solving. But the hardest problems are rarely solved by staring at code harder — they require structured thinking, systematic investigation, and the discipline to resist jumping to conclusions.

Structured Thinking#

Before solving a problem, make sure you understand it. Most debugging time is wasted solving the wrong problem.

The Problem-Solving Loop#

flowchart TD
 A[Observe: What's actually happening?] --> B[Define: What should be happening?]
 B --> C[Gap: What's the difference?]
 C --> D[Hypothesise: What could cause this gap?]
 D --> E[Test: Which hypothesis is correct?]
 E --> F{Solved?}
 F -->|No| D
 F -->|Yes| G[Fix and verify]
 G --> H[Prevent: How do we stop this recurring?]

Common Thinking Traps#

Trap	Description	Defence
Jumping to solutions	Fixing before understanding	Force yourself to describe the problem in words first
Confirmation bias	Only looking for evidence that supports your theory	Actively try to disprove your hypothesis
Anchoring	Fixating on the first clue	Generate multiple hypotheses before investigating
Availability bias	“Last time it was X, so it’s probably X again”	Check the obvious, but don’t stop there
Sunk cost	“I’ve spent 3 hours on this approach, I can’t switch”	Time spent is gone. Switch if the evidence says to.

Root Cause Analysis#

Surface-level fixes create recurring problems. Root cause analysis digs deeper to find and address the underlying cause.

Python: Decorators, Generators, and Context Managers

Mon, 01 Jan 0001 00:00:00 +0000

Decorators — Functions That Modify Functions#

How Decorators Work#

# A decorator is a function that takes a function and returns a function
def my_decorator(func):
 def wrapper(*args, **kwargs):
 # Before the original function
 print(f"Calling {func.__name__}")
 result = func(*args, **kwargs)
 # After the original function
 print(f"{func.__name__} returned {result}")
 return result
 return wrapper

@my_decorator
def add(a, b):
 return a + b

# @my_decorator is syntactic sugar for:
# add = my_decorator(add)

add(3, 4)
# Output:
# Calling add
# Returned 7

sequenceDiagram
 participant Caller
 participant Wrapper
 participant Original
 
 Caller->>Wrapper: add(3, 4)
 Note over Wrapper: Pre-processing
 Wrapper->>Original: func(3, 4)
 Original-->>Wrapper: return 7
 Note over Wrapper: Post-processing
 Wrapper-->>Caller: return 7

Preserving Function Metadata#

from functools import wraps

def timer(func):
 """Decorator that measures execution time."""
 @wraps(func) # Preserves __name__, __doc__, __module__
 def wrapper(*args, **kwargs):
 import time
 start = time.perf_counter()
 result = func(*args, **kwargs)
 elapsed = time.perf_counter() - start
 print(f"{func.__name__} took {elapsed:.4f}s")
 return result
 return wrapper

@timer
def slow_function():
 """This docstring is preserved thanks to @wraps."""
 import time
 time.sleep(1)

slow_function.__name__ # "slow_function" (not "wrapper")
slow_function.__doc__ # "This docstring is preserved..."

Decorators with Arguments#

from functools import wraps

def retry(max_attempts: int = 3, exceptions: tuple = (Exception,)):
 """Decorator factory — returns a decorator configured with arguments."""
 def decorator(func):
 @wraps(func)
 def wrapper(*args, **kwargs):
 for attempt in range(1, max_attempts + 1):
 try:
 return func(*args, **kwargs)
 except exceptions as e:
 if attempt == max_attempts:
 raise
 print(f"Attempt {attempt} failed: {e}")
 return wrapper
 return decorator

@retry(max_attempts=5, exceptions=(ConnectionError, TimeoutError))
def fetch_data(url):
 ...

# Three levels:
# retry(max_attempts=5) → returns decorator
# decorator(fetch_data) → returns wrapper
# wrapper(url) → calls fetch_data with retry logic

Class-Based Decorators#

class CacheResult:
 """Decorator as a class — useful when you need state."""
 
 def __init__(self, ttl_seconds: int = 300):
 self.ttl = ttl_seconds
 self.cache = {}
 
 def __call__(self, func):
 @wraps(func)
 def wrapper(*args, **kwargs):
 import time
 key = (args, tuple(sorted(kwargs.items())))
 
 if key in self.cache:
 result, timestamp = self.cache[key]
 if time.time() - timestamp < self.ttl:
 return result
 
 result = func(*args, **kwargs)
 self.cache[key] = (result, time.time())
 return result
 
 wrapper.clear_cache = self.cache.clear
 return wrapper
 
 def __repr__(self):
 return f"CacheResult(ttl={self.ttl}, entries={len(self.cache)})"

@CacheResult(ttl_seconds=60)
def expensive_query(user_id: int) -> dict:
 ...

expensive_query.clear_cache() # Exposed method

Stacking Decorators#

# Decorators apply bottom-up (closest to function first)
@log_calls # 3rd: wraps the authenticated+validated function
@authenticate # 2nd: wraps the validated function
@validate_input # 1st: wraps the original function
def create_user(data: dict) -> User:
 ...

# Equivalent to:
# create_user = log_calls(authenticate(validate_input(create_user)))

Generators — Lazy Evaluation#

Generator Functions#

def fibonacci():
 """Infinite Fibonacci sequence — generates values on demand."""
 a, b = 0, 1
 while True:
 yield a
 a, b = b, a + b

# Only computes values as requested
fib = fibonacci()
next(fib) # 0
next(fib) # 1
next(fib) # 1
next(fib) # 2

# Take first 10
from itertools import islice
first_10 = list(islice(fibonacci(), 10))
# [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]

Generator vs List — Memory Comparison#

import sys

# List: ALL values in memory at once
squares_list = [x**2 for x in range(1_000_000)]
sys.getsizeof(squares_list) # ~8.4 MB

# Generator: ONE value at a time
squares_gen = (x**2 for x in range(1_000_000))
sys.getsizeof(squares_gen) # ~200 bytes (constant!)

# Both produce the same results
sum(x**2 for x in range(1_000_000)) # Generator expression in function call

yield from — Delegating to Sub-generators#

def flatten(nested):
 """Recursively flatten any nested iterable."""
 for item in nested:
 if isinstance(item, (list, tuple, set)):
 yield from flatten(item) # Delegate to recursive call
 else:
 yield item

list(flatten([1, [2, 3], [4, [5, 6]], 7]))
# [1, 2, 3, 4, 5, 6, 7]

# yield from also forwards .send() and .throw() to sub-generator
def chain(*iterables):
 for it in iterables:
 yield from it

list(chain([1, 2], [3, 4], [5, 6])) # [1, 2, 3, 4, 5, 6]

Generators as Coroutines (send/throw)#

def running_average():
 """Generator that accepts values via .send() and yields running average."""
 total = 0.0
 count = 0
 average = None
 
 while True:
 value = yield average # Receive value, yield current average
 if value is None:
 break
 total += value
 count += 1
 average = total / count

avg = running_average()
next(avg) # Prime the generator (advance to first yield)
avg.send(10) # 10.0
avg.send(20) # 15.0
avg.send(30) # 20.0
avg.send(None) # StopIteration

Practical Generator Patterns#

def read_in_chunks(file_path: str, chunk_size: int = 8192):
 """Read a large file in chunks without loading it all into memory."""
 with open(file_path, "rb") as f:
 while chunk := f.read(chunk_size):
 yield chunk

def batch(iterable, size: int):
 """Split an iterable into fixed-size batches."""
 from itertools import islice
 iterator = iter(iterable)
 while batch := list(islice(iterator, size)):
 yield batch

# Process 1M records in batches of 100
for record_batch in batch(range(1_000_000), 100):
 process_batch(record_batch)

def sliding_window(iterable, size: int):
 """Sliding window over an iterable."""
 from collections import deque
 window = deque(maxlen=size)
 for item in iterable:
 window.append(item)
 if len(window) == size:
 yield tuple(window)

list(sliding_window([1, 2, 3, 4, 5], 3))
# [(1,2,3), (2,3,4), (3,4,5)]

Context Managers — Resource Management#

The Protocol#

class ManagedResource:
 """Context manager using the protocol directly."""
 
 def __init__(self, name: str):
 self.name = name
 print(f"Creating {name}")
 
 def __enter__(self):
 """Called when entering 'with' block. Return value binds to 'as'."""
 print(f"Acquiring {self.name}")
 return self # The 'as' variable gets this value
 
 def __exit__(self, exc_type, exc_val, exc_tb):
 """Called when leaving 'with' block (even on exception).
 
 Args:
 exc_type: Exception class (or None if no exception)
 exc_val: Exception instance (or None)
 exc_tb: Traceback (or None)
 
 Returns:
 True to suppress the exception, False/None to propagate it.
 """
 print(f"Releasing {self.name}")
 if exc_type is not None:
 print(f"Exception occurred: {exc_val}")
 return False # Don't suppress exceptions

with ManagedResource("database") as db:
 print("Using database")
# Output:
# Creating database
# Acquiring database
# Using database
# Releasing database

contextlib — Decorator-Based Context Managers#

from contextlib import contextmanager, asynccontextmanager
import time

@contextmanager
def timer(label: str):
 """Measure execution time of a block."""
 start = time.perf_counter()
 try:
 yield # Control passes to the 'with' block
 finally:
 elapsed = time.perf_counter() - start
 print(f"{label}: {elapsed:.4f}s")

with timer("data processing"):
 process_large_dataset()

@contextmanager
def temporary_directory():
 """Create a temp dir, clean up after use."""
 import tempfile
 import shutil
 
 path = tempfile.mkdtemp()
 try:
 yield path
 finally:
 shutil.rmtree(path)

with temporary_directory() as tmpdir:
 # Work with tmpdir
 pass # Automatically cleaned up

Nested and Stacked Context Managers#

from contextlib import ExitStack

def process_files(file_paths: list[str]) -> list[str]:
 """Open multiple files safely — all closed even if one fails."""
 with ExitStack() as stack:
 files = [
 stack.enter_context(open(path, "r"))
 for path in file_paths
 ]
 return [f.read() for f in files]

# ExitStack also supports dynamic cleanup
@contextmanager
def managed_connection_pool(size: int):
 """Pool that cleans up all connections on exit."""
 with ExitStack() as stack:
 connections = []
 for _ in range(size):
 conn = create_connection()
 stack.callback(conn.close) # Register cleanup
 connections.append(conn)
 yield connections

Async Context Managers#

from contextlib import asynccontextmanager

@asynccontextmanager
async def async_db_session():
 """Async context manager for database sessions."""
 session = await create_session()
 try:
 yield session
 await session.commit()
 except Exception:
 await session.rollback()
 raise
 finally:
 await session.close()

async def get_user(user_id: int):
 async with async_db_session() as session:
 return await session.query(User).get(user_id)

Combining All Three#

Use Case: Instrumented Pipeline#

from functools import wraps
from contextlib import contextmanager
import time
import logging

logger = logging.getLogger(__name__)

# Decorator: adds timing and logging to any function
def instrumented(func):
 @wraps(func)
 def wrapper(*args, **kwargs):
 start = time.perf_counter()
 try:
 result = func(*args, **kwargs)
 elapsed = time.perf_counter() - start
 logger.info(f"{func.__name__} completed in {elapsed:.3f}s")
 return result
 except Exception as e:
 elapsed = time.perf_counter() - start
 logger.error(f"{func.__name__} failed after {elapsed:.3f}s: {e}")
 raise
 return wrapper

# Generator: lazy data pipeline
def read_records(source):
 """Generator: yields records one at a time."""
 for raw in source:
 yield parse_record(raw)

def filter_valid(records):
 """Generator: passes through only valid records."""
 for record in records:
 if record.is_valid():
 yield record

def transform(records):
 """Generator: applies business logic."""
 for record in records:
 yield record.transform()

# Context manager: manages the pipeline lifecycle
@contextmanager
def pipeline(source, destination):
 """Context manager: sets up and tears down the pipeline."""
 logger.info("Pipeline starting")
 dest = open_destination(destination)
 try:
 yield dest
 finally:
 dest.flush()
 dest.close()
 logger.info("Pipeline complete")

# Putting it all together
@instrumented
def run_etl(source_path: str, dest_path: str):
 with pipeline(source_path, dest_path) as dest:
 records = read_records(open(source_path))
 valid = filter_valid(records)
 transformed = transform(valid)
 
 for batch in batch(transformed, size=1000):
 dest.write_batch(batch)

Key Takeaways#

Always use @wraps — preserves function metadata for debugging and documentation
Decorator factories (decorators with arguments) add a third nesting level
Generators are lazy — use them for large datasets, infinite sequences, and pipelines
yield from delegates to sub-generators and forwards send/throw
Context managers guarantee cleanup — prefer them over try/finally
ExitStack handles dynamic numbers of context managers
Combine all three for clean, composable, resource-safe code

Rate Limiting & Throttling

Mon, 01 Jan 0001 00:00:00 +0000

Rate Limiting & Throttling#

Rate limiting protects APIs from abuse, ensures fair resource allocation, and maintains service stability under heavy load. Understanding rate limiting algorithms and their trade-offs is essential for both API providers and consumers.

Why Rate Limit?#

Protect infrastructure from overload and cascading failures
Ensure fairness among consumers sharing a service
Prevent abuse (scraping, brute-force attacks, DDoS)
Control costs for metered or pay-per-use services
Meet SLAs by preserving capacity for critical traffic

Rate Limiting Algorithms#

Token Bucket#

The token bucket algorithm allows bursts while enforcing an average rate. Tokens are added to a bucket at a fixed rate; each request consumes one token. If the bucket is empty, the request is rejected.

Retrieval-Augmented Generation (RAG)

Mon, 01 Jan 0001 00:00:00 +0000

Retrieval-Augmented Generation combines the generative power of large language models with external knowledge retrieval. Instead of relying solely on what the model memorized during training, RAG fetches relevant documents at inference time and includes them in the prompt context. This grounds the model’s responses in factual, up-to-date information.

Prerequisites#

Understanding of how LLMs generate text and their context windows
Basic familiarity with embeddings and vector spaces
Knowledge of prompt engineering fundamentals

Why RAG?#

The Problem with Pure LLMs#

Large language models have fundamental limitations that RAG addresses:

Security Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

The CIA Triad#

All security aims to protect three properties:

flowchart TD
 C["Confidentiality<br/>(only authorized access)"]
 I["Integrity<br/>(data not tampered with)"]
 A["Availability<br/>(accessible when needed)"]

Property	Threat	Protection
Confidentiality	Data breach, eavesdropping	Encryption, access control
Integrity	Tampering, corruption	Hashing, digital signatures
Availability	DDoS, hardware failure	Redundancy, backups

Encryption#

Symmetric Encryption#

Same key encrypts and decrypts — fast, used for bulk data:

Algorithm	Key Size	Use Case
AES-256	256 bits	Data at rest, TLS data transfer
ChaCha20	256 bits	Mobile, TLS alternative

Asymmetric Encryption#

Key pair (public + private) — slower, used for key exchange and signatures:

TDD & BDD

Mon, 01 Jan 0001 00:00:00 +0000

TDD & BDD#

Test-Driven Development (TDD) and Behaviour-Driven Development (BDD) are development workflows — not testing techniques. They change when and how you write tests, fundamentally altering your approach to design and implementation.

Test-Driven Development (TDD)#

The Red-Green-Refactor Cycle#

graph LR
 RED["🔴 RED<br/>Write a failing test"] --> GREEN["🟢 GREEN<br/>Write minimal code to pass"]
 GREEN --> REFACTOR["🔵 REFACTOR<br/>Improve design, keep green"]
 REFACTOR --> RED

Phase	What You Do	Rules
Red	Write a test that describes desired behaviour	Test must fail. If it passes, you haven’t written the right test.
Green	Write the simplest code that makes the test pass	No more code than necessary. “Fake it ’til you make it.”
Refactor	Improve code structure without changing behaviour	All tests stay green. Remove duplication.

TDD Example: Building a Password Validator#

Step 1 — Red: Write the first test

The Type System (PHP 8.x)

Mon, 01 Jan 0001 00:00:00 +0000

Evolution of PHP Types#

PHP started as a loosely-typed scripting language. Modern PHP has a sophisticated gradual type system:

PHP Version	Type Feature Added
5.0	Class/interface type hints (parameters only)
7.0	Scalar types (int, string, float, bool), return types, strict mode
7.1	Nullable (`?Type`), `void`, `iterable`
7.2	`object` type
7.4	Typed properties
8.0	Union types (`int\|string`), `mixed`, `static` return, `null`, `false` as standalone
8.1	Intersection types (`A&B`), `never`, enums, fibers
8.2	DNF types (Disjunctive Normal Form), `null`, `true`, `false` standalone
8.3	Typed class constants
8.4	Property hooks

Strict Types#

By default, PHP coerces values. Enable strict mode per-file:

Unsupervised Learning

Mon, 01 Jan 0001 00:00:00 +0000

Unsupervised Learning#

Unsupervised learning finds structure in data without labels. Clustering groups similar points together, dimensionality reduction compresses features while preserving structure, and anomaly detection identifies outliers. These techniques are essential for exploration, preprocessing, and problems where labels are expensive or unavailable.

Clustering#

K-Means#

K-Means partitions data into K clusters by iteratively assigning points to the nearest centroid and updating centroids.

Algorithm:

Initialise K centroids (k-means++ for smart initialisation)
Assign each point to the nearest centroid
Recompute centroids as the mean of assigned points
Repeat until convergence

from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
import matplotlib.pyplot as plt
import numpy as np

X, y_true = make_blobs(n_samples=300, centers=4, cluster_std=0.8, random_state=42)

kmeans = KMeans(n_clusters=4, init="k-means++", n_init=10, random_state=42)
labels = kmeans.fit_predict(X)

plt.scatter(X[:, 0], X[:, 1], c=labels, cmap="viridis", s=20, alpha=0.7)
plt.scatter(kmeans.cluster_centers_[:, 0], kmeans.cluster_centers_[:, 1],
 c="red", marker="X", s=200, edgecolors="black")
plt.title("K-Means Clustering")
plt.show()

Choosing K: The Elbow Method and Silhouette Score#

from sklearn.metrics import silhouette_score

inertias = []
silhouettes = []
K_range = range(2, 10)

for k in K_range:
 km = KMeans(n_clusters=k, n_init=10, random_state=42)
 km.fit(X)
 inertias.append(km.inertia_)
 silhouettes.append(silhouette_score(X, km.labels_))

fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 4))
ax1.plot(K_range, inertias, "bo-")
ax1.set_xlabel("K")
ax1.set_ylabel("Inertia")
ax1.set_title("Elbow Method")

ax2.plot(K_range, silhouettes, "ro-")
ax2.set_xlabel("K")
ax2.set_ylabel("Silhouette Score")
ax2.set_title("Silhouette Analysis")
plt.tight_layout()
plt.show()

Method	What It Measures	Choose K Where
Elbow	Within-cluster sum of squares (inertia)	Curve “bends” (diminishing returns)
Silhouette	Cohesion vs separation (-1 to 1)	Maximum score
Gap Statistic	Compares inertia to random reference	Largest gap

K-Means Limitations#

Assumes spherical clusters of similar size
Sensitive to initialisation (mitigated by n_init)
Requires K to be specified in advance
Cannot detect non-convex clusters

DBSCAN#

Density-Based Spatial Clustering of Applications with Noise. Finds clusters of arbitrary shape and identifies noise points.

Workflow Orchestration

Mon, 01 Jan 0001 00:00:00 +0000

Why Orchestration?#

Data pipelines are not single scripts — they are collections of interdependent tasks that must run in a specific order, on a schedule, with retries, monitoring, and alerting. An orchestrator manages this complexity.

Without orchestration, teams resort to cron jobs, manual triggers, and tribal knowledge. This breaks down quickly: no dependency tracking, no retry logic, no visibility into failures, and no way to backfill historical data.

Apache Airflow#

Airflow is the most widely adopted workflow orchestrator. Originally created at Airbnb, it defines pipelines as Python code (DAGs) and provides a web UI, scheduler, and extensible operator framework.

Agents and Tool Use

Mon, 01 Jan 0001 00:00:00 +0000

AI agents are systems where a language model autonomously decides what actions to take, executes those actions via external tools, observes the results, and iterates until a goal is achieved. Unlike simple prompt-response interactions, agents maintain state, plan multi-step strategies, and interact with the real world through APIs, databases, file systems, and other software.

Prerequisites#

Understanding of LLM inference and prompt engineering
Familiarity with function calling and structured output
Basic knowledge of API design and software architecture

Function and Tool Calling#

Tool calling is the mechanism by which an LLM requests execution of external functions. The model doesn’t execute code — it generates structured requests that the application fulfills.

Alerting

Mon, 01 Jan 0001 00:00:00 +0000

Alerting#

Alerting bridges the gap between dashboards and action. A well-designed alerting system notifies the right person, at the right time, with enough context to act — without drowning the team in noise. This section covers SLO-based alerting, error budgets, burn rate alerts, alert design, and on-call integration.

SLIs, SLOs, and SLAs#

Service Level Indicators (SLIs)#

An SLI is a quantitative measure of a specific aspect of service quality. It’s the raw measurement:

Architectural Patterns

Mon, 01 Jan 0001 00:00:00 +0000

Architectural Patterns#

Architectural patterns are proven structural templates for organizing software systems. Each pattern makes different tradeoffs between complexity, scalability, flexibility, and maintainability. Choosing the right one depends on your constraints and quality attributes.

Layered Architecture#

The most common pattern — organizes code into horizontal layers with strict dependency rules:

┌─────────────────────────────┐
│ Presentation Layer │ UI, API endpoints
├─────────────────────────────┤
│ Business Logic Layer │ Domain rules, workflows
├─────────────────────────────┤
│ Persistence Layer │ Data access, repositories
├─────────────────────────────┤
│ Database Layer │ Storage engines
└─────────────────────────────┘
 Dependencies flow DOWN only

Aspect	Details
Principle	Each layer only depends on the layer directly below
Variants	Strict (only adjacent layers) vs relaxed (skip layers)
Strengths	Simple, well-understood, good separation of concerns
Weaknesses	Monolithic deployments, changes ripple through layers
Best for	Traditional business applications, CRUD systems, small teams

Sinkhole Anti-Pattern#

When requests pass through layers without adding value — just forwarding calls. If > 20% of requests are sinkholes, the architecture is over-layered.

Authentication and Authorization

Mon, 01 Jan 0001 00:00:00 +0000

AuthN vs AuthZ#

Concept	Question	Example
Authentication (AuthN)	Who are you?	Login with username/password
Authorization (AuthZ)	What can you do?	Can this user delete records?

Authentication always comes first — you must know WHO before deciding WHAT they can access.

Authentication Methods#

Method	How It Works	Security	Use Case
Password	User provides secret string	Low (phishing, reuse)	Legacy, with MFA
MFA	Password + second factor	High	All production systems
API Key	Static token in header	Medium	Service-to-service (simple)
OAuth 2.0 + OIDC	Token-based delegation	High	Web apps, SSO
mTLS	Both sides present certificates	Very high	Service mesh, zero-trust
SSH Keys	Public/private key pair	High	Server access

Multi-Factor Authentication (MFA)#

Combines two or more factors:

Backtracking

Mon, 01 Jan 0001 00:00:00 +0000

Backtracking#

Backtracking systematically explores all possible solutions by making choices, checking constraints, and undoing choices that lead to dead ends. It’s controlled brute force with pruning.

Pattern Summary#

Problem Type	Time	Space	Pruning Possible?
Permutations	O(n!)	O(n)	Yes (used elements)
Combinations	O(C(n,k))	O(k)	Yes (remaining count)
Subsets	O(2ⁿ)	O(n)	Minimal
Constraint Satisfaction	Varies	O(n²)	Yes (constraint checking)
String Partitioning	O(n · 2ⁿ)	O(n)	Yes (validity check)

The Backtracking Template#

Every backtracking problem follows this structure:

C++ Core

Mon, 01 Jan 0001 00:00:00 +0000

C++ Core#

C++ extends C with classes, references, namespaces, and RAII (Resource Acquisition Is Initialization). This section covers the foundational C++ features that replace C’s manual patterns with safer, more expressive abstractions.

Classes#

class User {
public:
 // Constructor
 User(std::string name, int age)
 : name_(std::move(name)), age_(age) {}

 // Destructor (called automatically when object is destroyed)
 ~User() = default;

 // Getter
 const std::string& name() const { return name_; }
 int age() const { return age_; }

 // Method
 std::string greeting() const {
 return "Hello, " + name_ + "!";
 }

private:
 std::string name_;
 int age_;
};

User alice("Alice", 30);
std::cout << alice.greeting() << std::endl;

Constructor Initialiser Lists#

Always use initialiser lists — they initialise members directly instead of default-constructing then assigning:

Cloud Security

Mon, 01 Jan 0001 00:00:00 +0000

Why This Matters#

Cloud environments are the primary target for modern attacks. Misconfigurations — not sophisticated exploits — cause most cloud breaches. The shared responsibility model means YOU are responsible for securing your workloads, data, and configurations.

Shared Responsibility Model#

flowchart TD
 subgraph You["YOUR Responsibility"]
 Data["Data & Encryption"]
 IAM["IAM & Access Control"]
 App["Application Security"]
 OS["OS & Network Config"]
 FW["Firewall Rules"]
 end
 
 subgraph Provider["PROVIDER Responsibility"]
 Physical["Physical Security"]
 Network["Global Network"]
 Hypervisor["Hypervisor"]
 Hardware["Hardware"]
 Regions["Regions & AZs"]
 end

Responsibility by Service Type#

Layer	IaaS (EC2)	PaaS (Lambda, RDS)	SaaS (S3, SQS)
Data classification & encryption	You	You	You
IAM policies	You	You	You
Application code	You	You	N/A
OS patching	You	Provider	Provider
Network config	You	Shared	Provider
Physical infrastructure	Provider	Provider	Provider

Key insight: Moving to managed services (PaaS/SaaS) shifts more responsibility to the provider — but data and access control are ALWAYS yours.

Collaboration at Scale

Mon, 01 Jan 0001 00:00:00 +0000

Collaboration at Scale#

When teams grow beyond a handful of developers, Git workflows become critical infrastructure. The right workflow reduces merge conflicts, accelerates delivery, and preserves code quality. The wrong one creates bottlenecks and frustration.

Git Workflows Compared#

GitFlow#

main ─────────●─────────────────●──────────── (releases)
 ╲ ╱
develop ────●───●───●───●───●─●──────────── (integration)
 ╲ ╱ ╲ ╱
feature/x ────●───● ╲ ╱
 feature/y ──●

Aspect	Details
Branches	main, develop, feature/, release/, hotfix/*
Merge style	Merge commits (no fast-forward)
Release cycle	Planned releases with release branches
Best for	Packaged software, mobile apps, versioned products
Drawback	Complex, long-lived branches, integration pain

GitHub Flow#

main ─────●──────●──────●──────●──────── (always deployable)
 ╲ ╱ ╲ ╱ ╲ ╱
feature ────●──● ●──● ●──●
 PR+CI PR+CI PR+CI

Aspect	Details
Branches	main + short-lived feature branches
Merge style	Squash or merge commit via Pull Request
Release cycle	Deploy on every merge to main
Best for	SaaS, web applications, continuous deployment
Drawback	Requires robust CI/CD and feature flags

Trunk-Based Development#

main ──●──●──●──●──●──●──●──●──●──── (everyone commits here)
 │ │ │ │
 small small small
 commits commits commits

Aspect	Details
Branches	main only (or very short-lived branches < 1 day)
Merge style	Direct commits or same-day PRs
Release cycle	Continuous; feature flags gate incomplete work
Best for	High-performing teams, CI/CD maturity, Google/Meta-style
Drawback	Requires excellent test coverage and feature flags

Ship / Show / Ask#

A nuanced model that categorizes changes by risk:

Computational Thinking and Problem Decomposition

Mon, 01 Jan 0001 00:00:00 +0000

Computational thinking is the process of formulating problems so they can be solved systematically. It’s not about computers — it’s about structured problem-solving that happens to be executable by machines.

The Four Pillars#

flowchart TD
 CT["Computational Thinking"] --> D["Decomposition"]
 CT --> PR["Pattern Recognition"]
 CT --> AB["Abstraction"]
 CT --> AL["Algorithm Design"]
 
 D --> D1["Break into manageable sub-problems"]
 PR --> PR1["Find similarities across problems"]
 AB --> AB1["Remove irrelevant detail"]
 AL --> AL1["Define step-by-step solutions"]

Decomposition#

Breaking Problems Down#

Every complex system is made of simpler parts. Decomposition is identifying those parts and their relationships.

Concurrency

Mon, 01 Jan 0001 00:00:00 +0000

Concurrency#

Go was designed with concurrency as a first-class feature. Goroutines are lightweight threads managed by the Go runtime, and channels are typed conduits for communication between them. The motto: “Don’t communicate by sharing memory; share memory by communicating.”

Goroutines#

A goroutine is a lightweight concurrent function invocation. Starting one costs ~2 KB of stack (grows as needed) and is scheduled by the Go runtime, not the OS.

go doWork() // starts doWork as a goroutine

go func() {
 // anonymous goroutine
 fmt.Println("running concurrently")
}()

Goroutines vs Threads#

	Goroutine	OS Thread
Stack size	~2 KB (grows dynamically)	~1 MB (fixed)
Creation cost	~1 μs	~100 μs
Scheduling	Go runtime (M:N)	OS kernel
Practical count	Millions	Thousands
Communication	Channels (safe by design)	Shared memory + locks

Channels#

Channels are typed conduits for sending and receiving values between goroutines:

Concurrency Primitives

Mon, 01 Jan 0001 00:00:00 +0000

Concurrency Primitives#

Concurrency is the art of managing multiple threads of execution that share resources. Without proper synchronization, programs exhibit race conditions — subtle, non-deterministic bugs that are notoriously difficult to reproduce and fix.

Race Conditions and Critical Sections#

A race condition occurs when the outcome depends on the timing of thread execution:

counter = 0

def increment():
 global counter
 # NOT atomic — three operations:
 # 1. Read counter into register
 # 2. Add 1
 # 3. Write back to counter
 counter += 1

# Two threads running increment() 1000 times each
# Expected: 2000 | Actual: unpredictable (e.g., 1847)

A critical section is a code region accessing shared resources that must not execute concurrently.

Containers in Production

Mon, 01 Jan 0001 00:00:00 +0000

Beyond `docker run`#

Running a container locally is one thing. Running dozens — or thousands — of containers reliably in production requires orchestration, security hardening, observability, and disciplined image management. This section bridges the gap between local Docker usage and production-grade container infrastructure.

Container Orchestration Concepts#

Why Orchestrate?#

A single container on a single host has no resilience. If it crashes, nothing restarts it. If traffic spikes, nothing scales it. Orchestration solves these problems:

CSRF & Other Attacks

Mon, 01 Jan 0001 00:00:00 +0000

Cross-Site Request Forgery (CSRF)#

CSRF tricks a victim’s browser into making an unwanted request to a site where the victim is authenticated. The browser automatically attaches cookies (including session cookies), so the server cannot distinguish between a legitimate request and a forged one.

How CSRF Works#

1. Victim logs into bank.com (session cookie set)
2. Victim visits evil.com
3. evil.com contains:
 <form action="https://bank.com/transfer" method="POST">
 <input type="hidden" name="to" value="attacker">
 <input type="hidden" name="amount" value="10000">
 </form>
 <script>document.forms[0].submit()</script>
4. Browser sends the POST to bank.com WITH the session cookie
5. bank.com processes the transfer — it sees a valid session

CSRF Attack Variants#

Vector	Technique
Auto-submitting form	`<form>` + `<script>document.forms[0].submit()</script>`
Image tag (GET only)	`<img src="https://bank.com/transfer?to=attacker&amount=10000">`
XMLHttpRequest	Blocked by CORS for non-simple requests, but simple requests (form POST) pass
Fetch API	Same CORS restrictions apply

Prevention: SameSite Cookies#

The SameSite cookie attribute controls when cookies are sent with cross-site requests:

Distributed Transactions

Mon, 01 Jan 0001 00:00:00 +0000

Distributed Transactions#

A distributed transaction spans multiple nodes, partitions, or services. It must either commit on all participants or abort on all — partial completion is not acceptable. This is far harder than single-node transactions because any participant can crash, and the network can partition at any point during the protocol.

This section covers the classical protocols (2PC, 3PC), modern approaches (deterministic databases, Spanner’s TrueTime), and the coordination mechanisms that make them work.

File Operations and Permissions

Mon, 01 Jan 0001 00:00:00 +0000

File Operations#

Creating Files and Directories#

touch file.txt # create empty file (or update timestamp)
mkdir mydir # create directory
mkdir -p path/to/nested/dir # create parent dirs as needed

Copying, Moving, Deleting#

# Copy
cp source.txt dest.txt
cp -r source_dir/ dest_dir/ # recursive (directories)
cp -p file.txt backup/ # preserve permissions and timestamps

# Move / Rename
mv old_name.txt new_name.txt
mv file.txt /other/directory/

# Delete
rm file.txt
rm -r directory/ # recursive
rm -f file.txt # force (no error if missing)
rm -rf directory/ # recursive + force (DANGEROUS)

Safety with `rm`#

# NEVER do this with unquoted or unset variables
rm -rf "$DIR/" # if DIR is empty → rm -rf / !!!

# Safe pattern
[[ -n "$DIR" ]] || die "DIR is empty"
[[ "$DIR" != "/" ]] || die "Refusing to delete /"
rm -rf "${DIR:?ERROR: DIR not set}"

find — The Swiss Army Knife#

Basic Searches#

# By name
find . -name "*.py" # exact glob match
find . -iname "*.py" # case-insensitive
find . -name "*.py" -o -name "*.js" # OR

# By type
find . -type f # regular files only
find . -type d # directories only
find . -type l # symlinks only

# By size
find . -size +100M # larger than 100MB
find . -size -1k # smaller than 1KB

# By time
find . -mtime -7 # modified in last 7 days
find . -mtime +30 # modified more than 30 days ago
find . -newer reference.txt # newer than reference file

Actions#

# Execute command on each result
find . -name "*.log" -exec rm {} \;
find . -name "*.py" -exec grep -l "TODO" {} \;

# Batch execution (more efficient)
find . -name "*.log" -exec rm {} +

# Print with custom format
find . -type f -printf "%s %p\n" # size and path

# Delete directly (faster than -exec rm)
find . -name "*.tmp" -delete

Combining Conditions#

# AND (implicit)
find . -name "*.log" -size +10M

# OR
find . \( -name "*.log" -o -name "*.tmp" \) -mtime +7

# NOT
find . -type f ! -name "*.py"

# Practical: find large old logs
find /var/log -name "*.log" -size +50M -mtime +30 -exec ls -lh {} \;

Permissions#

Understanding Permission Notation#

-rwxr-xr-- 1 alice developers 4096 Jan 1 12:00 script.sh
│├─┤├─┤├─┤
│ │ │ └── Others: read only (r--)
│ │ └───── Group: read + execute (r-x)
│ └───────── Owner: read + write + execute (rwx)
└─────────── Type: - = file, d = directory, l = symlink

Octal (Numeric) Permissions#

Octal	Binary	Permission
0	000	— (none)
1	001	–x (execute)
2	010	-w- (write)
3	011	-wx (write + execute)
4	100	r– (read)
5	101	r-x (read + execute)
6	110	rw- (read + write)
7	111	rwx (all)

Common Permission Patterns#

Octal	Meaning	Use Case
`755`	rwxr-xr-x	Executable scripts, directories
`644`	rw-r–r–	Regular files
`600`	rw——-	Private files (keys, secrets)
`700`	rwx——	Private directories
`775`	rwxrwxr-x	Shared group directories
`664`	rw-rw-r–	Shared group files

chmod — Change Permissions#

# Numeric
chmod 755 script.sh
chmod 644 config.yml
chmod 600 ~/.ssh/id_rsa

# Symbolic
chmod +x script.sh # add execute for all
chmod u+w file.txt # add write for owner
chmod go-r file.txt # remove read for group and others
chmod a+r file.txt # add read for all
chmod u=rwx,go=rx script.sh # set explicitly

chown — Change Ownership#

chown alice file.txt # change owner
chown alice:developers file.txt # change owner and group
chown -R alice:developers dir/ # recursive
chown :developers file.txt # change group only

Directory Permissions#

Directories have different semantics for the same bits:

GraphQL

Mon, 01 Jan 0001 00:00:00 +0000

GraphQL#

GraphQL is a query language for APIs and a runtime for executing those queries. Unlike REST, it lets clients request exactly the data they need in a single request, eliminating over-fetching and under-fetching.

Core Concepts#

Concept	Description
Schema	Defines the types, queries, mutations, and subscriptions available
Query	Read operation — fetches data
Mutation	Write operation — creates, updates, or deletes data
Subscription	Real-time operation — pushes data to clients via WebSocket
Resolver	Function that fetches the data for a specific field
Type	Defines the shape of data (object, scalar, enum, interface, union)

Schema Definition Language (SDL)#

Scalar Types#

GraphQL has five built-in scalars: Int, Float, String, Boolean, ID.

HTTP/HTTPS

Mon, 01 Jan 0001 00:00:00 +0000

What is HTTP?#

HTTP (HyperText Transfer Protocol) is the application-layer protocol for the web. It’s a request-response protocol — a client sends a request, a server returns a response.

sequenceDiagram
 participant C as Client
 participant S as Server
 
 C->>S: GET /api/users HTTP/1.1
 S->>C: HTTP/1.1 200 OK + body

HTTP Request Structure#

GET /api/users?page=1 HTTP/1.1 ← Request line
Host: api.example.com ← Headers
Authorization: Bearer eyJhbG...
Accept: application/json
 ← Empty line (end of headers)
 ← Body (for POST/PUT)

HTTP Methods#

Method	Purpose	Idempotent	Safe	Has Body
GET	Retrieve resource	✅	✅	No
POST	Create resource	❌	❌	Yes
PUT	Replace resource entirely	✅	❌	Yes
PATCH	Partial update	❌	❌	Yes
DELETE	Remove resource	✅	❌	Optional
HEAD	Get headers only (no body)	✅	✅	No
OPTIONS	Get allowed methods (CORS preflight)	✅	✅	No

Idempotent: Multiple identical requests have the same effect as one. Safe: Doesn’t modify server state.

Indexing and Query Optimization

Mon, 01 Jan 0001 00:00:00 +0000

Why Indexes Matter#

Without an index, the database must scan every row in a table to find matches (full table scan). An index is a separate data structure that allows the database to find rows without reading the entire table — like a book’s index vs reading every page.

Without Index	With Index
O(n) — scan all rows	O(log n) — tree traversal
1M row table → 1M reads	1M row table → ~20 reads
Fine for small tables	Essential for large tables

How B-Tree Indexes Work#

The default index type in all major databases is the B-tree (balanced tree):

Infrastructure Automation

Mon, 01 Jan 0001 00:00:00 +0000

Infrastructure Automation#

Infrastructure automation brings the same rigour to infrastructure that CI/CD brings to application code: version control, peer review, automated testing, and repeatable deployments. GitOps, Terraform in pipelines, and policy-as-code are the pillars.

GitOps#

GitOps uses Git as the single source of truth for infrastructure and application state. A controller running in the cluster continuously reconciles the live state against what’s declared in Git.

Core Principles#

Principle	Meaning
Declarative	Desired state is described, not scripted
Versioned and immutable	Git history is the audit trail
Pulled automatically	Controller pulls from Git, not pushed by CI
Continuously reconciled	Drift is detected and corrected automatically

Push vs Pull Deployment#

Push (traditional CI/CD):
 CI Pipeline ──push──▶ kubectl apply / helm upgrade ──▶ Cluster

Pull (GitOps):
 Developer ──commit──▶ Git Repo ◀──poll/watch── Controller ──reconcile──▶ Cluster

ArgoCD#

The most popular GitOps controller for Kubernetes:

Java: Concurrency

Mon, 01 Jan 0001 00:00:00 +0000

Thread Fundamentals#

// Creating threads
// Option 1: Runnable (preferred — separates task from execution)
Runnable task = () -> System.out.println("Running in " + Thread.currentThread().getName());
Thread thread = new Thread(task, "worker-1");
thread.start(); // Starts new thread
thread.join(); // Wait for completion

// Option 2: Callable (returns a value, can throw checked exceptions)
Callable<Integer> computation = () -> {
 Thread.sleep(1000);
 return 42;
};

Thread States#

stateDiagram-v2
 [*] --> NEW: Thread created
 NEW --> RUNNABLE: start()
 RUNNABLE --> BLOCKED: waiting for lock
 RUNNABLE --> WAITING: wait()/join()
 RUNNABLE --> TIMED_WAITING: sleep()/wait(timeout)
 BLOCKED --> RUNNABLE: lock acquired
 WAITING --> RUNNABLE: notify()/join completes
 TIMED_WAITING --> RUNNABLE: timeout/notify
 RUNNABLE --> TERMINATED: run() completes
 TERMINATED --> [*]

ExecutorService — Thread Pool Management#

import java.util.concurrent.*;

// Fixed thread pool — bounded, reuses threads
ExecutorService fixedPool = Executors.newFixedThreadPool(4);

// Cached pool — unbounded, creates threads as needed
ExecutorService cachedPool = Executors.newCachedThreadPool();

// Single thread — sequential execution, guaranteed ordering
ExecutorService singleThread = Executors.newSingleThreadExecutor();

// Virtual threads (Java 21+) — lightweight, millions possible
ExecutorService virtualPool = Executors.newVirtualThreadPerTaskExecutor();

// Custom pool with full control
ThreadPoolExecutor custom = new ThreadPoolExecutor(
 4, // core pool size
 16, // max pool size
 60, TimeUnit.SECONDS, // idle thread timeout
 new LinkedBlockingQueue<>(1000), // work queue (bounded!)
 new ThreadPoolExecutor.CallerRunsPolicy() // rejection policy
);

Submitting Tasks#

ExecutorService executor = Executors.newFixedThreadPool(4);

// Submit Runnable (no return value)
executor.submit(() -> processItem(item));

// Submit Callable (returns Future)
Future<String> future = executor.submit(() -> fetchData(url));

// Get result (blocks until complete)
try {
 String result = future.get(30, TimeUnit.SECONDS);
} catch (TimeoutException e) {
 future.cancel(true); // Interrupt the task
} catch (ExecutionException e) {
 Throwable cause = e.getCause(); // Original exception
}

// Submit multiple tasks
List<Callable<String>> tasks = urls.stream()
 .map(url -> (Callable<String>) () -> fetchData(url))
 .toList();

List<Future<String>> futures = executor.invokeAll(tasks); // Wait for all
String fastest = executor.invokeAny(tasks); // Return first to complete

// Shutdown
executor.shutdown(); // No new tasks, finish existing
executor.awaitTermination(60, TimeUnit.SECONDS);

CompletableFuture — Async Composition#

import java.util.concurrent.CompletableFuture;

// Async computation
CompletableFuture<String> future = CompletableFuture.supplyAsync(() -> {
 return fetchUserData(userId);
});

// Chaining transformations
CompletableFuture<OrderSummary> pipeline = CompletableFuture
 .supplyAsync(() -> fetchUser(userId))
 .thenApply(user -> fetchOrders(user.id())) // Transform result
 .thenApply(orders -> summarize(orders)) // Transform again
 .exceptionally(ex -> { // Handle errors
 log.error("Pipeline failed", ex);
 return OrderSummary.empty();
 });

// Combining multiple futures
CompletableFuture<String> userFuture = CompletableFuture.supplyAsync(() -> fetchUser(id));
CompletableFuture<List<Order>> ordersFuture = CompletableFuture.supplyAsync(() -> fetchOrders(id));
CompletableFuture<Double> balanceFuture = CompletableFuture.supplyAsync(() -> fetchBalance(id));

// Wait for all
CompletableFuture<Void> allDone = CompletableFuture.allOf(userFuture, ordersFuture, balanceFuture);
allDone.thenRun(() -> {
 String user = userFuture.join();
 List<Order> orders = ordersFuture.join();
 Double balance = balanceFuture.join();
 buildDashboard(user, orders, balance);
});

// Combine two futures
CompletableFuture<String> combined = userFuture.thenCombine(
 ordersFuture,
 (user, orders) -> "%s has %d orders".formatted(user, orders.size())
);

// Timeout (Java 9+)
CompletableFuture<String> withTimeout = future
 .orTimeout(5, TimeUnit.SECONDS)
 .exceptionally(ex -> "Timed out");

Synchronization#

synchronized and Locks#

// synchronized — intrinsic lock
public class Counter {
 private int count = 0;
 
 public synchronized void increment() { // Lock on 'this'
 count++;
 }
 
 public synchronized int get() {
 return count;
 }
}

// ReentrantLock — more flexible
public class BoundedBuffer<T> {
 private final Queue<T> queue = new LinkedList<>();
 private final int capacity;
 private final ReentrantLock lock = new ReentrantLock();
 private final Condition notFull = lock.newCondition();
 private final Condition notEmpty = lock.newCondition();
 
 public BoundedBuffer(int capacity) {
 this.capacity = capacity;
 }
 
 public void put(T item) throws InterruptedException {
 lock.lock();
 try {
 while (queue.size() == capacity) {
 notFull.await(); // Wait until space available
 }
 queue.add(item);
 notEmpty.signal(); // Wake up consumers
 } finally {
 lock.unlock();
 }
 }
 
 public T take() throws InterruptedException {
 lock.lock();
 try {
 while (queue.isEmpty()) {
 notEmpty.await(); // Wait until item available
 }
 T item = queue.remove();
 notFull.signal(); // Wake up producers
 return item;
 } finally {
 lock.unlock();
 }
 }
}

Atomic Variables — Lock-Free#

import java.util.concurrent.atomic.*;

// AtomicInteger — thread-safe without locks
AtomicInteger counter = new AtomicInteger(0);
counter.incrementAndGet(); // Atomic increment
counter.compareAndSet(5, 10); // CAS: set to 10 only if currently 5

// AtomicReference — for object references
AtomicReference<Config> configRef = new AtomicReference<>(loadConfig());
configRef.updateAndGet(current -> current.withTimeout(30));

// LongAdder — better than AtomicLong for high-contention counters
LongAdder requestCount = new LongAdder();
requestCount.increment(); // Distributed across cells
requestCount.sum(); // Aggregate when needed

Virtual Threads (Java 21+)#

// Virtual threads: lightweight, managed by JVM, not OS
// Can create millions without exhausting memory

// Simple creation
Thread.startVirtualThread(() -> {
 var result = blockingHttpCall(url);
 process(result);
});

// With executor
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
 List<Future<String>> futures = urls.stream()
 .map(url -> executor.submit(() -> fetch(url)))
 .toList();
 
 List<String> results = futures.stream()
 .map(f -> {
 try { return f.get(); }
 catch (Exception e) { return "error"; }
 })
 .toList();
}

// Structured Concurrency (Preview, Java 21+)
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
 Subtask<String> user = scope.fork(() -> fetchUser(id));
 Subtask<List<Order>> orders = scope.fork(() -> fetchOrders(id));
 
 scope.join(); // Wait for all
 scope.throwIfFailed(); // Propagate first failure
 
 return new Dashboard(user.get(), orders.get());
}

When to Use Virtual vs Platform Threads#

Use Virtual Threads	Use Platform Threads
I/O-bound tasks (HTTP, DB, file)	CPU-bound computation
High concurrency (thousands of tasks)	Low concurrency with shared mutable state
Simple blocking code style	Need thread-local storage
Replacing reactive/async frameworks	Pinning concerns (synchronized blocks)

Common Concurrency Patterns#

Producer-Consumer#

BlockingQueue<Task> queue = new LinkedBlockingQueue<>(100);

// Producers
for (int i = 0; i < 3; i++) {
 executor.submit(() -> {
 while (!Thread.interrupted()) {
 Task task = generateTask();
 queue.put(task); // Blocks if queue is full
 }
 });
}

// Consumers
for (int i = 0; i < 5; i++) {
 executor.submit(() -> {
 while (!Thread.interrupted()) {
 Task task = queue.take(); // Blocks if queue is empty
 process(task);
 }
 });
}

Read-Write Lock#

public class CachedData<T> {
 private T data;
 private final ReadWriteLock rwLock = new ReentrantReadWriteLock();
 
 public T read() {
 rwLock.readLock().lock(); // Multiple readers allowed
 try {
 return data;
 } finally {
 rwLock.readLock().unlock();
 }
 }
 
 public void write(T newData) {
 rwLock.writeLock().lock(); // Exclusive access
 try {
 this.data = newData;
 } finally {
 rwLock.writeLock().unlock();
 }
 }
}

Hypothetical Use Case: Parallel API Aggregator#

public class DashboardService {
 private final ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();
 
 public Dashboard buildDashboard(String userId) {
 var userFuture = CompletableFuture.supplyAsync(() -> userService.getUser(userId), executor);
 var ordersFuture = CompletableFuture.supplyAsync(() -> orderService.getRecent(userId), executor);
 var metricsFuture = CompletableFuture.supplyAsync(() -> metricsService.getSummary(userId), executor);
 var notifFuture = CompletableFuture.supplyAsync(() -> notificationService.getUnread(userId), executor);
 
 CompletableFuture.allOf(userFuture, ordersFuture, metricsFuture, notifFuture)
 .orTimeout(5, TimeUnit.SECONDS)
 .join();
 
 return new Dashboard(
 userFuture.join(),
 ordersFuture.join(),
 metricsFuture.resultNow(),
 notifFuture.resultNow()
 );
 }
}

Key Takeaways#

Never use raw Thread — use ExecutorService for thread management
CompletableFuture for async composition — chain, combine, handle errors
Virtual threads (Java 21+) replace reactive frameworks for I/O-bound work
Prefer AtomicInteger/LongAdder over synchronized for simple counters
Bounded queues prevent memory exhaustion in producer-consumer patterns
Always shut down executors — use try-with-resources (Java 19+) or shutdown/awaitTermination
Avoid synchronized on public methods — use private lock objects
Structured concurrency (preview) makes concurrent code as readable as sequential

Jetpack Compose Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

Jetpack Compose Fundamentals#

Jetpack Compose is Android’s modern declarative UI toolkit. Instead of building views imperatively through XML layouts and findViewById, you describe what the UI should look like as a function of state, and the framework handles rendering and updates.

Declarative vs Imperative UI#

Aspect	Imperative (XML + View)	Declarative (Compose)
UI definition	XML layouts + code wiring	Kotlin functions
State updates	Find view, call setters	Recompose with new state
View hierarchy	Mutable, held in memory	Immutable descriptions, rebuilt on change
Reusability	Custom views, includes	Composable functions
Preview	Layout editor	`@Preview` annotation

In Compose, the UI is a function of state. When state changes, the framework re-executes (recomposes) only the affected composable functions.

Lifecycle Rules and Dependencies

Mon, 01 Jan 0001 00:00:00 +0000

Dependency Graph#

Terraform builds a directed acyclic graph (DAG) of all resources and their dependencies. This graph determines the order of operations.

flowchart TD
 VPC["aws_vpc.main"] --> Subnet["aws_subnet.public"]
 VPC --> SG["aws_security_group.web"]
 Subnet --> Instance["aws_instance.web"]
 SG --> Instance
 Instance --> EIP["aws_eip.web"]

Terraform creates resources top-down (VPC first, EIP last) and destroys bottom-up (EIP first, VPC last).

Implicit Dependencies#

Most dependencies are automatic — Terraform detects them from attribute references:

resource "aws_vpc" "main" {
 cidr_block = "10.0.0.0/16"
}

# Implicit dependency: references aws_vpc.main.id
resource "aws_subnet" "public" {
 vpc_id = aws_vpc.main.id
 cidr_block = "10.0.1.0/24"
}

# Implicit dependency: references aws_subnet.public.id
resource "aws_instance" "web" {
 subnet_id = aws_subnet.public.id
 ami = data.aws_ami.latest.id
}

Terraform automatically knows: VPC → Subnet → Instance.

Linear Algebra

Mon, 01 Jan 0001 00:00:00 +0000

Linear Algebra#

Linear algebra is the mathematics of vectors, matrices, and linear transformations. It is the computational engine behind machine learning, computer graphics, data compression, search engines, and recommendation systems.

Vectors#

A vector is an ordered list of numbers. It can represent a point in space, a direction, a data record, or a feature embedding.

v = [3, 4] 2D vector
w = [1, 0, -2, 5] 4D vector

Vector Operations#

Operation	Formula	Example
Addition	u + v = [u₁+v₁, u₂+v₂, …]	[1,2] + [3,4] = [4,6]
Scalar multiplication	c·v = [cv₁, cv₂, …]	3·[1,2] = [3,6]
Dot product	u·v = Σ uᵢvᵢ	[1,2]·[3,4] = 3+8 = 11
Magnitude (norm)	‖v‖ = √(Σ vᵢ²)	‖[3,4]‖ = √(9+16) = 5
Unit vector	v̂ = v / ‖v‖	[3,4]/5 = [0.6, 0.8]

The Dot Product#

The dot product captures the notion of alignment between two vectors:

Load Balancing

Mon, 01 Jan 0001 00:00:00 +0000

Why Load Balancers?#

Load balancers distribute incoming traffic across multiple targets, providing:

High availability — if one target fails, traffic routes to healthy ones
Scalability — add/remove targets without changing client configuration
SSL termination — handle encryption at the LB, not each server
Health checks — automatically detect and remove unhealthy targets

flowchart TD
 Users["Users"] --> LB["Load Balancer"]
 LB --> T1["Target 1 ✅"]
 LB --> T2["Target 2 ✅"]
 LB --> T3["Target 3 ❌ unhealthy"]
 
 LB -.->|"Health check fails"| T3

ELB Types#

Type	Layer	Protocol	Use Case
ALB (Application)	7 (HTTP)	HTTP, HTTPS, gRPC	Web apps, microservices, APIs
NLB (Network)	4 (TCP)	TCP, UDP, TLS	Ultra-low latency, gaming, IoT
GLB (Gateway)	3 (IP)	IP	Third-party appliances (firewalls)

Application Load Balancer (ALB)#

The most common choice for web applications:

Meetings

Mon, 01 Jan 0001 00:00:00 +0000

Meetings#

Meetings are the most expensive activity in engineering organisations. A one-hour meeting with six engineers costs six hours of engineering time plus the context-switch cost on either side. Use them wisely — they’re a powerful tool when needed and a destructive waste when not.

When NOT to Have a Meeting#

Before scheduling a meeting, ask: “Could this be an async message, document, or decision?”

Situation	Meeting needed?	Alternative
Sharing a status update	No	Written update in Slack/ticket
Announcing a decision already made	No	Email or wiki post
Gathering input from 2 people	Probably not	Async doc with comments
Brainstorming with 3+ people	Maybe	Try async first; meet if it stalls
Complex decision with multiple stakeholders	Yes	But only with preparation
Resolving an active disagreement	Yes	Async debates escalate and misread tone
Building relationships / onboarding	Yes	Some things need real-time connection
Incident response coordination	Yes	Speed matters; async is too slow

The “No Meeting” Test#

If you can’t fill in ALL of these, cancel the meeting:

Microservices Architecture

Mon, 01 Jan 0001 00:00:00 +0000

Microservices Architecture#

Microservices decompose a system into small, independently deployable services, each owning a specific business capability. This architectural style enables teams to move fast and scale independently — but introduces significant distributed systems complexity.

Monolith vs Microservices#

Aspect	Monolith	Microservices
Deployment	Single deployable unit	Each service deployed independently
Scaling	Scale entire application	Scale individual services
Technology	Single tech stack	Polyglot (per-service choice)
Data	Shared database	Database per service
Team structure	One team or many teams on one codebase	Small teams per service (2-pizza rule)
Complexity	In-process calls, simple	Network calls, distributed complexity
Consistency	Strong (ACID transactions)	Eventual consistency (sagas)
Testing	End-to-end straightforward	Integration testing complex
Latency	In-process (nanoseconds)	Network (milliseconds)
Debugging	Single process, stack traces	Distributed tracing required
Initial velocity	Faster to start	Slower to start (infrastructure overhead)
Long-term velocity	Slows as codebase grows	Maintains speed per service

The Monolith Is Not the Enemy#

A well-structured monolith (modular monolith) is a valid architecture. Many successful systems start as monoliths and only decompose when specific scaling or team boundaries demand it.

Model Evaluation

Mon, 01 Jan 0001 00:00:00 +0000

Model Evaluation#

A model is only as trustworthy as the method used to evaluate it. This section covers the metrics, validation strategies, and diagnostic tools that tell you whether your model genuinely works — or just memorised the training data.

Classification Metrics#

Confusion Matrix#

The confusion matrix is the foundation of all classification metrics:

 Predicted
 Pos Neg
Actual Pos [ TP | FN ]
 Neg [ FP | TN ]

Term	Meaning
TP (True Positive)	Correctly predicted positive
TN (True Negative)	Correctly predicted negative
FP (False Positive)	Incorrectly predicted positive (Type I error)
FN (False Negative)	Incorrectly predicted negative (Type II error)

from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay
from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = LogisticRegression(max_iter=5000, random_state=42)
model.fit(X_train, y_train)
y_pred = model.predict(X_test)

cm = confusion_matrix(y_test, y_pred)
ConfusionMatrixDisplay(cm, display_labels=["Malignant", "Benign"]).plot(cmap="Blues")

Core Metrics#

Metric	Formula	When to Use
Accuracy	(TP+TN) / (TP+TN+FP+FN)	Balanced classes only
Precision	TP / (TP+FP)	Cost of false positives is high (spam filter)
Recall (Sensitivity)	TP / (TP+FN)	Cost of false negatives is high (disease detection)
F1 Score	2 × (Precision × Recall) / (Precision + Recall)	Need balance between precision and recall
Specificity	TN / (TN+FP)	True negative rate matters

from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score

print(f"Accuracy: {accuracy_score(y_test, y_pred):.4f}")
print(f"Precision: {precision_score(y_test, y_pred):.4f}")
print(f"Recall: {recall_score(y_test, y_pred):.4f}")
print(f"F1: {f1_score(y_test, y_pred):.4f}")

Why Accuracy Fails on Imbalanced Data#

If 95% of emails are not spam, a model that always predicts “not spam” achieves 95% accuracy while catching zero spam. Accuracy is misleading when classes are imbalanced.

Namespaces, Autoloading, and Composer

Mon, 01 Jan 0001 00:00:00 +0000

Namespaces#

Namespaces prevent class name collisions and organize code into logical hierarchies:

<?php
// File: src/Payment/Stripe/Client.php
namespace App\Payment\Stripe;

class Client
{
 public function charge(float $amount): Receipt { /* ... */ }
}

<?php
// File: src/Http/Client.php
namespace App\Http;

class Client
{
 public function get(string $url): Response { /* ... */ }
}

Both classes are named Client, but they don’t conflict because they’re in different namespaces.

Using Namespaced Classes#

<?php
namespace App\Service;

// Import with 'use'
use App\Payment\Stripe\Client as StripeClient;
use App\Http\Client as HttpClient;
use App\Entity\{User, Order, Product}; // Group imports

class PaymentService
{
 public function __construct(
 private StripeClient $stripe,
 private HttpClient $http,
 ) {}
}

Convention: PSR-4#

The standard maps namespace to directory structure:

npm Ecosystem and Package Management

Mon, 01 Jan 0001 00:00:00 +0000

The JavaScript Package Ecosystem#

npm (Node Package Manager) is the world’s largest software registry with over 2 million packages. Understanding how it works — dependency resolution, versioning, security — is essential for any JavaScript project.

package.json#

The manifest file that defines your project:

{
 "name": "@myorg/api-server",
 "version": "2.1.0",
 "description": "REST API server",
 "type": "module",
 "main": "./dist/index.js",
 "types": "./dist/index.d.ts",
 "exports": {
 ".": {
 "import": "./dist/index.js",
 "require": "./dist/index.cjs",
 "types": "./dist/index.d.ts"
 },
 "./utils": {
 "import": "./dist/utils.js",
 "types": "./dist/utils.d.ts"
 }
 },
 "scripts": {
 "dev": "node --watch src/index.js",
 "build": "tsc",
 "test": "vitest",
 "test:coverage": "vitest --coverage",
 "lint": "eslint src/",
 "format": "prettier --write src/",
 "prepare": "husky"
 },
 "dependencies": {
 "express": "^4.18.2",
 "zod": "^3.22.0"
 },
 "devDependencies": {
 "typescript": "^5.3.0",
 "vitest": "^1.0.0",
 "@types/express": "^4.17.21"
 },
 "peerDependencies": {
 "react": "^18.0.0"
 },
 "engines": {
 "node": ">=20.0.0"
 },
 "files": ["dist/"],
 "license": "MIT"
}

Dependency Types#

Field	Installed in	Purpose
`dependencies`	Production + dev	Runtime requirements
`devDependencies`	Dev only	Build tools, test frameworks, linters
`peerDependencies`	NOT auto-installed	“I work with this, but you provide it”
`optionalDependencies`	If available	Non-critical enhancements

Semantic Versioning (SemVer)#

MAJOR.MINOR.PATCH
 │ │ └── Bug fixes (backward-compatible)
 │ └──────── New features (backward-compatible)
 └─────────────── Breaking changes

Version Ranges#

Syntax	Meaning	Example: `^1.2.3`
`1.2.3`	Exact	Only 1.2.3
`^1.2.3`	Compatible	≥1.2.3, <2.0.0
`~1.2.3`	Patch-level	≥1.2.3, <1.3.0
`>=1.0.0`	Minimum	1.0.0 or higher
`1.x`	Any minor/patch	≥1.0.0, <2.0.0
`*`	Any	Latest

flowchart LR
 subgraph "^1.2.3 (caret)"
 A["1.2.3"] --> B["1.2.4 ✓"]
 B --> C["1.3.0 ✓"]
 C --> D["1.9.9 ✓"]
 D --> E["2.0.0 ✗"]
 end

Best practice: Use ^ (caret) for most dependencies. Use exact versions for critical production dependencies where even patch updates could cause issues.

Performance & Loading

Mon, 01 Jan 0001 00:00:00 +0000

Why Performance Matters for PWAs#

A PWA that loads slowly on first visit will never become an installed app. Performance is not a separate concern — it is a prerequisite for every other PWA capability. Users on mobile networks with 3G-equivalent speeds will abandon a page that takes more than 3 seconds to become interactive. Google uses Core Web Vitals as a ranking signal, directly linking performance to discoverability.

Core Web Vitals#

Google’s Core Web Vitals are three metrics that measure real-user experience:

Property Testing & Fuzzing

Mon, 01 Jan 0001 00:00:00 +0000

Property Testing & Fuzzing#

Traditional unit tests verify specific examples. Property-based testing verifies that invariants hold across all possible inputs, automatically generating hundreds or thousands of test cases. Fuzzing takes this further by feeding random or mutated data to find crashes and security vulnerabilities.

Property-Based Testing#

The Core Idea#

Instead of writing:

def test_sort_specific():
 assert sort([3, 1, 2]) == [1, 2, 3]

You write:

def test_sort_property(random_list):
 result = sort(random_list)
 assert len(result) == len(random_list) # preserves length
 assert all(result[i] <= result[i+1] for i in range(len(result)-1)) # ordered
 assert set(result) == set(random_list) # same elements

The framework generates many random lists and verifies the properties hold for all of them.

Python: Type Hints and Static Analysis

Mon, 01 Jan 0001 00:00:00 +0000

Why Type Hints#

# Without types — what does this accept? What does it return?
def process(data, config):
 ...

# With types — self-documenting, IDE-supported, statically checkable
def process(data: list[dict[str, Any]], config: ProcessConfig) -> ProcessResult:
 ...

Benefits:

IDE autocompletion and error detection
Static analysis catches bugs before runtime
Self-documenting code
Refactoring confidence

Important: Type hints are NOT enforced at runtime by default. They’re for tools (mypy, pyright, IDEs).

Basic Type Annotations#

from typing import Any

# Variables
name: str = "Alice"
age: int = 30
active: bool = True
score: float = 95.5

# Functions
def greet(name: str, excited: bool = False) -> str:
 if excited:
 return f"Hello, {name}!!!"
 return f"Hello, {name}"

# None return
def log(message: str) -> None:
 print(message)

# Collections (Python 3.9+ — use built-in types directly)
numbers: list[int] = [1, 2, 3]
mapping: dict[str, int] = {"a": 1, "b": 2}
coordinates: tuple[float, float] = (3.14, 2.71)
unique: set[str] = {"apple", "banana"}

# Variable-length tuple
values: tuple[int, ...] = (1, 2, 3, 4, 5)

Union Types and Optional#

# Union — accepts multiple types
def parse_id(value: int | str) -> int: # Python 3.10+ syntax
 if isinstance(value, str):
 return int(value)
 return value

# Optional — shorthand for X | None
def find_user(user_id: int) -> dict | None:
 """Returns user dict or None if not found."""
 ...

# Pre-3.10 syntax (still valid)
from typing import Union, Optional
def parse_id(value: Union[int, str]) -> int: ...
def find_user(user_id: int) -> Optional[dict]: ...

Generics#

from typing import TypeVar, Generic

T = TypeVar("T")
K = TypeVar("K")
V = TypeVar("V")

# Generic function
def first(items: list[T]) -> T | None:
 return items[0] if items else None

# The return type matches the input type
first([1, 2, 3]) # inferred as int | None
first(["a", "b"]) # inferred as str | None

# Bounded TypeVar — restricts to specific types
from typing import TypeVar
Numeric = TypeVar("Numeric", int, float)

def add(a: Numeric, b: Numeric) -> Numeric:
 return a + b

# Generic class
class Stack(Generic[T]):
 def __init__(self) -> None:
 self._items: list[T] = []
 
 def push(self, item: T) -> None:
 self._items.append(item)
 
 def pop(self) -> T:
 if not self._items:
 raise IndexError("Stack is empty")
 return self._items.pop()
 
 def peek(self) -> T:
 if not self._items:
 raise IndexError("Stack is empty")
 return self._items[-1]
 
 def __len__(self) -> int:
 return len(self._items)

# Usage — type checker knows the element type
int_stack: Stack[int] = Stack()
int_stack.push(42)
value: int = int_stack.pop() # Type checker knows this is int

Protocols — Structural Typing (Duck Typing with Types)#

from typing import Protocol, runtime_checkable

# Protocol defines what methods/attributes an object must have
# WITHOUT requiring inheritance

@runtime_checkable
class Renderable(Protocol):
 def render(self) -> str: ...

@runtime_checkable
class Sized(Protocol):
 def __len__(self) -> int: ...

class HTMLWidget:
 """No inheritance from Renderable — just implements the method."""
 def render(self) -> str:
 return "<div>Widget</div>"

class JSONResponse:
 def render(self) -> str:
 return '{"status": "ok"}'

# Both satisfy Renderable without inheriting from it
def display(item: Renderable) -> None:
 print(item.render())

display(HTMLWidget()) # OK
display(JSONResponse()) # OK

# Runtime check (because @runtime_checkable)
isinstance(HTMLWidget(), Renderable) # True

# Complex protocol
class Repository(Protocol[T]):
 def get(self, id: str) -> T | None: ...
 def save(self, entity: T) -> None: ...
 def delete(self, id: str) -> bool: ...
 def list(self, limit: int = 100) -> list[T]: ...

TypedDict — Typed Dictionaries#

from typing import TypedDict, Required, NotRequired

class UserProfile(TypedDict):
 name: str
 email: str
 age: int
 bio: NotRequired[str] # Optional key

class APIResponse(TypedDict):
 status: int
 data: dict[str, Any]
 errors: NotRequired[list[str]]

def create_user(profile: UserProfile) -> APIResponse:
 # Type checker knows exactly which keys exist and their types
 print(profile["name"]) # OK — str
 # print(profile["phone"]) # Error — key doesn't exist
 
 return {"status": 201, "data": {"id": "123", **profile}}

# TypedDict with inheritance
class BaseEvent(TypedDict):
 event_type: str
 timestamp: str

class ClickEvent(BaseEvent):
 x: int
 y: int
 element_id: str

Callable Types#

from typing import Callable, ParamSpec, Concatenate
from collections.abc import Callable as CallableABC

# Function that accepts a callback
def apply_operation(
 values: list[int],
 operation: Callable[[int], int] # Takes int, returns int
) -> list[int]:
 return [operation(v) for v in values]

apply_operation([1, 2, 3], lambda x: x * 2) # [2, 4, 6]

# ParamSpec — preserve parameter types through decorators
P = ParamSpec("P")
R = TypeVar("R")

def logged(func: Callable[P, R]) -> Callable[P, R]:
 """Decorator that preserves the exact signature of the wrapped function."""
 @wraps(func)
 def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
 print(f"Calling {func.__name__}")
 return func(*args, **kwargs)
 return wrapper

@logged
def add(a: int, b: int) -> int:
 return a + b

# Type checker knows add still takes (int, int) -> int

Literal, Final, and TypeGuard#

from typing import Literal, Final, TypeGuard

# Literal — restrict to specific values
def set_direction(direction: Literal["north", "south", "east", "west"]) -> None:
 ...

set_direction("north") # OK
set_direction("up") # Type error!

# Final — cannot be reassigned or overridden
MAX_RETRIES: Final = 3
MAX_RETRIES = 5 # Type error!

class Base:
 @final
 def critical_method(self) -> None:
 """Subclasses cannot override this."""
 ...

# TypeGuard — narrow types in conditional checks
def is_string_list(val: list[Any]) -> TypeGuard[list[str]]:
 """Returns True if all elements are strings."""
 return all(isinstance(x, str) for x in val)

def process(data: list[Any]) -> None:
 if is_string_list(data):
 # Type checker now knows data is list[str]
 for s in data:
 print(s.upper()) # OK — s is str

Type Aliases and NewType#

from typing import NewType, TypeAlias

# Type alias — just a shorthand (no runtime distinction)
JSON: TypeAlias = dict[str, "JSON"] | list["JSON"] | str | int | float | bool | None
Headers: TypeAlias = dict[str, str]

def fetch(url: str, headers: Headers) -> JSON:
 ...

# NewType — creates a distinct type (catches mixing up same-typed values)
UserId = NewType("UserId", int)
OrderId = NewType("OrderId", int)

def get_user(user_id: UserId) -> dict:
 ...

def get_order(order_id: OrderId) -> dict:
 ...

uid = UserId(42)
oid = OrderId(42)

get_user(uid) # OK
get_user(oid) # Type error! OrderId is not UserId
get_user(42) # Type error! int is not UserId

Overload — Multiple Signatures#

from typing import overload

@overload
def fetch(url: str, raw: Literal[True]) -> bytes: ...
@overload
def fetch(url: str, raw: Literal[False] = ...) -> str: ...

def fetch(url: str, raw: bool = False) -> bytes | str:
 response = requests.get(url)
 if raw:
 return response.content
 return response.text

# Type checker knows:
result1 = fetch("http://example.com", raw=True) # bytes
result2 = fetch("http://example.com", raw=False) # str
result3 = fetch("http://example.com") # str

Running Static Analysis#

mypy Configuration#

# pyproject.toml
[tool.mypy]
python_version = "3.12"
strict = true
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_any_generics = true
check_untyped_defs = true

# Per-module overrides
[[tool.mypy.overrides]]
module = "tests.*"
disallow_untyped_defs = false

[[tool.mypy.overrides]]
module = "third_party_lib.*"
ignore_missing_imports = true

# Run mypy
mypy src/
mypy src/ --strict
mypy src/module.py --show-error-codes

Common mypy Errors and Fixes#

# Error: Incompatible return value type (got "Optional[str]", expected "str")
def get_name(user: dict) -> str:
 return user.get("name") # .get() returns Optional[str]

# Fix: handle the None case
def get_name(user: dict) -> str:
 name = user.get("name")
 if name is None:
 raise ValueError("User has no name")
 return name

# Error: Item "None" of "Optional[User]" has no attribute "email"
user = find_user(123)
print(user.email) # user might be None!

# Fix: narrow the type
user = find_user(123)
if user is not None:
 print(user.email) # OK — type narrowed to User

# Or use assert (for cases where None is a bug)
user = find_user(123)
assert user is not None, f"User 123 must exist"
print(user.email)

Hypothetical Use Case: Typed Event System#

from typing import TypeVar, Generic, Callable, Protocol
from dataclasses import dataclass

E = TypeVar("E", bound="Event")

@dataclass(frozen=True)
class Event:
 """Base event class."""
 pass

@dataclass(frozen=True)
class UserCreated(Event):
 user_id: str
 email: str

@dataclass(frozen=True)
class OrderPlaced(Event):
 order_id: str
 user_id: str
 total: float

class EventHandler(Protocol[E]):
 def __call__(self, event: E) -> None: ...

class EventBus:
 def __init__(self) -> None:
 self._handlers: dict[type[Event], list[Callable]] = {}
 
 def subscribe(self, event_type: type[E], handler: EventHandler[E]) -> None:
 self._handlers.setdefault(event_type, []).append(handler)
 
 def publish(self, event: Event) -> None:
 for handler in self._handlers.get(type(event), []):
 handler(event)

# Usage — fully type-safe
bus = EventBus()

def on_user_created(event: UserCreated) -> None:
 send_welcome_email(event.email)

def on_order_placed(event: OrderPlaced) -> None:
 process_payment(event.order_id, event.total)

bus.subscribe(UserCreated, on_user_created)
bus.subscribe(OrderPlaced, on_order_placed)

bus.publish(UserCreated(user_id="u1", email="alice@example.com"))

Key Takeaways#

Type hints don’t run — they’re for tools (mypy, pyright, IDEs)
Use X | None instead of Optional[X] (Python 3.10+)
Protocols enable duck typing with type safety — no inheritance required
Generics make reusable, type-safe containers and functions
NewType prevents mixing up same-typed values (UserId vs OrderId)
TypedDict types dictionary shapes — essential for JSON/API data
ParamSpec preserves function signatures through decorators
Start with mypy --strict on new projects — retrofitting is harder

Query Processing

Mon, 01 Jan 0001 00:00:00 +0000

From SQL Text to Result Set#

A SQL query goes through several transformation stages before producing results. Understanding this pipeline explains why some queries are fast and others are slow — regardless of how the SQL is written.

┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ Parser │ → │ Analyser │ → │ Rewriter │ → │ Planner/ │ → │ Executor │
│ │ │ │ │ │ │Optimiser │ │ │
│ SQL text │ │ Semantic │ │ View │ │ Cost- │ │ Runs the │
│ → parse │ │ checks, │ │ expansion│ │ based │ │ chosen │
│ tree │ │ resolve │ │ rule │ │ plan │ │ plan │
│ │ │ names │ │ rewrites │ │ selection│ │ │
└──────────┘ └──────────┘ └──────────┘ └──────────┘ └──────────┘

Parsing#

The parser converts SQL text into an abstract syntax tree (AST), also called a parse tree.

Security

Mon, 01 Jan 0001 00:00:00 +0000

Security#

Controlling access to the cluster and its resources, securing workloads, managing secrets safely, and hardening the supply chain.

Security Layers#

graph TB
 subgraph Cluster Access
 AUTH[Authentication]
 AUTHZ[Authorization - RBAC]
 ADM[Admission Controllers]
 end
 subgraph Workload Security
 PSS[Pod Security Standards]
 NP[Network Policies]
 SEC[Secrets Management]
 end
 subgraph Supply Chain
 IMG[Image Security]
 SCAN[Vulnerability Scanning]
 SIGN[Image Signing]
 end
 AUTH --> AUTHZ --> ADM --> PSS
 PSS --> NP
 NP --> SEC
 IMG --> SCAN --> SIGN

RBAC (Role-Based Access Control)#

RBAC controls who can do what on which resources. It’s the primary authorization mechanism in Kubernetes.

Serverless Architecture

Mon, 01 Jan 0001 00:00:00 +0000

Serverless Architecture#

Serverless doesn’t mean “no servers.” It means you don’t manage, provision, or think about servers. The provider handles scaling, patching, and availability. You write code, define triggers, and pay per execution. This section covers when serverless is the right choice, how to design serverless systems, and the anti-patterns that make serverless expensive or fragile.

Event-Driven Serverless#

Serverless functions are inherently event-driven: something happens, a function runs, and it stops. This model maps naturally to many workloads.

Streaming & Real-Time Data

Mon, 01 Jan 0001 00:00:00 +0000

Why Streaming?#

Batch processing assumes data can wait. Streaming processes data as it arrives — enabling fraud detection within milliseconds, real-time dashboards, live recommendations, and event-driven architectures. Any use case where latency of minutes or hours is unacceptable requires streaming.

Apache Kafka#

Kafka is the dominant distributed event streaming platform. It acts as a durable, high-throughput, fault-tolerant message broker that decouples producers from consumers.

Core Concepts#

Concept	Description
Topic	A named feed of messages (like a table for events)
Partition	A topic is split into partitions for parallelism and ordering
Offset	A sequential ID for each message within a partition
Producer	Publishes messages to topics
Consumer	Reads messages from topics
Consumer Group	A set of consumers that share the work of reading a topic
Broker	A Kafka server; a cluster has multiple brokers
Replication	Each partition is replicated across brokers for fault tolerance

Kafka Architecture#

Producer ──▶ Topic: orders ──▶ Consumer Group: analytics
 │ ├── Consumer 1 (partition 0, 1)
 │ └── Consumer 2 (partition 2, 3)
 │
 └──▶ Consumer Group: fraud-detection
 └── Consumer 3 (all 4 partitions)

Key insight: each consumer group independently tracks its position (offsets). Multiple groups can read the same topic at different speeds without interfering.

SwiftUI Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

SwiftUI Fundamentals#

SwiftUI is Apple’s declarative UI framework. Instead of imperatively constructing and mutating view hierarchies, you describe what the UI should look like for a given state, and the framework handles the rendering and updates.

The View Protocol#

Every SwiftUI view conforms to the View protocol:

protocol View {
 associatedtype Body: View
 @ViewBuilder var body: Self.Body { get }
}

A view is a lightweight struct that describes a piece of UI:

struct GreetingView: View {
 var body: some View {
 Text("Hello, World!")
 }
}

graph TD
 A[State changes] --> B[SwiftUI re-evaluates body]
 B --> C[Diff against previous render]
 C --> D[Apply minimal updates to screen]
 D --> E[User sees updated UI]
 E -->|User interaction| A

Key Insight: Views Are Values#

SwiftUI views are value types (structs). They are cheap to create, destroyed after each render, and recreated when state changes. The framework uses the view hierarchy as a blueprint to manage the actual rendered UI efficiently.

Traits & Generics

Mon, 01 Jan 0001 00:00:00 +0000

Traits & Generics#

Traits define shared behaviour. Generics enable type-safe code that works across multiple types. Together they form Rust’s mechanism for polymorphism — replacing inheritance and interfaces from object-oriented languages.

Traits#

A trait defines a set of methods a type can implement:

trait Summary {
 fn summarize(&self) -> String;

 // Default implementation (can be overridden)
 fn preview(&self) -> String {
 format!("{}...", &self.summarize()[..20])
 }
}

Implementing a Trait#

struct Article {
 title: String,
 author: String,
 content: String,
}

impl Summary for Article {
 fn summarize(&self) -> String {
 format!("{}, by {}", self.title, self.author)
 }
 // preview() uses the default implementation
}

let article = Article { /* ... */ };
println!("{}", article.summarize());

Orphan Rule#

You can implement a trait for a type only if either the trait or the type is defined in your crate. You cannot implement a foreign trait for a foreign type.

Web Performance

Mon, 01 Jan 0001 00:00:00 +0000

Why Performance Matters#

Every 100ms of latency costs 1% in sales (Amazon). A 1-second delay reduces conversions by 7%. Performance is not a feature — it’s a requirement.

flowchart LR
 subgraph Metrics["Core Web Vitals"]
 LCP["LCP<br>Largest Contentful Paint<br>Loading: < 2.5s"]
 INP["INP<br>Interaction to Next Paint<br>Interactivity: < 200ms"]
 CLS["CLS<br>Cumulative Layout Shift<br>Stability: < 0.1"]
 end

Core Web Vitals#

Metric	Measures	Good	Needs Work	Poor
LCP	Loading speed (largest element visible)	≤ 2.5s	≤ 4.0s	> 4.0s
INP	Responsiveness (input → visual update)	≤ 200ms	≤ 500ms	> 500ms
CLS	Visual stability (unexpected shifts)	≤ 0.1	≤ 0.25	> 0.25

LCP (Largest Contentful Paint)#

What counts as LCP element:

Architecture Decisions

Mon, 01 Jan 0001 00:00:00 +0000

Architecture Decisions#

Every software system is shaped by hundreds of decisions — technology choices, structural patterns, integration approaches. The best teams make these decisions explicitly, traceably, and reversibly. Architecture Decision Records (ADRs) are the primary tool for this discipline.

Architecture Decision Records (ADRs)#

An ADR documents one significant architectural decision along with its context, rationale, and consequences.

Standard Format#

# ADR-001: Use PostgreSQL for order storage

## Status
Accepted (2024-03-15)

## Context
The orders service needs persistent storage supporting:
- ACID transactions for financial data
- Complex queries for reporting
- High write throughput (10K orders/hour peak)

We considered PostgreSQL, DynamoDB, and MongoDB.

## Decision
We will use PostgreSQL (via AWS RDS).

## Consequences
### Positive
- Strong consistency guarantees
- Rich query capabilities (JOIN, aggregates)
- Team expertise exists

### Negative
- Vertical scaling limits (vs DynamoDB)
- Operational overhead (backups, failover)
- Schema migrations required for changes

### Risks
- May need read replicas at 100K orders/hour

ADR Lifecycle#

Status	Meaning
Proposed	Under discussion, not yet approved
Accepted	Approved and active
Deprecated	No longer recommended but existing code may use it
Superseded	Replaced by another ADR (link to successor)

Where to Store ADRs#

Location	Pros	Cons
Repository (`docs/adr/`)	Versioned with code, close to context	Scattered across repos
Wiki/Confluence	Searchable, cross-project visibility	Not versioned with code
Dedicated ADR repo	Central registry, easy to browse	Disconnected from implementation

Recommendation: Store in the repo for service-specific decisions; use a central wiki for cross-cutting decisions.

Binary Search — Advanced Patterns

Mon, 01 Jan 0001 00:00:00 +0000

Binary Search — Advanced Patterns#

Binary search goes far beyond sorted arrays. The key insight: whenever you can define a monotonic predicate over a search space, binary search applies. This unlocks solutions to capacity problems, optimisation tasks, and rotated-array puzzles.

Pattern Summary#

Pattern	Typical Problem	Search Space	Time
Classic sorted search	Find target in array	Array indices	O(log n)
Bisect on answer	Minimum capacity to ship	Answer range [lo, hi]	O(n · log(answer range))
Rotated array	Search in rotated sorted array	Array indices	O(log n)
Minimise maximum	Split array into k subarrays	[max_element, total_sum]	O(n · log S)
2D matrix search	Search row-sorted + col-sorted	Flattened index or staircase	O(log(m·n)) or O(m+n)
First/last occurrence	Lower bound / upper bound	Array indices	O(log n)

The Binary Search Template#

Most problems use one of two forms — find the leftmost position where a condition becomes true:

C++ Object-Oriented

Mon, 01 Jan 0001 00:00:00 +0000

C++ Object-Oriented Programming#

C++ supports classical OOP — inheritance, polymorphism, encapsulation, and abstract classes. Modern C++ uses OOP selectively, favouring composition and generic programming for most designs.

Inheritance#

class Shape {
public:
 Shape(double x, double y) : x_(x), y_(y) {}
 virtual ~Shape() = default; // virtual destructor — essential for polymorphism

 double x() const { return x_; }
 double y() const { return y_; }

 virtual double area() const = 0; // pure virtual — must be overridden
 virtual std::string name() const = 0;

 virtual void describe() const {
 std::cout << name() << " at (" << x_ << ", " << y_
 << ") area=" << area() << std::endl;
 }

private:
 double x_, y_;
};

class Circle : public Shape {
public:
 Circle(double x, double y, double radius)
 : Shape(x, y), radius_(radius) {}

 double area() const override { return M_PI * radius_ * radius_; }
 std::string name() const override { return "Circle"; }

private:
 double radius_;
};

class Rectangle : public Shape {
public:
 Rectangle(double x, double y, double w, double h)
 : Shape(x, y), width_(w), height_(h) {}

 double area() const override { return width_ * height_; }
 std::string name() const override { return "Rectangle"; }

private:
 double width_, height_;
};

Key Rules#

Rule	Why
Always make base class destructor `virtual`	Prevents undefined behaviour when deleting through base pointer
Use `override` keyword	Catches signature mismatches at compile time
Use `= 0` for pure virtual (abstract)	Forces derived classes to implement
Prefer `public` inheritance	Models “is-a” relationship

Polymorphism#

void print_shapes(const std::vector<std::unique_ptr<Shape>>& shapes) {
 for (const auto& shape : shapes) {
 shape->describe(); // virtual dispatch — correct method called
 }
}

std::vector<std::unique_ptr<Shape>> shapes;
shapes.push_back(std::make_unique<Circle>(0, 0, 5));
shapes.push_back(std::make_unique<Rectangle>(1, 1, 3, 4));
print_shapes(shapes);

Virtual Functions Internals#

Each class with virtual functions has a vtable — a table of function pointers. Each object carries a hidden vptr pointing to its class’s vtable. Virtual calls are indirect function calls through this table.

Compose State Management

Mon, 01 Jan 0001 00:00:00 +0000

Compose State Management#

State management is the central challenge in Compose. The framework re-executes composable functions whenever state changes (recomposition), so understanding how to hold, hoist, and observe state determines whether your UI is correct, performant, and testable.

remember and mutableStateOf#

mutableStateOf creates an observable state holder. When its value changes, Compose recomposes any composable that reads it. remember preserves the value across recompositions:

@Composable
fun Counter() {
 var count by remember { mutableStateOf(0) }

 Column(horizontalAlignment = Alignment.CenterHorizontally) {
 Text("Count: $count", style = MaterialTheme.typography.headlineMedium)
 Button(onClick = { count++ }) {
 Text("Increment")
 }
 }
}

remember vs rememberSaveable#

Function	Survives recomposition	Survives configuration change	Survives process death
`remember`	Yes	No	No
`rememberSaveable`	Yes	Yes	Yes (if using Saver)

// Survives screen rotation
var text by rememberSaveable { mutableStateOf("") }

// Custom saver for complex types
data class SearchState(val query: String, val page: Int)

val SearchStateSaver = Saver<SearchState, List<Any>>(
 save = { listOf(it.query, it.page) },
 restore = { SearchState(it[0] as String, it[1] as Int) }
)

var searchState by rememberSaveable(stateSaver = SearchStateSaver) {
 mutableStateOf(SearchState("", 1))
}

State Delegation Patterns#

// Delegation with 'by' — read/write directly
var name by remember { mutableStateOf("") }
name = "Alice" // Direct assignment
println(name) // Direct read

// Without delegation — explicit .value
val nameState = remember { mutableStateOf("") }
nameState.value = "Alice"
println(nameState.value)

// Destructuring
val (name, setName) = remember { mutableStateOf("") }
setName("Alice") // Setter function

State Hoisting#

State hoisting moves state up to a caller, making the composable stateless and reusable. The pattern is: state goes down as parameters, events go up as callbacks:

Containers from the OS Perspective

Mon, 01 Jan 0001 00:00:00 +0000

Containers from the OS Perspective#

Containers are not a single technology — they are an orchestration of several Linux kernel features that together provide process isolation. Understanding these primitives demystifies Docker and reveals what containers actually are: isolated groups of processes sharing the host kernel.

Linux Namespaces#

Namespaces partition kernel resources so that one set of processes sees one set of resources, and another set sees a different set.

Namespace	Isolates	Flag	Effect
PID	Process IDs	`CLONE_NEWPID`	Process sees itself as PID 1
NET	Network stack	`CLONE_NEWNET`	Own interfaces, IPs, routing tables
MNT	Mount points	`CLONE_NEWNS`	Own filesystem tree
UTS	Hostname/domain	`CLONE_NEWUTS`	Own hostname
IPC	IPC resources	`CLONE_NEWIPC`	Own message queues, semaphores
USER	UID/GID mappings	`CLONE_NEWUSER`	Root inside, unprivileged outside
CGROUP	Cgroup hierarchy	`CLONE_NEWCGROUP`	Own view of cgroup tree
TIME	System clocks	`CLONE_NEWTIME`	Own boot/monotonic time (Linux 5.6+)

Creating Namespaces#

# Create a new PID + MNT + NET namespace and run bash inside
sudo unshare --pid --mount --net --fork bash

# Inside: PID 1, no network interfaces, isolated mounts
ps aux # only see processes in this namespace
ip link # only loopback
hostname # still host's (need --uts to isolate)

PID Namespace in Action#

Host view: Container view:
PID 1 (systemd) PID 1 (nginx) ← mapped to PID 4521 on host
PID 4521 (nginx) PID 2 (worker)
PID 4522 (worker)

User Namespaces — Rootless Containers#

# Map container root (UID 0) to host UID 100000
unshare --user --map-root-user bash
id # uid=0(root) — but only inside the namespace

This enables rootless containers: full “root” power inside, zero privilege on the host.

CRDTs & Eventual Consistency

Mon, 01 Jan 0001 00:00:00 +0000

CRDTs & Eventual Consistency#

Conflict-free Replicated Data Types (CRDTs) are data structures that can be replicated across multiple nodes, modified independently and concurrently, and merged automatically — always converging to the same state without coordination. They provide strong eventual consistency (Section 3): any two replicas that have received the same set of updates are in the same state, immediately.

This section covers the theory behind CRDTs, the most important CRDT types, and the anti-entropy mechanisms that propagate updates between replicas.

Data Lakes & Lakehouses

Mon, 01 Jan 0001 00:00:00 +0000

Data Lake Architecture#

A data lake stores raw data in object storage (S3, GCS, ADLS) at low cost, in any format. The medallion architecture (bronze / silver / gold) organises data by quality level.

Medallion Architecture#

Sources ──▶ Bronze (Raw) ──▶ Silver (Cleaned) ──▶ Gold (Business-ready)

Layer	Purpose	Data Quality	Format	Consumers
Bronze	Raw ingestion, exact copy of source	As-is (may contain errors)	JSON, CSV, Avro, Parquet	Data engineers
Silver	Cleaned, deduplicated, conformed	Validated, typed, deduplicated	Parquet / Delta / Iceberg	Data engineers, analysts
Gold	Aggregated, business-ready models	Trusted, documented, SLA-backed	Parquet / Delta / Iceberg	Analysts, BI tools, ML

Example Directory Layout#

s3://data-lake/
├── bronze/
│ ├── orders/
│ │ ├── year=2025/month=06/day=15/
│ │ │ └── orders_20250615_143022.parquet
│ │ └── _schema/orders_v1.avsc
│ └── customers/
│ └── full_export_20250615.json
├── silver/
│ ├── orders/
│ │ └── year=2025/month=06/
│ │ └── part-00000.parquet
│ └── customers/
│ └── part-00000.parquet
└── gold/
 ├── fact_orders/
 │ └── year=2025/month=06/
 └── dim_customers/
 └── part-00000.parquet

The Problem with Plain Data Lakes#

Raw files on object storage lack features that databases take for granted:

DNS: Route 53

Mon, 01 Jan 0001 00:00:00 +0000

What is Route 53?#

Route 53 is AWS’s managed DNS service. It translates domain names (like api.example.com) into IP addresses, and provides health checking and traffic routing.

flowchart LR
 User["User types<br>api.example.com"] --> Resolver["DNS Resolver"]
 Resolver --> R53["Route 53"]
 R53 --> IP["Returns: 52.28.100.50"]
 User -->|"Connects to IP"| ALB["ALB at 52.28.100.50"]

The name “Route 53” comes from TCP/UDP port 53, the standard DNS port.

Core Concepts#

Hosted Zones#

A hosted zone is a container for DNS records for a domain:

Error Handling

Mon, 01 Jan 0001 00:00:00 +0000

Error Handling#

Go does not have exceptions. Errors are values — returned from functions, inspected by callers, and handled explicitly. This is a deliberate design choice that makes error paths visible and forces you to think about failure at every call site.

The `error` Interface#

type error interface {
 Error() string
}

Any type with an Error() string method is an error. The most common way to create errors:

import "errors"
import "fmt"

// Simple error
err := errors.New("something went wrong")

// Formatted error
err := fmt.Errorf("user %d not found", userID)

The Error Return Pattern#

Functions that can fail return an error as the last value:

Error Handling Patterns

Mon, 01 Jan 0001 00:00:00 +0000

Error Philosophy in JavaScript#

JavaScript has a unique error landscape: dynamic typing means errors that other languages catch at compile time only surface at runtime. TypeScript helps, but runtime errors from external data, network failures, and edge cases are inevitable.

The Error Object#

// Built-in error types
new Error("Something went wrong"); // generic
new TypeError("Expected string, got number"); // wrong type
new RangeError("Index out of bounds"); // value out of range
new ReferenceError("x is not defined"); // undeclared variable
new SyntaxError("Unexpected token"); // parse error
new URIError("Invalid URI"); // malformed URI

// Error properties
const err = new Error("Connection failed");
err.message; // "Connection failed"
err.name; // "Error"
err.stack; // Stack trace (non-standard but universal)
err.cause; // Chained error (ES2022)

Custom Error Classes#

class AppError extends Error {
 constructor(
 message: string,
 public readonly code: string,
 public readonly statusCode: number = 500,
 options?: ErrorOptions
 ) {
 super(message, options);
 this.name = "AppError";
 }
}

class NotFoundError extends AppError {
 constructor(resource: string, id: string) {
 super(`${resource} with id '${id}' not found`, "NOT_FOUND", 404);
 this.name = "NotFoundError";
 }
}

class ValidationError extends AppError {
 constructor(
 message: string,
 public readonly fields: Record<string, string>
 ) {
 super(message, "VALIDATION_ERROR", 400);
 this.name = "ValidationError";
 }
}

// Usage with error cause (ES2022)
async function fetchUser(id: string) {
 try {
 const response = await fetch(`/api/users/${id}`);
 if (!response.ok) throw new Error(`HTTP ${response.status}`);
 return await response.json();
 } catch (err) {
 throw new AppError("Failed to fetch user", "FETCH_ERROR", 502, { cause: err });
 }
}

Try/Catch Patterns#

Synchronous#

function parseJSON<T>(raw: string): T {
 try {
 return JSON.parse(raw);
 } catch (err) {
 throw new ValidationError("Invalid JSON input", {
 raw: "Must be valid JSON"
 });
 }
}

// Catch specific error types
function handleError(err: unknown) {
 if (err instanceof NotFoundError) {
 return { status: 404, body: { error: err.message } };
 }
 if (err instanceof ValidationError) {
 return { status: 400, body: { error: err.message, fields: err.fields } };
 }
 // Unknown error — don't expose internals
 console.error("Unexpected error:", err);
 return { status: 500, body: { error: "Internal server error" } };
}

Asynchronous#

// async/await — try/catch works naturally
async function processOrder(orderId: string) {
 try {
 const order = await fetchOrder(orderId);
 const payment = await chargePayment(order);
 await sendConfirmation(order, payment);
 return { success: true };
 } catch (err) {
 if (err instanceof PaymentError) {
 await refund(orderId);
 throw new AppError("Payment failed", "PAYMENT_FAILED", 402, { cause: err });
 }
 throw err; // re-throw unexpected errors
 } finally {
 await releaseInventoryLock(orderId); // always runs
 }
}

The Result Pattern#

Instead of throwing exceptions for expected failures, return a discriminated union that forces the caller to handle both paths:

Hyperparameter Tuning

Mon, 01 Jan 0001 00:00:00 +0000

Hyperparameter Tuning#

Hyperparameters are settings that control the learning process — they are not learned from data. Choosing the right hyperparameters can be the difference between a mediocre model and a production-ready one. This section covers systematic approaches to finding optimal configurations.

Parameters vs Hyperparameters#

Type	Set By	Examples
Parameters	Learned from data during training	Linear regression weights, tree split thresholds
Hyperparameters	Set before training begins	Learning rate, max_depth, C, n_estimators, kernel

Parameters are optimised by the algorithm (e.g., gradient descent minimises the loss). Hyperparameters are optimised by you — using the techniques in this section.

Inference and Deployment

Mon, 01 Jan 0001 00:00:00 +0000

Deploying large language models in production requires understanding the mechanics of text generation, memory constraints, optimization techniques, and the tradeoffs between latency, throughput, and cost. This chapter covers how LLMs generate text at the hardware level and how to serve them efficiently at scale.

Prerequisites#

Understanding of transformer architecture (attention, KV cache)
Basic knowledge of GPU computing and memory hierarchies
Familiarity with model parameters and floating-point representations

Inference Mechanics#

Autoregressive Generation#

LLMs generate text one token at a time. Each new token is conditioned on all previous tokens. This sequential dependency is the fundamental constraint of LLM inference.

Java: Streams and Functional Programming

Mon, 01 Jan 0001 00:00:00 +0000

Functional Interfaces#

// A functional interface has exactly ONE abstract method
@FunctionalInterface
interface Transformer<T, R> {
 R transform(T input);
}

// Built-in functional interfaces (java.util.function)
Function<String, Integer> // T → R
Predicate<String> // T → boolean
Consumer<String> // T → void
Supplier<String> // () → T
UnaryOperator<String> // T → T (same type)
BinaryOperator<Integer> // (T, T) → T
BiFunction<String, Integer, Boolean> // (T, U) → R

Lambda Expressions#

// Lambda syntax
Comparator<String> byLength = (a, b) -> Integer.compare(a.length(), b.length());
Predicate<Integer> isPositive = n -> n > 0;
Consumer<String> printer = System.out::println; // Method reference

// Method references (4 types)
Function<String, Integer> parse = Integer::parseInt; // Static method
Function<String, String> upper = String::toUpperCase; // Instance method (unbound)
Supplier<List<String>> factory = ArrayList::new; // Constructor
Consumer<String> print = System.out::println; // Instance method (bound)

// Effectively final — lambdas capture variables that don't change
String prefix = "Hello"; // Effectively final
Function<String, String> greeter = name -> prefix + " " + name;
// prefix = "Hi"; // Would make it non-effectively-final → compile error

Stream API#

Stream Pipeline#

flowchart LR
 S["Source<br>(Collection, array, generator)"] --> I1["filter()"]
 I1 --> I2["map()"]
 I2 --> I3["sorted()"]
 I3 --> T["Terminal<br>(collect, forEach, reduce)"]
 
 style S fill:#e1f5fe
 style I1 fill:#fff3e0
 style I2 fill:#fff3e0
 style I3 fill:#fff3e0
 style T fill:#c8e6c9

List<String> names = List.of("Alice", "Bob", "Charlie", "Diana", "Eve");

// Basic pipeline
List<String> result = names.stream()
 .filter(name -> name.length() > 3) // Intermediate: filter
 .map(String::toUpperCase) // Intermediate: transform
 .sorted() // Intermediate: sort
 .toList(); // Terminal: collect
// ["ALICE", "CHARLIE", "DIANA"]

// Streams are LAZY — nothing happens until terminal operation
Stream<String> lazy = names.stream()
 .filter(name -> {
 System.out.println("Filtering: " + name); // Not printed yet!
 return name.length() > 3;
 });
// No output until: lazy.toList() or lazy.forEach(...)

Common Operations#

List<Order> orders = getOrders();

// filter — keep elements matching predicate
List<Order> expensive = orders.stream()
 .filter(o -> o.total() > 100)
 .toList();

// map — transform each element
List<String> emails = orders.stream()
 .map(Order::customerEmail)
 .toList();

// flatMap — one-to-many transformation (flatten nested collections)
List<OrderItem> allItems = orders.stream()
 .flatMap(order -> order.items().stream())
 .toList();

// distinct, sorted, limit, skip
List<String> uniqueSorted = names.stream()
 .distinct()
 .sorted()
 .limit(5)
 .toList();

// peek — debug (don't use for side effects in production)
orders.stream()
 .peek(o -> log.debug("Processing order: {}", o.id()))
 .filter(Order::isPending)
 .forEach(this::processOrder);

// anyMatch, allMatch, noneMatch
boolean hasExpensive = orders.stream().anyMatch(o -> o.total() > 1000);
boolean allShipped = orders.stream().allMatch(o -> o.status() == SHIPPED);

// findFirst, findAny
Optional<Order> first = orders.stream()
 .filter(o -> o.customer().equals("Alice"))
 .findFirst();

// count
long pendingCount = orders.stream()
 .filter(Order::isPending)
 .count();

Collectors#

import java.util.stream.Collectors;

// toList, toSet, toMap
Map<String, Order> orderById = orders.stream()
 .collect(Collectors.toMap(Order::id, Function.identity()));

// groupingBy
Map<OrderStatus, List<Order>> byStatus = orders.stream()
 .collect(Collectors.groupingBy(Order::status));

// groupingBy with downstream collector
Map<String, Double> totalByCustomer = orders.stream()
 .collect(Collectors.groupingBy(
 Order::customerId,
 Collectors.summingDouble(Order::total)
 ));

Map<OrderStatus, Long> countByStatus = orders.stream()
 .collect(Collectors.groupingBy(
 Order::status,
 Collectors.counting()
 ));

// partitioningBy (split into true/false groups)
Map<Boolean, List<Order>> partitioned = orders.stream()
 .collect(Collectors.partitioningBy(o -> o.total() > 100));
List<Order> expensive = partitioned.get(true);
List<Order> cheap = partitioned.get(false);

// joining
String csv = names.stream()
 .collect(Collectors.joining(", ", "[", "]"));
// "[Alice, Bob, Charlie]"

// statistics
DoubleSummaryStatistics stats = orders.stream()
 .collect(Collectors.summarizingDouble(Order::total));
stats.getAverage(); // mean
stats.getMax(); // maximum
stats.getCount(); // count

Reduce#

// reduce — fold elements into a single value
int sum = List.of(1, 2, 3, 4, 5).stream()
 .reduce(0, Integer::sum); // 15

Optional<Integer> max = numbers.stream()
 .reduce(Integer::max);

// Complex reduce
String concatenated = words.stream()
 .reduce("", (a, b) -> a.isEmpty() ? b : a + ", " + b);

// Parallel-safe reduce with combiner
int parallelSum = numbers.parallelStream()
 .reduce(0, Integer::sum, Integer::sum);
// identity accumulator combiner

Primitive Streams#

// Avoid boxing overhead for numeric operations
IntStream.range(0, 100) // 0 to 99
 .filter(n -> n % 2 == 0)
 .sum(); // 2450

IntStream.rangeClosed(1, 10) // 1 to 10
 .average() // OptionalDouble
 .orElse(0.0);

// Convert between object and primitive streams
List<String> words = List.of("hello", "world", "java");
IntStream lengths = words.stream().mapToInt(String::length);
Stream<Integer> boxed = IntStream.range(0, 10).boxed();

// Generate streams
DoubleStream.generate(Math::random).limit(10);
IntStream.iterate(1, n -> n * 2).limit(10); // 1, 2, 4, 8, 16...

Parallel Streams#

// Parallel processing — uses ForkJoinPool
long count = hugeList.parallelStream()
 .filter(item -> expensiveCheck(item))
 .count();

// When to use parallel streams:
// ✓ Large datasets (>10,000 elements)
// ✓ CPU-bound operations (no I/O)
// ✓ Stateless, independent operations
// ✓ Source supports efficient splitting (ArrayList, arrays)

// When NOT to use:
// ✗ Small datasets (overhead > benefit)
// ✗ I/O-bound operations (use async instead)
// ✗ Order-dependent operations
// ✗ LinkedList (poor splitting)
// ✗ Shared mutable state

// Custom thread pool for parallel streams
ForkJoinPool customPool = new ForkJoinPool(8);
List<Result> results = customPool.submit(() ->
 items.parallelStream()
 .map(this::heavyComputation)
 .toList()
).get();

Practical Patterns#

Data Transformation Pipeline#

public record SalesReport(
 String region,
 double totalRevenue,
 long orderCount,
 double averageOrderValue
) {}

public List<SalesReport> generateReport(List<Order> orders) {
 return orders.stream()
 .filter(o -> o.status() == OrderStatus.COMPLETED)
 .collect(Collectors.groupingBy(Order::region))
 .entrySet().stream()
 .map(entry -> {
 String region = entry.getKey();
 List<Order> regionOrders = entry.getValue();
 double total = regionOrders.stream().mapToDouble(Order::total).sum();
 long count = regionOrders.size();
 return new SalesReport(region, total, count, total / count);
 })
 .sorted(Comparator.comparingDouble(SalesReport::totalRevenue).reversed())
 .toList();
}

Optional + Stream Composition#

public Optional<String> findBestOffer(String customerId) {
 return findCustomer(customerId)
 .map(Customer::loyaltyTier)
 .flatMap(tier -> getOffers(tier).stream()
 .filter(Offer::isActive)
 .max(Comparator.comparingDouble(Offer::discount))
 .map(Offer::code));
}

Key Takeaways#

Streams are lazy — intermediate operations do nothing until a terminal operation triggers execution
Streams are single-use — cannot be reused after a terminal operation
Prefer method references (String::toUpperCase) over lambdas when possible
flatMap for one-to-many transformations (nested collections, Optional chains)
Use primitive streams (IntStream, DoubleStream) to avoid boxing overhead
Parallel streams only for CPU-bound work on large datasets with good splittability
Collectors.groupingBy with downstream collectors replaces complex loops
Never mutate external state from within a stream — streams assume stateless operations

Lifetimes

Mon, 01 Jan 0001 00:00:00 +0000

Lifetimes#

Lifetimes are Rust’s way of ensuring that references are always valid. They are annotations that tell the compiler how long a reference must live. Most of the time, the compiler infers lifetimes automatically. You only need explicit annotations when the compiler cannot figure it out.

What Lifetimes Prevent#

// C — dangling pointer (compiles, crashes at runtime)
int* dangling() {
 int x = 42;
 return &x; // x is freed when function returns — pointer is invalid
}

// Rust — compiler rejects this
fn dangling() -> &i32 {
 let x = 42;
 &x // ERROR: `x` does not live long enough
}

The compiler tracks how long every reference is valid and rejects code where a reference outlives its referent.

Lock Management

Mon, 01 Jan 0001 00:00:00 +0000

Why Locks Still Matter with MVCC#

MVCC handles most read/write concurrency without locks, but locks are still essential for:

Write–write conflicts — two transactions updating the same row need coordination
DDL operations — schema changes must not conflict with concurrent queries
Explicit locking — SELECT ... FOR UPDATE, advisory locks
Internal structures — B+tree page latches, buffer pool frame pins

This section covers the lock manager: the subsystem that grants, queues, detects deadlocks, and escalates locks.

Multi-Cloud & Hybrid Patterns

Mon, 01 Jan 0001 00:00:00 +0000

Multi-Cloud & Hybrid Patterns#

Multi-cloud is one of the most over-sold and under-scrutinised strategies in cloud architecture. Vendors and analysts promote it as best practice; practitioners know it doubles operational complexity. This section separates the legitimate use cases from the cargo-cult adoption and covers the patterns that make multi-cloud and hybrid architectures work when they’re genuinely needed.

When Multi-Cloud Makes Sense#

Legitimate Reasons#

Reason	Scenario	Example
Best-of-breed services	A specific provider has a uniquely superior service	BigQuery for analytics + AWS for everything else
M&A inheritance	Acquisition brings workloads on a different cloud	Company A (AWS) acquires Company B (Azure)
Regulatory requirement	Regulator mandates no single-vendor dependency	Financial services in some jurisdictions
Data sovereignty	Provider lacks a region in a required country	GCP has no region in a required market; Azure does
Negotiation leverage	Credible multi-cloud capability improves pricing negotiations	Demonstrably portable workloads

Bad Reasons (But Commonly Cited)#

Claimed Reason	Reality
“Avoid vendor lock-in”	You trade one lock-in (cloud) for another (abstraction layer). The abstraction layer is your own code, which is harder to maintain than managed services.
“Redundancy across clouds”	Cross-cloud failover is extraordinarily complex. Within-cloud multi-region is far simpler and better tested.
“We want flexibility”	Flexibility without usage is waste. Design for portability only when you’ll actually port.
“Our team knows both”	Your team will become mediocre at two clouds instead of excellent at one.

The Cost of Multi-Cloud#

Multi-cloud is not free. Quantify these costs before committing:

Negotiation & Influence

Mon, 01 Jan 0001 00:00:00 +0000

Negotiation & Influence#

Engineers negotiate constantly — deadlines, scope, technical decisions, resource allocation, team priorities, and their own compensation. The engineers who can influence outcomes without relying on authority are the ones who shape what gets built and how.

Negotiating Deadlines and Scope#

Every project exists in the tension between scope, time, and quality. When someone asks for more than is possible, negotiation determines which variable gives.

The Iron Triangle#

flowchart TD
 A[Scope] --- B[Time]
 B --- C[Quality]
 C --- A
 D[Pick two. The third adjusts.] --> A
 D --> B
 D --> C

When asked for an unrealistic deadline, don’t say “that’s impossible.” Instead, make the trade-off explicit:

Network Security

Mon, 01 Jan 0001 00:00:00 +0000

Defense in Depth#

Network security uses multiple layers — if one fails, others still protect:

flowchart TD
 Internet["Internet"]
 Internet --> Edge["Edge: WAF, DDoS protection"]
 Edge --> FW["Perimeter: Firewall"]
 FW --> SG["Host: Security Groups / iptables"]
 SG --> App["Application: Auth, input validation"]
 App --> Data["Data: Encryption at rest"]

Firewalls#

Types#

Type	Layer	What It Inspects	Example
Packet filter	3-4	IP, port, protocol	iptables, ACLs
Stateful	3-4	Connection state + packets	AWS Security Groups
Application (WAF)	7	HTTP content, patterns	AWS WAF, Cloudflare
Next-gen (NGFW)	3-7	Deep packet inspection	Palo Alto, Fortinet

Stateful vs Stateless#

Aspect	Stateless (NACLs)	Stateful (Security Groups)
Tracks connections	No	Yes
Return traffic	Must explicitly allow	Automatically allowed
Rule evaluation	All rules, in order	All rules, most permissive wins
Use case	Subnet-level control	Instance-level control

Security Group Example (AWS)#

Inbound Rules:
 Allow TCP 443 from 0.0.0.0/0 (HTTPS from anywhere)
 Allow TCP 22 from 10.0.0.0/16 (SSH from VPC only)

Outbound Rules:
 Allow All traffic to 0.0.0.0/0 (default — allow all outbound)

Principle of least privilege: Only open ports that are needed, only from sources that need access.

Observability in CI/CD

Mon, 01 Jan 0001 00:00:00 +0000

Observability in CI/CD#

You can’t improve what you don’t measure. Observability applied to CI/CD pipelines means tracking DORA metrics, understanding where time is spent, catching regressions early, and verifying deployments automatically.

DORA Metrics#

The four key metrics from the DevOps Research and Assessment (DORA) team that predict software delivery performance:

Metric	Definition	Elite	High	Medium	Low
Deployment Frequency	How often code reaches production	On-demand (multiple/day)	Weekly to monthly	Monthly to 6-monthly	Fewer than once per 6 months
Lead Time for Changes	Commit to production	< 1 hour	1 day to 1 week	1 week to 1 month	> 6 months
Change Failure Rate	% of deployments causing failure	0-15%	16-30%	16-30%	> 30%
Mean Time to Recovery (MTTR)	Time to restore service after failure	< 1 hour	< 1 day	1 day to 1 week	> 6 months

Measuring Deployment Frequency#

# GitLab CI: emit deployment event to Datadog
deploy_prod:
 stage: deploy
 script:
 - ./deploy.sh
 - |
 curl -X POST "https://api.datadoghq.eu/api/v1/events" \
 -H "DD-API-KEY: ${DD_API_KEY}" \
 -d @- <<EOF
 {
 "title": "Production Deployment",
 "text": "Deployed ${CI_PROJECT_NAME} ${CI_COMMIT_TAG}",
 "tags": ["service:${CI_PROJECT_NAME}", "env:prod", "team:${TEAM}"],
 "alert_type": "success"
 }
 EOF

Measuring Lead Time#

Lead time = time from first commit on branch to production deployment:

Observability Platforms

Mon, 01 Jan 0001 00:00:00 +0000

Observability Platforms#

Choosing an observability platform is one of the most impactful infrastructure decisions a team makes. The platform determines how you collect, store, query, and alert on telemetry — and it significantly affects your monthly bill. This section compares the major platforms: Datadog, Grafana Cloud, New Relic, and AWS-native tools.

Platform Architecture Models#

There are two fundamental approaches:

All-in-One SaaS#

A single vendor handles metrics, logs, traces, and dashboards. You send telemetry to their endpoints and use their UI.

Operations

Mon, 01 Jan 0001 00:00:00 +0000

Operations#

Running Kubernetes in production requires proficiency in debugging, monitoring, logging, upgrades, and disaster recovery. This section covers day-to-day operational tasks and the tooling that supports them.

Essential kubectl Commands#

Task	Command
List all pods (all namespaces)	`kubectl get pods -A`
Describe a pod (events + config)	`kubectl describe pod <name> -n <ns>`
Get pod logs	`kubectl logs <pod> -n <ns>`
Follow logs in real time	`kubectl logs -f <pod> -n <ns>`
Logs from a previous crash	`kubectl logs <pod> --previous`
Exec into a container	`kubectl exec -it <pod> -- /bin/sh`
Port-forward for local access	`kubectl port-forward svc/<name> 8080:80`
Get events sorted by time	`kubectl get events --sort-by=.metadata.creationTimestamp`
Get resource YAML	`kubectl get pod <name> -o yaml`
Apply a manifest	`kubectl apply -f manifest.yaml`
Dry-run a change	`kubectl apply -f manifest.yaml --dry-run=server`
Scale a deployment	`kubectl scale deployment <name> --replicas=5`
Rollout status	`kubectl rollout status deployment/<name>`
Rollback a deployment	`kubectl rollout undo deployment/<name>`
Top nodes (resource usage)	`kubectl top nodes`
Top pods	`kubectl top pods -A --sort-by=memory`

Debugging Workflow#

When a pod is unhealthy, follow this escalation path:

Performance Testing

Mon, 01 Jan 0001 00:00:00 +0000

Performance Testing#

Performance testing validates that a system meets its speed, scalability, and stability requirements under expected and extreme conditions. It identifies bottlenecks before they reach production and provides data to support capacity planning and SLO validation.

Types of Performance Tests#

Type	Purpose	Duration	Load Pattern
Load test	Validate behavior under expected load	5–30 min	Ramp to target, hold steady
Stress test	Find breaking point	15–60 min	Ramp beyond capacity
Soak test	Detect memory leaks, resource exhaustion	2–24 hours	Constant moderate load
Spike test	Validate auto-scaling and recovery	5–15 min	Sudden burst, then drop
Breakpoint test	Find maximum capacity	Variable	Incremental ramp until failure
Benchmark	Establish baseline for comparison	1–5 min	Fixed load, repeatable

Load Testing Tools Comparison#

Feature	k6	Locust	JMeter	Gatling
Language	JavaScript	Python	XML/GUI	Scala/Java
Protocol	HTTP, gRPC, WebSocket	HTTP, custom	HTTP, JDBC, JMS, FTP	HTTP, WebSocket
Scripting	Code-first	Code-first	GUI + code	Code-first (DSL)
Resource usage	Low (Go runtime)	Medium (Python)	High (Java)	Medium (JVM)
Distributed	Built-in (k6 Cloud)	Built-in (master/worker)	JMeter Remote	Distributed by design
Reporting	CLI + JSON + Grafana	Web UI	HTML/XML reports	HTML reports
Learning curve	Low	Low	Medium	Medium
Best for	CI/CD integration, developers	Python teams, custom protocols	Enterprise, complex scenarios	High-throughput HTTP

k6 — Developer-Friendly Load Testing#

Basic Load Test#

import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
 stages: [
 { duration: '1m', target: 50 }, // Ramp up to 50 users
 { duration: '3m', target: 50 }, // Hold at 50 users
 { duration: '1m', target: 100 }, // Ramp to 100 users
 { duration: '3m', target: 100 }, // Hold at 100 users
 { duration: '1m', target: 0 }, // Ramp down
 ],
 thresholds: {
 http_req_duration: ['p(95)<300', 'p(99)<500'], // SLO validation
 http_req_failed: ['rate<0.01'], // <1% error rate
 },
};

export default function () {
 const res = http.get('https://api.example.com/users');

 check(res, {
 'status is 200': (r) => r.status === 200,
 'response time < 300ms': (r) => r.timings.duration < 300,
 'body has users': (r) => JSON.parse(r.body).length > 0,
 });

 sleep(1); // Think time between requests
}

Running k6#

# Local run
k6 run load-test.js

# With custom output
k6 run --out json=results.json load-test.js

# With environment variables
k6 run -e API_URL=https://staging.example.com load-test.js

Locust — Python-Based Load Testing#

from locust import HttpUser, task, between

class WebsiteUser(HttpUser):
 wait_time = between(1, 3) # Think time: 1-3 seconds

 def on_start(self):
 """Login on start."""
 self.client.post("/login", json={
 "username": "testuser",
 "password": "testpass"
 })

 @task(3) # Weight: 3x more likely than other tasks
 def view_items(self):
 self.client.get("/api/items")

 @task(1)
 def create_item(self):
 self.client.post("/api/items", json={
 "name": "Test Item",
 "price": 29.99
 })

 @task(2)
 def view_item_detail(self):
 item_id = 42 # In practice, pick from a pool
 self.client.get(f"/api/items/{item_id}")

Running Locust#

# Start with web UI
locust -f locustfile.py --host=https://api.example.com

# Headless mode (for CI)
locust -f locustfile.py --headless -u 100 -r 10 --run-time 5m \
 --host=https://api.example.com

Stress Testing#

Push beyond expected capacity to find the breaking point and observe degradation patterns.

Probability

Mon, 01 Jan 0001 00:00:00 +0000

Probability#

Probability quantifies uncertainty. It is the mathematical framework behind reliability engineering, machine learning, A/B testing, cryptographic security, and capacity planning. If you build systems that interact with the real world, you reason about probability whether you formalise it or not.

Foundations#

Sample Space and Events#

Experiment: A process with uncertain outcome (rolling a die, receiving a request)
Sample space (Ω): The set of all possible outcomes
Event (A): A subset of the sample space

Rolling a die:
 Ω = {1, 2, 3, 4, 5, 6}
 A = "even number" = {2, 4, 6}
 P(A) = |A| / |Ω| = 3/6 = 1/2

Axioms of Probability#

P(A) ≥ 0 for any event A
P(Ω) = 1 (something must happen)
For mutually exclusive events: P(A ∪ B) = P(A) + P(B)

Basic Rules#

Rule	Formula
Complement	P(not A) = 1 - P(A)
Union	P(A ∪ B) = P(A) + P(B) - P(A ∩ B)
Mutually exclusive	P(A ∪ B) = P(A) + P(B)
Certain event	P(Ω) = 1
Impossible event	P(∅) = 0

Conditional Probability#

The probability of A given that B has occurred:

Python: Concurrency and Parallelism

Mon, 01 Jan 0001 00:00:00 +0000

The GIL — Global Interpreter Lock#

flowchart LR
 subgraph CPython
 GIL["GIL (one lock)"]
 T1["Thread 1"] -.->|"holds GIL"| GIL
 T2["Thread 2"] -.->|"waiting"| GIL
 T3["Thread 3"] -.->|"waiting"| GIL
 end
 
 subgraph "What this means"
 A["CPU-bound: threads DON'T help"]
 B["I/O-bound: threads DO help"]
 end

The GIL ensures only one thread executes Python bytecode at a time.

Task Type	Use	Why
I/O-bound (network, disk, DB)	`asyncio` or `threading`	GIL is released during I/O waits
CPU-bound (math, parsing, ML)	`multiprocessing`	Separate processes, each with own GIL
Mixed	`asyncio` + `ProcessPoolExecutor`	Async for I/O, processes for CPU

Threading — Concurrent I/O#

import threading
from concurrent.futures import ThreadPoolExecutor, as_completed
import time

# Basic thread
def download(url: str) -> str:
 time.sleep(1) # Simulates network I/O
 return f"Downloaded {url}"

# ThreadPoolExecutor — the modern way
def download_all(urls: list[str]) -> list[str]:
 results = []
 with ThreadPoolExecutor(max_workers=10) as executor:
 # Submit all tasks
 futures = {executor.submit(download, url): url for url in urls}
 
 # Process results as they complete
 for future in as_completed(futures):
 url = futures[future]
 try:
 result = future.result(timeout=30)
 results.append(result)
 except Exception as e:
 print(f"Failed {url}: {e}")
 
 return results

# 10 downloads in ~1 second instead of ~10 seconds
urls = [f"http://example.com/page/{i}" for i in range(10)]
download_all(urls)

Thread Safety#

import threading

# Race condition example
counter = 0

def increment():
 global counter
 for _ in range(100_000):
 counter += 1 # NOT atomic! Read-modify-write

threads = [threading.Thread(target=increment) for _ in range(4)]
for t in threads: t.start()
for t in threads: t.join()
print(counter) # Less than 400,000 — race condition!

# Fix: use a Lock
lock = threading.Lock()
counter = 0

def safe_increment():
 global counter
 for _ in range(100_000):
 with lock: # Only one thread at a time
 counter += 1

# Other synchronization primitives
event = threading.Event() # Signal between threads
semaphore = threading.Semaphore(5) # Limit concurrent access
barrier = threading.Barrier(3) # Wait for N threads to arrive

Multiprocessing — True Parallelism#

from multiprocessing import Pool, Process, Queue
from concurrent.futures import ProcessPoolExecutor
import os

# CPU-bound task
def compute_heavy(n: int) -> int:
 """Simulate CPU-intensive work."""
 return sum(i * i for i in range(n))

# ProcessPoolExecutor — cleanest API
def parallel_compute(tasks: list[int]) -> list[int]:
 with ProcessPoolExecutor(max_workers=os.cpu_count()) as executor:
 results = list(executor.map(compute_heavy, tasks))
 return results

# Each process has its own Python interpreter and GIL
# Data is serialized (pickled) between processes

# Pool with imap for large datasets (lazy, ordered)
def process_large_dataset(items: list) -> list:
 with Pool(processes=4) as pool:
 # imap_unordered: results come as they finish (faster)
 results = []
 for result in pool.imap_unordered(transform, items, chunksize=100):
 results.append(result)
 return results

Shared State Between Processes#

from multiprocessing import Value, Array, Manager

# Shared memory (fast, limited types)
shared_counter = Value('i', 0) # 'i' = integer
shared_array = Array('d', [0.0] * 10) # 'd' = double

def worker(counter, array, index):
 with counter.get_lock():
 counter.value += 1
 array[index] = counter.value

# Manager (slower, supports complex types)
with Manager() as manager:
 shared_dict = manager.dict()
 shared_list = manager.list()
 
 # These can be passed to child processes
 # Changes are synchronized automatically

asyncio — Asynchronous I/O#

Core Concepts#

flowchart TD
 EL["Event Loop"] --> C1["Coroutine 1"]
 EL --> C2["Coroutine 2"]
 EL --> C3["Coroutine 3"]
 
 C1 -->|"await (I/O)"| EL
 C2 -->|"await (I/O)"| EL
 C3 -->|"running"| CPU["CPU"]
 
 Note["Single thread, cooperative multitasking.<br>Coroutines yield control at 'await' points."]

import asyncio
import aiohttp

# Coroutine — defined with async def
async def fetch_url(session: aiohttp.ClientSession, url: str) -> str:
 async with session.get(url) as response:
 return await response.text()

# Run multiple coroutines concurrently
async def fetch_all(urls: list[str]) -> list[str]:
 async with aiohttp.ClientSession() as session:
 tasks = [fetch_url(session, url) for url in urls]
 return await asyncio.gather(*tasks)

# Entry point
results = asyncio.run(fetch_all(urls))

asyncio Patterns#

import asyncio
from asyncio import Semaphore, Queue

# Rate limiting with semaphore
async def rate_limited_fetch(urls: list[str], max_concurrent: int = 5) -> list[str]:
 semaphore = Semaphore(max_concurrent)
 
 async def fetch_with_limit(url: str) -> str:
 async with semaphore: # At most max_concurrent at a time
 return await fetch_url(url)
 
 tasks = [fetch_with_limit(url) for url in urls]
 return await asyncio.gather(*tasks)

# Producer-consumer with asyncio.Queue
async def producer(queue: Queue, items: list):
 for item in items:
 await queue.put(item)
 await queue.put(None) # Sentinel to signal completion

async def consumer(queue: Queue, name: str):
 while True:
 item = await queue.get()
 if item is None:
 queue.task_done()
 break
 await process(item)
 queue.task_done()

async def pipeline(items: list, num_consumers: int = 3):
 queue: Queue = Queue(maxsize=100)
 
 producers = [asyncio.create_task(producer(queue, items))]
 consumers = [
 asyncio.create_task(consumer(queue, f"consumer-{i}"))
 for i in range(num_consumers)
 ]
 
 await asyncio.gather(*producers)
 await queue.join() # Wait for all items to be processed
 
 for c in consumers:
 c.cancel()

# Timeout handling
async def fetch_with_timeout(url: str, timeout: float = 10.0) -> str:
 try:
 return await asyncio.wait_for(fetch_url(url), timeout=timeout)
 except asyncio.TimeoutError:
 return f"Timeout fetching {url}"

# TaskGroup (Python 3.11+) — structured concurrency
async def fetch_all_structured(urls: list[str]) -> list[str]:
 results = []
 async with asyncio.TaskGroup() as tg:
 for url in urls:
 tg.create_task(fetch_and_append(url, results))
 # All tasks complete or all are cancelled if one raises
 return results

Async Generators and Iterators#

async def stream_events(url: str):
 """Async generator — yields events as they arrive."""
 async with aiohttp.ClientSession() as session:
 async with session.get(url) as response:
 async for line in response.content:
 if line.strip():
 yield json.loads(line)

# Consuming async generators
async def process_stream():
 async for event in stream_events("http://api.example.com/events"):
 handle_event(event)

# Async comprehension
async def get_all_users() -> list[dict]:
 return [user async for user in fetch_users_paginated()]

Combining Approaches#

import asyncio
from concurrent.futures import ProcessPoolExecutor

async def hybrid_pipeline(urls: list[str]) -> list[dict]:
 """Fetch data async, process CPU-heavy in parallel."""
 
 # Step 1: Fetch all data concurrently (I/O-bound → asyncio)
 raw_data = await fetch_all(urls)
 
 # Step 2: Process data in parallel (CPU-bound → multiprocessing)
 loop = asyncio.get_event_loop()
 with ProcessPoolExecutor() as pool:
 processed = await asyncio.gather(*[
 loop.run_in_executor(pool, cpu_heavy_transform, data)
 for data in raw_data
 ])
 
 return processed

Synchronization Primitives Comparison#

Primitive	threading	asyncio	Use Case
Lock	`threading.Lock()`	`asyncio.Lock()`	Mutual exclusion
Semaphore	`threading.Semaphore(n)`	`asyncio.Semaphore(n)`	Limit concurrency
Event	`threading.Event()`	`asyncio.Event()`	Signal between tasks
Queue	`queue.Queue()`	`asyncio.Queue()`	Producer-consumer
Barrier	`threading.Barrier(n)`	`asyncio.Barrier(n)`	Wait for N tasks

Hypothetical Use Cases#

Use Case: Web Scraper with Rate Limiting#

import asyncio
import aiohttp
from dataclasses import dataclass, field
from datetime import datetime

@dataclass
class ScrapeResult:
 url: str
 status: int
 content_length: int
 elapsed_ms: float
 error: str | None = None

async def scrape_with_rate_limit(
 urls: list[str],
 max_concurrent: int = 10,
 delay_between: float = 0.1,
) -> list[ScrapeResult]:
 """Scrape URLs with concurrency limit and politeness delay."""
 semaphore = asyncio.Semaphore(max_concurrent)
 results: list[ScrapeResult] = []
 
 async def fetch_one(session: aiohttp.ClientSession, url: str) -> ScrapeResult:
 async with semaphore:
 start = datetime.now()
 try:
 async with session.get(url, timeout=aiohttp.ClientTimeout(total=30)) as resp:
 content = await resp.read()
 elapsed = (datetime.now() - start).total_seconds() * 1000
 return ScrapeResult(
 url=url, status=resp.status,
 content_length=len(content), elapsed_ms=elapsed,
 )
 except Exception as e:
 elapsed = (datetime.now() - start).total_seconds() * 1000
 return ScrapeResult(
 url=url, status=0, content_length=0,
 elapsed_ms=elapsed, error=str(e),
 )
 finally:
 await asyncio.sleep(delay_between)
 
 async with aiohttp.ClientSession() as session:
 tasks = [fetch_one(session, url) for url in urls]
 results = await asyncio.gather(*tasks)
 
 return results

Use Case: Parallel Data Processing Pipeline#

from multiprocessing import Pool
from concurrent.futures import ProcessPoolExecutor
from pathlib import Path
import json

def process_file(file_path: str) -> dict:
 """CPU-intensive: parse, validate, transform a data file."""
 path = Path(file_path)
 data = json.loads(path.read_text())
 
 # Heavy computation
 transformed = {
 "source": path.name,
 "record_count": len(data),
 "summary": compute_statistics(data),
 "anomalies": detect_anomalies(data),
 }
 return transformed

def parallel_process_directory(data_dir: str) -> list[dict]:
 """Process all JSON files in a directory using all CPU cores."""
 files = list(Path(data_dir).glob("*.json"))
 
 with ProcessPoolExecutor() as executor:
 results = list(executor.map(
 process_file,
 [str(f) for f in files],
 chunksize=10, # Send 10 files per worker batch
 ))
 
 return results

Key Takeaways#

GIL means threads don’t help for CPU-bound work — use multiprocessing
asyncio is for I/O-bound concurrency — single thread, cooperative scheduling
ThreadPoolExecutor is the simplest way to parallelize I/O operations
ProcessPoolExecutor gives true parallelism for CPU-bound work
asyncio.gather runs coroutines concurrently; TaskGroup (3.11+) adds structured cancellation
Semaphores limit concurrency — essential for rate limiting
Never mix blocking I/O with asyncio — use run_in_executor for blocking calls
Data between processes is pickled — keep shared data minimal

Software Engineering Learning Paths

Architecture Fundamentals

Architecture Fundamentals#

What Is Software Architecture?#

Arrays & Strings

Arrays & Strings#

Pattern Complexity Table#

Two Pointers#

AWS Global Infrastructure

The Physical Layer#

Regions#

C Fundamentals

C Fundamentals#

Hello World#

Anatomy#

Compilation Pipeline#

Compiler Flags You Should Always Use#

Types#

Integer Types#

Cloud Computing Fundamentals

Cloud Computing Fundamentals#

Cloud Service Models#

Service Model Comparison#

Choosing a Service Model#

The Shared Responsibility Model#

Communication

Communication#

Written Communication#

The Pyramid Principle#

Computer Architecture

The Von Neumann Model#

CPU (Central Processing Unit)#

The Instruction Cycle#

Core Concepts

Core Concepts#

What Kubernetes Solves#

Data Engineering Fundamentals

What Is Data Engineering?#

The Data Engineer’s Responsibilities#

Data Engineering vs Adjacent Roles#

The Data Lifecycle#

DevOps Principles

DevOps Principles#

The Three Ways of DevOps#

DevOps Culture — CALMS#

Distributed Systems Fundamentals

Distributed Systems Fundamentals#

Fundamentals

Why System Design?#

Core Metrics#

Latency#

Getting Started

Getting Started#

Installation#

Toolchain Management#

Hello World#

What’s println!?#

Git

What Is Git?#

Why Git?#

Core Concepts#

The Three States#

Git Internals

Git Internals#

The .git Directory Anatomy#

The Four Object Types#

Go Fundamentals

Go Fundamentals#

Philosophy#

The Go Proverbs#

HTML Semantics and Accessibility

Why Semantics Matter#

Document Structure#

The Semantic Skeleton#

Heading Hierarchy#

Infrastructure as Code Concepts

The Problem IaC Solves#

Introduction to Machine Learning

Introduction to Machine Learning#

What Is Machine Learning?#

What’s `println!`?#