Operating Systems on Software Engineering Learning Paths

Processes & Threads

Mon, 01 Jan 0001 00:00:00 +0000

Processes & Threads#

A process is the OS’s abstraction for a running program. It encapsulates the program’s code, data, and execution state into an isolated unit that the OS can manage, schedule, and protect.

The Process Model#

A process is more than just executing code. It includes:

Component	Description
Text segment	The compiled program instructions
Data segment	Global and static variables
Heap	Dynamically allocated memory (grows upward)
Stack	Function call frames, local variables (grows downward)
Registers	CPU register state (PC, SP, general-purpose)
Open files	File descriptors the process holds
Credentials	User ID, group ID, permissions

Process Memory Layout#

High addresses
┌──────────────────┐
│ Stack │ ← grows downward
│ ↓ │
├──────────────────┤
│ │
│ (free space) │
│ │
├──────────────────┤
│ ↑ │
│ Heap │ ← grows upward
├──────────────────┤
│ BSS (uninit) │
├──────────────────┤
│ Data (init) │
├──────────────────┤
│ Text │ ← program code
└──────────────────┘
Low addresses

Process States#

Every process transitions through a well-defined set of states:

CPU Scheduling

Mon, 01 Jan 0001 00:00:00 +0000

CPU Scheduling#

The CPU scheduler decides which ready process gets the CPU next. This decision directly impacts system responsiveness, throughput, and fairness. A good scheduler makes a fast machine feel fast; a bad one makes it feel sluggish regardless of hardware.

Scheduling Criteria#

Criterion	Definition	Optimize
CPU utilization	Percentage of time CPU is busy	Maximize
Throughput	Processes completed per time unit	Maximize
Turnaround time	Total time from submission to completion	Minimize
Waiting time	Time spent in the ready queue	Minimize
Response time	Time from submission to first response	Minimize

These criteria often conflict. Maximizing throughput may hurt response time; optimizing for interactive responsiveness may reduce batch throughput.

Memory Management

Mon, 01 Jan 0001 00:00:00 +0000

Memory Management#

The OS must multiplex physical memory among many processes while giving each the illusion of a large, private, contiguous address space. Memory management handles allocation, protection, and efficient use of RAM.

Address Spaces#

Every process operates in its own virtual address space — a logical view of memory that the hardware translates to physical addresses.

Concept	Description
Logical (virtual) address	What the process sees (generated by CPU)
Physical address	Actual location in RAM
Address translation	Mapping logical → physical (done by MMU)
Protection	One process cannot access another’s memory

Paging#

Paging divides memory into fixed-size blocks, eliminating external fragmentation:

Virtual Memory

Mon, 01 Jan 0001 00:00:00 +0000

Virtual Memory#

Virtual memory extends the basic paging model into a complete system for managing address spaces larger than physical RAM. It encompasses the page table structures, swap management, memory overcommit policies, and hardware-aware optimizations like NUMA and huge pages.

Virtual vs Physical Addresses#

Aspect	Virtual Address	Physical Address
Generated by	CPU / program	MMU (after translation)
Space size	Full architecture width (e.g., 48-bit on x86-64)	Limited by installed RAM
Per-process	Yes — each process has its own	Shared — single physical space
Contiguous?	Appears contiguous to process	Scattered frames
Bounds	0 to 2^48 - 1 (256 TB on x86-64)	0 to installed_ram - 1

Process A sees: Physical RAM: Process B sees:
┌──────────┐ ┌──────────┐ ┌──────────┐
│ 0x0000 │────────┐ │ Frame 0 │ ┌─────│ 0x0000 │
│ 0x1000 │──┐ └───▶│ Frame 1 │ │ │ 0x1000 │
│ 0x2000 │ │ │ Frame 2 │◀────┘ │ 0x2000 │
└──────────┘ └─────────▶│ Frame 3 │ └──────────┘
 │ Frame 4 │◀──── Process B page 1
 └──────────┘

Page Table Structures#

A flat page table for a 48-bit address space with 4KB pages would require 2^36 entries × 8 bytes = 512 GB. Clearly impractical. Real systems use hierarchical or inverted page tables.

File Systems

Mon, 01 Jan 0001 00:00:00 +0000

File Systems#

A file system organizes persistent data on storage devices into a logical structure of files and directories. It translates high-level operations (open, read, write) into low-level block I/O while maintaining data integrity, supporting concurrent access, and managing metadata.

File System Concepts#

Concept	Description
File	Named collection of related data stored on secondary storage
Directory	Container that maps file names to file metadata
Path	Hierarchical location identifier (`/home/user/docs/file.txt`)
Mount point	Location where a file system attaches to the directory tree
Superblock	File system metadata (size, block count, free space)
Block	Minimum allocation unit (typically 4 KB)

File Metadata (stored separately from data)#

┌─────────────────────────────────────────┐
│ File type │ Regular, directory, symlink, device, pipe │
│ Permissions │ rwxr-xr-x (owner/group/other) │
│ Owner/Group │ UID, GID │
│ Size │ Bytes of data │
│ Timestamps │ atime, mtime, ctime │
│ Link count │ Number of hard links │
│ Block pointers │ Locations of data blocks on disk │
└─────────────────────────────────────────┘

Inodes#

An inode (index node) stores all metadata about a file except its name:

I/O Systems

Mon, 01 Jan 0001 00:00:00 +0000

I/O Systems#

Input/Output systems bridge the gap between the CPU and the external world — disks, networks, keyboards, GPUs. Understanding how the OS manages I/O is critical for writing performant software and diagnosing bottlenecks.

I/O Hardware Fundamentals#

Every I/O device connects to the system through a controller that exposes registers to the CPU:

Register	Purpose
Status	Indicates device state (busy, ready, error)
Command	CPU writes operations (read, write, seek)
Data-in	CPU reads data from device
Data-out	CPU writes data to device

Port-Mapped vs Memory-Mapped I/O#

Approach	Mechanism	Example
Port-mapped	Special CPU instructions (`in`/`out`) access device registers	x86 legacy devices
Memory-mapped	Device registers mapped to memory addresses	Modern GPUs, NVMe

Memory-mapped I/O is dominant today — it allows the same load/store instructions to access both RAM and devices.

Concurrency Primitives

Mon, 01 Jan 0001 00:00:00 +0000

Concurrency Primitives#

Concurrency is the art of managing multiple threads of execution that share resources. Without proper synchronization, programs exhibit race conditions — subtle, non-deterministic bugs that are notoriously difficult to reproduce and fix.

Race Conditions and Critical Sections#

A race condition occurs when the outcome depends on the timing of thread execution:

counter = 0

def increment():
 global counter
 # NOT atomic — three operations:
 # 1. Read counter into register
 # 2. Add 1
 # 3. Write back to counter
 counter += 1

# Two threads running increment() 1000 times each
# Expected: 2000 | Actual: unpredictable (e.g., 1847)

A critical section is a code region accessing shared resources that must not execute concurrently.

Containers from the OS Perspective

Mon, 01 Jan 0001 00:00:00 +0000

Containers from the OS Perspective#

Containers are not a single technology — they are an orchestration of several Linux kernel features that together provide process isolation. Understanding these primitives demystifies Docker and reveals what containers actually are: isolated groups of processes sharing the host kernel.

Linux Namespaces#

Namespaces partition kernel resources so that one set of processes sees one set of resources, and another set sees a different set.

Namespace	Isolates	Flag	Effect
PID	Process IDs	`CLONE_NEWPID`	Process sees itself as PID 1
NET	Network stack	`CLONE_NEWNET`	Own interfaces, IPs, routing tables
MNT	Mount points	`CLONE_NEWNS`	Own filesystem tree
UTS	Hostname/domain	`CLONE_NEWUTS`	Own hostname
IPC	IPC resources	`CLONE_NEWIPC`	Own message queues, semaphores
USER	UID/GID mappings	`CLONE_NEWUSER`	Root inside, unprivileged outside
CGROUP	Cgroup hierarchy	`CLONE_NEWCGROUP`	Own view of cgroup tree
TIME	System clocks	`CLONE_NEWTIME`	Own boot/monotonic time (Linux 5.6+)

Creating Namespaces#

# Create a new PID + MNT + NET namespace and run bash inside
sudo unshare --pid --mount --net --fork bash

# Inside: PID 1, no network interfaces, isolated mounts
ps aux # only see processes in this namespace
ip link # only loopback
hostname # still host's (need --uts to isolate)

PID Namespace in Action#

Host view: Container view:
PID 1 (systemd) PID 1 (nginx) ← mapped to PID 4521 on host
PID 4521 (nginx) PID 2 (worker)
PID 4522 (worker)

User Namespaces — Rootless Containers#

# Map container root (UID 0) to host UID 100000
unshare --user --map-root-user bash
id # uid=0(root) — but only inside the namespace

This enables rootless containers: full “root” power inside, zero privilege on the host.