Beyond docker run#

Running a container locally is one thing. Running dozens — or thousands — of containers reliably in production requires orchestration, security hardening, observability, and disciplined image management. This section bridges the gap between local Docker usage and production-grade container infrastructure.

Container Orchestration Concepts#

Why Orchestrate?#

A single container on a single host has no resilience. If it crashes, nothing restarts it. If traffic spikes, nothing scales it. Orchestration solves these problems:

Core Concepts#

Desired State Reconciliation — You declare what you want (e.g., “run 3 replicas of service X”), and the orchestrator continuously works to make reality match your declaration. If a container dies, the orchestrator starts a new one without human intervention.

Scheduling — The orchestrator decides which host runs each container. It considers available CPU, memory, affinity rules (keep related containers together), anti-affinity rules (spread replicas across failure domains), and constraints (e.g., GPU-only nodes).

Service Discovery — Containers need to find each other. Rather than hardcoding IP addresses that change constantly, orchestrators provide DNS-based or environment-variable-based discovery. A container asks for “the database service” and gets routed to a healthy instance automatically.

Load Balancing — Traffic is distributed across replicas. The orchestrator tracks which instances are healthy and routes only to those that pass health checks.

Orchestration Platforms Compared#

Container Registries#

A registry is where container images are stored, versioned, and distributed — think of it as “npm/PyPI for containers.”

Push/Pull Workflow#

Developer Machine           Registry              Production Host
─────────────────          ─────────              ───────────────
docker build .        →    docker push        →   docker pull
  (creates image)          (uploads to registry)  (downloads image)
                           stores layers           runs container
                           tracks tags

# Build and tag for a registry
docker build -t registry.example.com/myapp:1.2.0 .

# Authenticate
docker login registry.example.com

# Push to registry
docker push registry.example.com/myapp:1.2.0

# Pull from registry (usually automated by orchestrator)
docker pull registry.example.com/myapp:1.2.0

Registry Options#

Tagging Strategies#

Tags identify specific versions of an image. Poor tagging causes deployment chaos.

Best practice: Use immutable tags (semver or git SHA) in production. Reserve latest for development only. Never deploy latest to production — you cannot know what version is running or roll back reliably.

Vulnerability Scanning#

Registries can scan images for known CVEs in OS packages and application dependencies:

# Scan locally with Trivy
trivy image myapp:1.2.0

# Scan with Docker Scout
docker scout cves myapp:1.2.0

# Scan with Grype
grype myapp:1.2.0

Most managed registries (ECR, GCR, Harbor) offer automatic scanning on push.

Image Security#

Base Image Selection#

Your base image is your security foundation. Every package in the base image is a potential vulnerability.

Rules of thumb:

• Use the smallest base that meets your needs
• Prefer -slim variants over full images
• Use distroless for production when you don’t need shell access for debugging
• Pin base image versions (not latest)

Multi-Stage Builds (Security Perspective)#

Multi-stage builds are covered in the Docker section, but their security benefit deserves emphasis:

# Stage 1: Build (has compilers, dev tools — large attack surface)
FROM node:22 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

# Stage 2: Production (minimal — no build tools, no source code)
FROM node:22-slim
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
USER node
CMD ["node", "dist/server.js"]

The final image contains only what’s needed to run — no compilers, no source code, no dev dependencies. This dramatically reduces the attack surface.

Image Signing and Trust#

Image signing proves that an image was built by a trusted party and hasn’t been tampered with:

# Sign with Cosign (keyless, uses OIDC identity)
cosign sign registry.example.com/myapp:1.2.0

# Verify
cosign verify registry.example.com/myapp:1.2.0

Scanning Tools Comparison#

Container Logging and Monitoring#

The stdout/stderr Convention#

Containers follow a simple logging rule: write everything to stdout and stderr. Don’t write to files inside the container.

Application → stdout/stderr → Container Runtime → Log Driver → Aggregation

Why:

• The container runtime captures stdout/stderr automatically
• Log drivers forward logs to any backend (CloudWatch, Datadog, ELK, Loki)
• No log files fill up the container filesystem
• Logs survive container restarts (they’re outside the container)
• Consistent collection regardless of application language or framework

# Python — correct (logs to stdout)
import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
logger.info("Request processed")

# Wrong — logs to file inside container
logging.basicConfig(filename='/var/log/app.log')  # Don't do this

Log Aggregation Patterns#

Health Checks#

Health checks let the orchestrator know if a container is functioning correctly:

HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
  CMD curl -f http://localhost:8080/health || exit 1

Readiness vs. Liveness Probes#

Example scenarios:

• App is deadlocked → Liveness probe fails → Container restarted
• App is loading a large cache → Readiness probe fails → No traffic routed until ready
• App takes 60s to start → Startup probe gives it time before liveness kicks in

Bridge to Kubernetes#

What Kubernetes Solves#

Kubernetes (K8s) is the industry-standard container orchestrator. It implements all the orchestration concepts above — scheduling, service discovery, desired state, scaling — at massive scale.

Key Abstractions (High Level)#

Pod — The smallest deployable unit. One or more containers that share networking and storage. Usually one main container per pod.

Service — A stable network endpoint for a set of pods. Pods come and go, but the Service DNS name stays constant. Internal services get cluster-internal IPs; external services get load balancers.

Deployment — Declares the desired state for a set of pods: which image, how many replicas, resource limits, update strategy. The Deployment controller creates and manages ReplicaSets to achieve the desired state.

Namespace — A logical partition of a cluster. Teams or environments (dev, staging) get their own namespace for isolation.

Kubernetes vs. Simpler Alternatives#

When to choose Kubernetes: Multi-cloud requirements, complex service mesh needs, large engineering team with platform expertise, or when the ecosystem tooling (Helm charts, operators, GitOps) solves real problems for you.

When to avoid Kubernetes: Small team, single cloud provider, fewer than ~10 services, or when a managed service (ECS, Cloud Run, App Runner) meets your needs with less operational burden.

Key Takeaways#

1. Orchestration solves resilience, scaling, and deployment — You declare desired state; the system reconciles reality to match
2. Use immutable image tags in production — Semantic versions or git SHAs, never latest
3. Scan images for vulnerabilities — Automate scanning on push to your registry; block critical CVEs from deploying
4. Minimise your base image — Smaller images mean fewer vulnerabilities and faster pulls; prefer slim/distroless/scratch
5. Log to stdout/stderr — Let the platform handle log collection and routing, not your application
6. Implement health checks — Liveness probes detect crashes; readiness probes prevent traffic to unhealthy instances
7. Choose orchestration complexity to match your needs — ECS for AWS simplicity, Kubernetes for flexibility and scale
8. Multi-stage builds are a security tool — They keep build dependencies out of your production image