Quartz 4

Kubernetes

5 min read

Kubernetes (often abbreviated as K8s) is an open-source container orchestration platform designed to automate deploying, scaling, and managing containerized applications. Originally developed by Google and now maintained by the Cloud Native Computing Foundation (CNCF), Kubernetes has become the de facto standard for container orchestration.

History and Background

  • Origin: Developed by Google based on their internal system called Borg
  • Release: Open-sourced in 2014
  • Name: Greek for “helmsman” or “pilot” (hence the ship’s wheel logo)
  • CNCF: Became the first graduated project of the Cloud Native Computing Foundation in 2018

Core Concepts

Kubernetes Architecture

Kubernetes follows a master-worker (also called control plane and node) architecture:

Control Plane Components

  • API Server: Front-end for the Kubernetes control plane, exposing the Kubernetes API
  • etcd: Consistent and highly-available key-value store for all cluster data
  • Scheduler: Watches for newly created pods with no assigned node and selects nodes for them to run on
  • Controller Manager: Runs controller processes that regulate the state of the cluster
  • Cloud Controller Manager: Links the cluster to cloud provider APIs

Node Components

  • Kubelet: An agent that runs on each node, ensuring containers are running in a pod
  • Kube-proxy: Network proxy that maintains network rules on nodes
  • Container Runtime: Software responsible for running containers (e.g., Docker, containerd, CRI-O)

Kubernetes Objects

Pods

The smallest deployable units in Kubernetes:

  • Group of one or more containers with shared storage/network resources
  • Ephemeral (not designed to survive failures)
  • Should be managed by higher-level controllers, not directly
apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod
spec:
  containers:
  - name: nginx
    image: nginx:1.14.2
    ports:
    - containerPort: 80

Deployments

Controllers for creating and updating instances of your applications:

  • Define desired state for your application
  • Handle rolling updates and rollbacks
  • Manage ReplicaSets
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

Services

An abstraction to expose applications running on pods:

  • Provides stable network endpoint
  • Enables load balancing
  • Facilitates service discovery

Types of services:

  • ClusterIP: Internal only (default)
  • NodePort: Exposes on each node’s IP at a static port
  • LoadBalancer: Exposes externally using cloud provider’s load balancer
  • ExternalName: Maps service to DNS name
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app: nginx
  ports:
  - port: 80
    targetPort: 80
  type: ClusterIP

StatefulSets

Manages the deployment and scaling of a set of pods with persistent identities:

  • Stable, unique network identifiers
  • Stable, persistent storage
  • Ordered, graceful deployment and scaling
  • Used for stateful applications (databases, etc.)

DaemonSets

Ensures all (or some) nodes run a copy of a pod:

  • Used for node monitoring, log collection
  • Useful for cluster-wide services (e.g., networking plugins)
  • Automatically adds pods to new nodes

ConfigMaps and Secrets

For configuration and sensitive data:

  • ConfigMaps: Store non-confidential configuration data
  • Secrets: Store sensitive information (passwords, tokens, keys)

Namespaces

Virtual clusters inside a physical cluster:

  • Provide scope for names
  • Allow resource quotas
  • Enable multi-tenant environments

Kubernetes Networking

Kubernetes networking addresses four concerns:

  1. Container-to-container communication: Solved by pods and localhost communications
  2. Pod-to-pod communication: Flat network space where pods can communicate with all other pods
  3. Pod-to-service communication: Through kube-proxy and virtual IPs
  4. External-to-internal communication: Through services of type NodePort, LoadBalancer, or Ingress resources

Network Policies

Specifications of how groups of pods are allowed to communicate:

  • Similar to network firewalls
  • Restrict traffic to/from pods based on rules

Storage in Kubernetes

Kubernetes provides several abstractions for persistent storage:

Volumes

Basic building block for storage that outlives containers:

  • Many volume types (e.g., emptyDir, hostPath, nfs, cloud provider volumes)
  • Mounted into pods

Persistent Volumes (PV) and Persistent Volume Claims (PVC)

Decouple storage provisioning from usage:

  • PV: Cluster resource provisioned by administrator or dynamically
  • PVC: Request for storage by a user
  • Storage Classes: Define types of storage and provisioners

Resource Management

Kubernetes provides mechanisms for resource control:

Resource Requests and Limits

  • Requests: Minimum resources guaranteed to the container
  • Limits: Maximum resources a container can use
resources:
  requests:
    memory: "64Mi"
    cpu: "250m"
  limits:
    memory: "128Mi"
    cpu: "500m"

Horizontal Pod Autoscaler (HPA)

Automatically scales the number of pods based on observed metrics:

  • CPU utilization
  • Memory usage
  • Custom metrics
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: nginx-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: nginx-deployment
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50

Vertical Pod Autoscaler (VPA)

Automatically adjusts resource requests and limits for containers:

  • Recommends and can automatically update resource configurations
  • Helps right-size container resources

Kubernetes Extensions and Ecosystem

Helm

The package manager for Kubernetes:

  • Templates for Kubernetes resources
  • Manages releases of applications
  • Facilitates sharing applications through Helm charts

Operators

Pattern for encoding domain knowledge into Kubernetes:

  • Custom controllers that extend Kubernetes API
  • Manage complex applications like databases, monitoring systems
  • Automate operational tasks

Service Meshes

Infrastructure layer for service-to-service communication:

  • Examples: Istio, Linkerd, Consul
  • Provide traffic management, security, observability
  • Decouple application code from network functionality

Ingress Controllers

Manage external access to services:

  • Examples: Nginx Ingress, Traefik, HAProxy
  • Implement HTTP routing rules
  • Often provide SSL termination

Kubernetes Deployment Options

Self-Managed

  • Kubeadm: Tool for creating Kubernetes clusters
  • kubespray: Ansible playbooks for deploying Kubernetes
  • kOps: Kubernetes Operations, production-grade tooling
  • Minikube: Local Kubernetes for development

Managed Services

  • Amazon EKS: Elastic Kubernetes Service
  • Google GKE: Google Kubernetes Engine
  • Azure AKS: Azure Kubernetes Service
  • DigitalOcean DOKS: DigitalOcean Kubernetes
  • IBM Cloud Kubernetes Service
  • Oracle Container Engine for Kubernetes

Advantages of Kubernetes

  • Portability: Run applications consistently across environments
  • Scalability: Automatic scaling based on demand
  • High Availability: Self-healing, automatic placement
  • Extensibility: API-driven, customizable with CRDs
  • Service Discovery: Built-in DNS and load balancing
  • Rolling Updates: Zero-downtime deployments
  • Secret Management: Secure handling of sensitive data

Challenges and Considerations

  • Complexity: Steep learning curve
  • Resource Overhead: Control plane requires resources
  • Stateful Applications: More complex to manage
  • Security: Requires careful configuration
  • Observability: Needs additional tooling for monitoring

Graph View

Backlinks