Linux for Backend Engineers

• Part 1: The Foundation
• Part 2: Files, Filesystems, and Permissions
• Part 3: Processes and Threads ← current
• Part 4: Memory Management
• Part 5: Signals and Communication
• Part 6: IPC, Networking, and Containers

9. Processes & Threads: The Execution Environment

So far we've seen how Linux manages files, permissions, and hardware. Now we need to understand what actually runs on the CPU.

The two fundamental concepts are:

• Process — a running program together with the resources it owns.
• Thread — an execution context inside a process.

Understanding these two concepts is essential for understanding Node.js, concurrency, containers, and performance.

9.1 Program vs Process

A program is a passive file containing instructions. It sits on disk, doing nothing.

A process is a running instance of that program. It's alive, consuming resources, executing instructions.

Disk                    Memory (RAM)
─────                   ────────────
node app.js  ──────►    Process
(program)               (running instance)
                        ├── Virtual address space
                        ├── CPU registers
                        ├── File descriptors
                        └── Execution state

When you run node app.js:

The kernel:

1. Reads the executable file from disk
2. Allocates virtual memory for the process
3. Sets up the execution environment
4. Begins executing instructions

A process is the environment in which a program executes.

The program is the recipe. The process is the kitchen where the recipe comes alive.

9.2 Process Attributes

Every process has a rich set of attributes maintained by the kernel:

Inspect it yourself:

# Start a long-running Node process
node -e "setTimeout(() => {}, 60000)" &
PID=$!
 
# See everything about it
cat /proc/$PID/status
 
# Or specific attributes
ps -o pid,ppid,uid,state,pri,vsz,rss,cmd -p $PID

From Node.js:

console.log("Process ID:", process.pid);
console.log("Parent PID:", process.ppid);
console.log("User ID:", process.getuid());
console.log("Current directory:", process.cwd());
console.log("Environment:", process.env.PATH);
console.log("Memory usage:", process.memoryUsage());
console.log("CPU time:", process.cpuUsage());

9.3 What Does a Process Look Like?

A process has its own virtual address space.

┌──────────────────────────────┐
│          Code / Text         │
├──────────────────────────────┤
│      Global / Static Data    │
├──────────────────────────────┤
│          Heap                │
│        ↓ grows               │
│                              │
│       Memory Mappings        │
│                              │
│        ↑ grows               │
│          Stack               │
└──────────────────────────────┘

The process sees a virtual address space that is isolated from other processes:

Process A                         Process B

Virtual Memory                    Virtual Memory
┌──────────────┐                  ┌──────────────┐
│ Code         │                  │ Code         │
│ Heap         │                  │ Heap         │
│ Stack        │                  │ Stack        │
└──────────────┘                  └──────────────┘

Process A normally cannot directly access Process B's memory. This isolation is one of the fundamental protections provided by the kernel.

Section 10 explains how these virtual addresses are mapped to physical RAM.

9.4 The Process Tree: Parent-Child Relationships

Every process (except the first) is created by another process. This forms a tree structure:

systemd (PID 1)
│
├── sshd (PID 1200)
│   └── sshd (PID 2300) — connection from user
│       └── bash (PID 2301)
│           └── node app.js (PID 4500)
│               └── node worker.js (PID 4501)
│
├── nginx (PID 800)
│   ├── nginx worker (PID 801)
│   ├── nginx worker (PID 802)
│   └── nginx worker (PID 803)
│
└── postgres (PID 500)
    ├── postgres writer (PID 501)
    ├── postgres checkpointer (PID 502)
    └── postgres stats collector (PID 503)

Inspect the tree:

# See the full process tree
pstree -p
 
# See just your process's tree
pstree -p $$
 
# See a specific process's ancestry
ps -o pid,ppid,cmd --forest

Why this matters:

• Signals propagate - Ctrl+C in a terminal sends SIGINT to the entire foreground process group
• Orphaned processes - if a parent dies, children get re-parented to PID 1
• Resource accounting - container limits apply to entire process trees
• Cleanup - if you kill a parent, children might survive (or not, depending on how you kill it)

9.5 Process Lifecycle

A process doesn't simply "run or stop." It moves through different states.

             ┌──────────────┐
             │   Created    │
             └──────┬───────┘
                    │ fork()
                    ▼
             ┌──────────────┐
             │   Runnable   │
             │ waiting CPU  │
             └──────┬───────┘
                    │ scheduled
                    ▼
             ┌──────────────┐
      ┌──────│   Running    │──────┐
      │      └──────────────┘      │
      ▼                            ▼
┌──────────┐                  ┌─────────┐
│ Sleeping │                  │ exit()  │
│ waiting  │                  │ called  │
│ for I/O  │                  └────┬────┘
└────┬─────┘                       ▼
     │                        ┌─────────┐
     │                        │ Zombie  │ ← dead, exit code held
     │                        │         │   for parent to collect
     │                        └────┬────┘
     │                             │ parent calls wait()
     ▼                             ▼
  Runnable                    Reaped / removed
                              from process table

Common states (from ps STAT column):

Try it:

# Start a process and watch its states
node -e "
console.log('PID:', process.pid);
setInterval(() => {}, 1000);  // Running
" &
 
PID=$!
 
# Check state
ps -o pid,stat,cmd -p $PID
# PID  STAT CMD
# 4500  S   node -e ...    (sleeping — waiting for timer)
 
# Stop it (SIGSTOP)
kill -STOP $PID
ps -o pid,stat,cmd -p $PID
# PID  STAT CMD
# 4500  T   node -e ...    (stopped)
 
# Resume it (SIGCONT)
kill -CONT $PID
ps -o pid,stat,cmd -p $PID
# PID  STAT CMD
# 4500  S   node -e ...    (sleeping again)
 
# Kill it (SIGTERM)
kill -TERM $PID

9.6 Zombies: Dead but Not Gone

A zombie process has exited but hasn't been reaped by its parent. It's dead, not running, not consuming CPU, but still occupies an entry in the process table.

How zombies happen:

Parent Process                          Child Process
    │                                       │
    │  fork()                               │
    ├──────────────────────────────────────►│
    │                                       │
    │  (doing work)                         │  exit()
    │                                       ├───────┐
    │                                       │       │
    │                                       │◄──────┘
    │                                       │
    │  (still working,                      │  Zombie!
    │   hasn't called wait())               │  Waiting to be reaped
    │                                       │
    │  wait() — finally                     │
    ├──────────────────────────────────────►│
    │                                       │  Reaped (gone)

Why this matters in containers:

Node.js actually reaps its own direct children fine. libuv calls waitpid() on any process spawned via child_process as part of its normal bookkeeping, whether or not you attach an .on('exit') listener. So this does not produce zombies:

Container
└── Node.js (PID 1)
    ├── child_process.spawn("ffmpeg")  — exits, reaped by libuv, no zombie
    ├── child_process.spawn("curl")    — exits, reaped by libuv, no zombie
    └── child_process.spawn("python")  — exits, reaped by libuv, no zombie

The real risk is grandchildren. If a process you spawn itself forks children and then exits before they do, those grandchildren get reparented to whatever is PID 1 in the container, your Node process. PID 1 has a special kernel duty to reap any orphan reparented to it, not just processes it spawned directly. Node doesn't implement that generic reaping. It only reaps its own direct children, so orphaned grandchildren linger as zombies until Node itself exits.

Container
└── Node.js (PID 1)
    └── child_process.spawn("some-shell-pipeline")
        └── forks a subprocess, then exits early
            └── subprocess reparented to Node (PID 1) — never reaped — zombie

Detect zombies:

# Find zombie processes
ps aux | awk '$8 == "Z" {print}'
 
# Count them
ps aux | awk '$8 == "Z"' | wc -l

What actually fixes the grandchild/orphan problem: use an init system as PID 1

# Use tini as init system
FROM node:20
RUN apt install -y tini
ENTRYPOINT ["/usr/bin/tini", "--"]
CMD ["node", "app.js"]

Or in docker-compose:

services:
  app:
    image: node:20
    init: true # Docker injects tini automatically as PID 1
    command: node app.js

9.7 Creating Processes: fork(), exec(), wait()

Unix process management is built on three primitives:

fork() - Clone the Current Process

Creates a new process that's a near-copy of the parent. The child gets:

• Same memory (via copy-on-write)
• Same file descriptors
• Same execution point
• Different PID

Parent
  │
 fork()
  ├──────────► Parent
  │
  └──────────► Child

Why copy-on-write matters:

Parent Memory                Physical RAM
┌──────────────┐             ┌──────────────┐
│ Page A       │────────────►│ Frame 7      │◄───────────┐
│ Page B       │────────────►│ Frame 2      │            │
│ Page C       │────────────►│ Frame 15     │            │
└──────────────┘             └──────────────┘            │
                                                         │
Child Memory (after fork)                                │
┌──────────────┐                                         │
│ Page A       │─────────────────────────────────────────┘
│ Page B       │────────────► New Frame 20 (only when written)
│ Page C       │────────────► New Frame 22 (only when written)
└──────────────┘

Initially, both processes share the same physical memory. Only when one writes to a page does the kernel create a private copy.

exec() - Replace with New Program

Completely replaces the current process's program. The PID stays the same, but everything else changes:

Before exec():           After exec():
Node.js (PID 101)  ──►   /usr/bin/ls (PID 101)
├── V8 heap              ├── New code
├── JS stack             ├── New stack
└── Node modules         └── New memory

The process is still 101, but it's now running a completely different program.

wait() - Collect Child's Exit Status

Allows a parent process to wait for a child to terminate, collect its exit status, and reap the child.

Shell (Parent)
 │
 ├── fork()
 │     │
 │     └── Child
 │          │
 │          └── exec("ls")
 │                │
 │                └── ls runs
 │
 ├── wait()
 │     │
 │     │  (waits if child is still running)
 │     │
 │     ◄──────── child exits
 │
 └── child reaped

This model explains what happens conceptually when your shell launches commands.

How Node.js maps onto these primitives:

9.8 What Is a Thread?

A thread is an execution context within a process. While a process defines what resources exist, a thread defines what's actually executing.

Single-Threaded Process                Multi-Threaded Process
┌──────────────────────────────┐      ┌──────────────────────────────┐
│            PROCESS           │      │            PROCESS           │
│                              │      │                              │
│  Shared Resources            │      │  Shared Resources            │
│  ├── Virtual Address Space   │      │  ├── Virtual Address Space   │
│  ├── Heap                    │      │  ├── Heap                    │
│  ├── Global Variables        │      │  ├── Global Variables        │
│  ├── Shared Libraries        │      │  ├── Shared Libraries        │
│  └── File Descriptors        │      │  └── File Descriptors        │
│                              │      │                              │
│  ┌────────────────────────┐  │      │  ┌──────────┐  ┌──────────┐  │
│  │       THREAD 1         │  │      │  │ THREAD 1 │  │ THREAD 2 │  │
│  │                        │  │      │  │          │  │          │  │
│  │  ├── Stack             │  │      │  │ Stack    │  │ Stack    │  │
│  │  ├── Registers         │  │      │  │ Registers│  │ Registers│  │
│  │  └── Program Counter   │  │      │  │ PC       │  │ PC       │  │
│  └────────────────────────┘  │      │  └──────────┘  └──────────┘  │
│                              │      │                              │
└──────────────────────────────┘      └──────────────────────────────┘

What threads share vs. own:

9.9 Process vs Thread: The Comparison

Performance reality check:

Specific numbers vary by hardware and kernel, but general orders of magnitude:

• fork() without exec(): sub-millisecond (thanks to copy-on-write)
• fork() + exec(): several milliseconds (new program load)
• Same-process thread switch: ~1-5 microseconds
• Cross-process thread switch: ~2-10 microseconds (MMU overhead)

These differences explain why high-concurrency servers use threads or async I/O rather than spawning processes per connection.

9.10 The Program Counter: How Threads Execute

Every thread has its own Program Counter (PC), a CPU register that points to the next instruction to be executed.

The CPU execution cycle is brutally simple:

1. Fetch instruction at PC address
2. Decode instruction
3. Execute instruction
4. Update PC to next instruction
5. Repeat

Two threads running the exact same function can be at completely different points in it, because each has its own PC:

function processItems(items) {
  for (let i = 0; i < items.length; i++) {
    // Thread 1 PC: at line 4, processing i=5
    // Thread 2 PC: at line 2, initializing i=0
    // Thread 3 PC: at line 6, processing i=12
    processItem(items[i]);
  }
}

Each thread advances its PC independently. This is the concrete mechanism behind "concurrent code". Even threads executing the same shared function are really just several independent PCs walking through the same instructions at their own pace.

9.11 Concurrency vs Parallelism

These terms are often confused but mean different things:

Concurrency (Dealing with multiple tasks)

Concurrency means multiple tasks are in progress during overlapping periods of time.

On a single CPU core, tasks cannot execute CPU instructions at exactly the same instant. The system switches between tasks, or a task waits for I/O while another task makes progress.

Time →
 
CPU Core:  ├─Task A─┤├─Task B─┤├─Task C─┤├─Task A─┤├─Task B─┤├─Task C─┤

The CPU rapidly switches between tasks. At any instant, only one task runs, but all make progress over time.

Parallelism (Executing multiple tasks simultaneously)

Parallelism means multiple tasks are actually executing at the same time using multiple CPU cores.

Core 1: ├──────────Task A──────────┤
Core 2: ├──────────Task B──────────┤
Core 3: ├──────────Task C──────────┤
Core 4: ├──────────Task D──────────┤

Truly simultaneous execution.

The relationship:

Concurrency → Multiple tasks are in progress.
 
Parallelism → Multiple tasks execute simultaneously.
 
Concurrency can exist without parallelism.
Parallelism generally involves concurrent execution.
 
Single core:
    Concurrency without parallelism
 
Multiple cores:
    Concurrency + Parallelism

Node.js example:

// Concurrency (even on one core)
async function handleRequests() {
  await Promise.all([
    fetch("https://api1.example.com"),
    fetch("https://api2.example.com"),
    fetch("https://api3.example.com"),
  ]);
  // Three requests "in flight" simultaneously
  // But JS code executes one at a time
}
 
// Parallelism (requires multiple cores)
const { Worker } = require("worker_threads");
 
function runInParallel() {
  const workers = [
    new Worker("./cpu-intensive-task.js"),
    new Worker("./cpu-intensive-task.js"),
    new Worker("./cpu-intensive-task.js"),
  ];
  // Three tasks actually executing simultaneously on different cores
}

Concurrency is about dealing with multiple tasks. Parallelism is about executing multiple tasks simultaneously.

9.12 CPU Scheduling & Context Switching

The Linux scheduler (CFS - Completely Fair Scheduler) decides which runnable task gets CPU time. On a multi-core system, different threads can run simultaneously on different cores.

What the scheduler does:

1. Maintains a queue of runnable threads
2. Gives each thread a time slice (typically 1-10ms)
3. Switches between threads when slices expire
4. Handles priorities and CPU affinity
5. Balances load across multiple cores

Context switching - what happens:

Thread A Running          Thread B Running
    │                          │
    │ Timer interrupt          │
    ├──────────────────►       │
    │ Save A's state:          │
    │  ├── PC                  │
    │  ├── Registers           │
    │  └── Stack pointer       │
    │                          │
    │ Load B's state:          │
    │  ├── PC                  │
    │  ├── Registers           │
    │  └── Stack pointer       │
    │                          │
    │                    ◄─────┤
    │                          │ B continues
    │ A waits                  │

Why context switching is expensive:

• Saving/restoring CPU state takes time
• Cache misses (B's data isn't in cache)
• TLB flushes (B's memory mappings aren't loaded)

With thousands of switches per second, overhead adds up. This is why spawning 10,000 threads when you have 4 cores often reduces performance rather than improving it.

Check your system's scheduler:

# See context switch rates
vmstat 1
 
# See per-process context switches
pidstat -w -p <PID> 1
 
# See run queue length (how many threads want CPU)
uptime
# load average: 2.15, 1.98, 2.01

9.13 Inspecting Processes and Threads

Linux exposes everything through /proc. Here's your toolkit:

# Process basics
ps aux                          # All processes
ps -ef                          # All processes (different format)
top                             # Interactive process monitor
htop                            # Better top (if installed)
 
# Process tree
pstree -p                       # Show process hierarchy
ps --forest                     # Forest view
 
# Threads
ps -L -p <PID>                  # Threads of a process
ls /proc/<PID>/task             # Kernel's view of threads
 
# File descriptors
ls -l /proc/<PID>/fd            # Open FDs
 
# Memory
cat /proc/<PID>/maps            # Memory mappings
cat /proc/<PID>/status          # Full status
 
# CPU and scheduling
ps -o pid,pri,ni,psr,pcpu -p <PID>   # Priority, CPU, usage
chrt -p <PID>                        # Scheduling policy
taskset -p <PID>                     # CPU affinity
 
# Signals
cat /proc/<PID>/status | grep Sig  # Pending/blocked signals

From Node.js:

// Process info
console.log(process.pid);
console.log(process.ppid);
console.log(process.cwd());
console.log(process.memoryUsage());
console.log(process.cpuUsage());
console.log(process.resourceUsage()); // Node 12+
 
// Thread info (worker threads)
const { threadId } = require("worker_threads");
console.log(threadId); // Different for each thread

9.14 The Mental Model

                         Linux Kernel
                              │
          ┌───────────────────┼───────────────────┐
          │                   │                   │
      Process A           Process B           Process C
       PID 100              PID 200              PID 300
          │                   │                   │
     ┌────┴────┐              │              ┌────┴────┐
     │         │              │              │         │
 Thread 1   Thread 2       Thread 1        Thread 1   Thread 2
  TID 101   TID 102        TID 201         TID 301    TID 302
     │         │              │              │         │
     └────┬────┘              │              └────┬────┘
          │                   │                   │
          ▼                   ▼                   ▼
   ┌──────────────┐     ┌──────────────┐    ┌──────────────┐
   │ Shared       │     │ Process B's  │    │ Shared       │
   │ Address      │     │ Address      │    │ Address      │
   │ Space        │     │ Space        │    │ Space        │
   │ + Resources  │     │ + Resources  │    │ + Resources  │
   └──────────────┘     └──────────────┘    └──────────────┘

Key principles to remember:

Processes are isolated resource containers. Threads are execution units within containers.

Every process gets its own virtual address space - this is the fundamental isolation boundary.

Threads within a process share everything except stack, registers, and scheduling state.

Process creation is expensive (new address space); thread creation is cheaper (shared address space).

Concurrency is about structure; parallelism is about execution.

The scheduler decides which thread runs on which CPU core.

Context switching has real costs.

Zombie processes are dead children waiting for their parent to collect them - this matters in containers.