ismile@portfolio:~$ cat notes/0005-linux-for-backend-engineers-part-4.md
Linux for Backend Engineers Part 4
Linux for Backend Engineers
- Part 1: The Foundation
- Part 2: Files, Filesystems, and Permissions
- Part 3: Processes and Threads
- Part 4: Memory Management ← current
- Part 5: Signals and Communication
- Part 6: IPC, Networking, and Containers
10. Virtual Memory: How Processes See Memory
In Section 9, we saw that every process has its own virtual address space. But your computer has a limited amount of physical RAM. How can every process behave as if it has its own large, private memory?
The answer is virtual memory - one of the most important abstractions in modern computing.
Virtual memory creates the illusion that each process has a large, private, continuous address space, while the kernel and CPU manage how those addresses map to physical memory.
This abstraction is so fundamental that it enables:
- Process isolation - one process can't corrupt another's memory
- Efficient memory use - physical RAM is shared among processes
- On-demand loading - programs only load what they actually use
- Memory protection - code can't accidentally execute data
- Shared libraries - one copy of libc serves all processes
10.1 The Illusion of Private Memory
Every process believes it owns the entire memory space:
Process A's View Process B's View
┌─────────────────────┐ ┌─────────────────────┐
│ 0x0000000000000000 │ │ 0x0000000000000000 │
│ │ │ │
│ Code (text) │ │ Code (text) │
│ Data │ │ Data │
│ Heap │ │ Heap │
│ ↓ │ │ ↓ │
│ (grows down) │ │ (grows down) │
│ │ │ │
│ (grows up) │ │ (grows up) │
│ ↑ │ │ ↑ │
│ Stack │ │ Stack │
│ │ │ │
│ 0x7FFFFFFFFFFF │ │ 0x7FFFFFFFFFFF │
└─────────────────────┘ └─────────────────────┘
Both processes think they have the same addresses. But they don't. The addresses are virtual. They map to different physical locations.
10.2 Virtual Memory vs Physical Memory
There are two different things:
Virtual Memory (What Processes See)
The addresses that a process uses. Virtual memory is the address space that a process uses. Each process has its own virtual address space, which is mapped to physical memory by the operating system and the CPU's memory-management hardware.
On a 64-bit system, the theoretical address space can be extremely large, although the actual usable address range depends on the CPU and operating system.
Virtual Address Space (per process)
├── Code segment (program instructions)
├── Data segment (global variables)
├── Heap (dynamic allocation, grows up)
├── Memory-mapped regions (libraries, files)
├── Stack (function calls, grows down)
└── Kernel space (mapped but protected)
Physical Memory (Actual Hardware)
The real RAM installed in the machine. Typically 8GB, 16GB, 32GB, etc.
Physical RAM (shared by all processes)
┌──────────────────┐
│ Frame 0 (4KB) │
├──────────────────┤
│ Frame 1 (4KB) │
├──────────────────┤
│ Frame 2 (4KB) │
├──────────────────┤
│ Frame 3 (4KB) │
├──────────────────┤
│ ... │
└──────────────────┘
The Mapping
The kernel maintains a mapping between virtual pages and physical frames:
Process A Virtual Pages Physical RAM
┌──────────────┐
│ Page 0 │───────────────► Frame 7
├──────────────┤
│ Page 1 │───────────────► Frame 12
├──────────────┤
│ Page 2 │───────────────► Frame 2
├──────────────┤
│ Page 3 │───────────────► Frame 15
└──────────────┘
Process B Virtual Pages Physical RAM
┌──────────────┐
│ Page 0 │───────────────► Frame 20
├──────────────┤
│ Page 1 │───────────────► Frame 23
├──────────────┤
│ Page 2 │───────────────► Frame 8
├──────────────┤
│ Page 3 │───────────────► Frame 30
└──────────────┘Virtual pages do not need to be stored next to each other in physical RAM. This allows the kernel to use physical memory much more flexibly.
A process does not normally access physical RAM directly. Instead:
Program
│
│ virtual address
▼
CPU / MMU
│
│ page-table translation
▼
Physical RAM
The MMU (Memory Management Unit) is hardware that performs these address translations automatically.
10.3 Pages and Frames
Virtual memory is divided into fixed-size blocks called pages. Physical memory is divided into corresponding blocks called frames.
The standard page size on Linux is 4 KB (4096 bytes), though Linux also supports:
| Page Size | Used For | Example |
|---|---|---|
| 4 KB | Default for most processes | Regular application memory |
| 2 MB | Huge pages | Databases, large allocations |
| 1 GB | Gigantic pages | Specialized workloads |
Check your system's page size:
getconf PAGE_SIZE
# 4096 (4KB)
# Or
getconf PAGESIZE
# 409610.4 Page Tables: The Translation Mechanism
How does the CPU know where a virtual page actually lives in physical memory? The kernel maintains page tables, data structures that map virtual pages to physical frames.
Simplified View:
Virtual Address
│
▼
┌─────────────────────────┐
│ Page Table │
├─────────────────────────┤
│ Page 0 → Frame 7 │
│ Page 1 → Frame 2 │
│ Page 2 → Frame 15 │
│ Page 3 → Frame 4 │
│ Page 4 → Not present │
│ Page 5 → Frame 21 │
└─────────────────────────┘
│
▼
Physical Frame
The CPU's MMU uses these mappings to translate virtual addresses into physical addresses. Modern CPUs use multi-level page tables rather than one giant flat table.
Check a process's page tables:
# See memory mappings
cat /proc/<PID>/maps
# See detailed memory statistics
cat /proc/<PID>/smaps10.5 The MMU: Hardware Address Translation
The Memory Management Unit (MMU) is hardware that performs address translation automatically:
Application Accesses Memory
│
│ virtual address (e.g., 0x7ffe12345678)
▼
┌─────────────────┐
│ MMU │
└─────────────────┘
│
│ Look up in TLB (cache)
│
├── Hit → Get physical address immediately
│
└── Miss → Walk page tables
│
▼
Physical address (e.g., 0x1a2b3c4d5e6f)
│
▼
Physical RAMThis happens for every memory access, every variable read, every function call, every array access.
10.6 TLB: Making Address Translation Fast
Translating every memory access through page tables would be expensive. CPUs therefore use a cache called the TLB (Translation Lookaside Buffer) that stores recent translations.
Virtual Address
│
▼
┌─────────┐
│ TLB │
└─────────┘
│
├── Hit (~1 cycle) → Physical address directly
│
└── Miss (~100+ cycles) → Walk page tables
If the translation is already in the TLB, the CPU can avoid walking the page tables.
10.7 Page Faults: When Translation Fails
What happens when a process accesses a virtual address that isn't mapped to physical memory? The CPU raises a page fault, handing control to the kernel.
Page faults are normal. They're part of how virtual memory works:
Program accesses virtual address
│
▼
Is page mapped?
/ \
Yes No
│ │
▼ ▼
Access Page Fault
memory │
▼
Kernel handles it
│
┌───────────┼───────────┐
│ │ │
Invalid Not Swapped
access loaded out
│ │ │
▼ ▼ ▼
SIGSEGV Load from Read from
(crash) disk/file swap
│ │
▼ ▼
Map page Bring to RAM
│ │
└─────┬─────┘
▼
Resume process
The kernel may need to:
- Establish a new mapping
- Load data from a file
- Bring a swapped-out page back into memory
- Reject the access if it violates the memory permissions
10.8 Minor vs Major Page Fault
Not all page faults require disk I/O.
- Minor Page Fault: The required data is already available in memory, but the process's page tables need to be updated, or a mapping needs to be established.
- Major Page Fault: The required page is not currently available in physical memory, so the kernel must perform storage I/O to load the page into RAM before the process can continue. This can happen when a page has been swapped out or when a required file-backed page is not yet in memory.
Page Fault
│
├── Minor → page available in RAM
│ └── no storage I/O required
│
└── Major → page not available in RAM
└── storage I/O requiredThis distinction matters when diagnosing memory and application performance.
10.9 Copy-on-Write: Efficient Process Creation
Remember from Section 9 that fork() creates a child process. Does Linux copy all memory immediately? No, it uses copy-on-write (COW).
Initial State: Shared Memory
Parent Process Physical RAM
┌──────────────┐ ┌──────────────┐
│ Page A │──┐ │ Frame 7 │
│ Page B │──┼─────────────►│ Frame 12 │
│ Page C │──┘ │ Frame 15 │
└──────────────┘ └──────────────┘
▲
Child Process (after fork) │
┌──────────────┐ │
│ Page A │──┐ │
│ Page B │──┼─────────────────────┘
│ Page C │──┘ (all marked read-only)
└──────────────┘
Both processes initially share the same physical pages. The pages are marked read-only to detect writes.
When Someone Writes:
Parent Process writes to Page B:
Parent Process Physical RAM
┌──────────────┐ ┌──────────────┐
│ Page A │────────────────►│ Frame 7 │
│ Page B │──┐ │ Frame 12 │◄─────────┐
│ Page C │──┼─────────────►│ Frame 15 │ │
└──────────────┘ └─────────────►│ Frame 20 │◄── New │
└──────────────┘ Page │
│
│
│
Child Process (untouched) │
┌──────────────┐ │
│ Page A │───────────────────────────────────────────┤
│ Page B │───────────────────────────────────────────┤
│ Page C │───────────────────────────────────────────┘
└──────────────┘
Original Frame 12 → still shared by Child's Page B
New Frame 20 → Parent's modified Page B
Only the modified page gets copied. Everything else remains shared.
10.10 Shared Memory: Deliberate Sharing
Virtual memory also allows processes to deliberately share memory:
Process A ──────┐
├────► Physical Page
Process B ──────┘
Both processes can map the same physical memory into their virtual address spaces. This can be extremely fast for IPC because the data doesn't need to be copied through a traditional pipe or socket for every exchange.
But shared memory introduces synchronization problems. Two threads/processes modifying the same data simultaneously can cause race conditions.
That's where mechanisms such as:
- Mutexes
- Semaphores
- Atomics
become important.
10.11 Memory-Mapped Files
Linux can map a file directly into a process's virtual address space using mmap(). The file becomes accessible like memory:
File on Disk Process Virtual Memory
┌──────────────┐ ┌──────────────┐
│ 0-4KB │──┐ │ Page 0 │──┐
│ 4-8KB │ ├───────────►│ Page 1 │ ├── File content
│ 8-12KB │ │ │ Page 2 │ │ accessible as
│ 12-16KB │──┘ │ Page 3 │──┘ memory
└──────────────┘ └──────────────┘
│
▼
Read/write triggers
automatic file I/O
Benefits:
- No explicit read/write calls needed
- Operating system handles caching
- Multiple processes can map the same file (shared access)
- Efficient for large files
Memory mapping is used extensively by operating systems, databases, runtimes, and high-performance applications.
10.12 Swap: When RAM Runs Out
RAM is limited. When physical memory is exhausted, Linux can move less-used pages to disk, this is swap.
Physical RAM (8GB) Swap Space (4GB on disk)
┌──────────────┐ ┌──────────────┐
│ Active pages │ │ Swapped page │
│ Active pages │ │ Swapped page │
│ Active pages │ │ Swapped page │
│ Less-used pg │──► evicted ────►│ Swapped page │
│ Less-used pg │──► evicted ────►│ Swapped page │
└──────────────┘ └──────────────┘
When a swapped page is needed:
Process accesses swapped page
│
▼
Page fault (major)
│
▼
Read from swap disk
│
▼
Bring back to RAM
│
▼
Resume process
(100+ ms later!)
Storage is dramatically slower than RAM, so heavy swapping can cause severe performance degradation.
This is commonly called thrashing when the system spends excessive time moving pages rather than doing useful work.
10.13 Memory Protection
Virtual memory isn't only about addresses. It's also about protection. Page-table entries also contain permissions. A page can be marked approximately as:
Read
Write
Execute
Typical permissions:
Memory Region Permissions
───────────── ─────────────
Code (text) Read + Execute
Read-only data Read
Data Read + Write
Heap Read + Write
Stack Read + Write
This helps prevent a process from:
- Executing memory that should only contain data
- Writing to read-only memory
- Accessing unmapped memory (Segfaults)
10.13 The OOM Killer: When Memory Runs Out
When both RAM and swap are exhausted, the kernel must do something drastic. It invokes the Out-Of-Memory (OOM) Killer, which terminates a process to free memory.
How it chooses a victim:
Score = Memory usage × OOM score factorProcesses with higher scores are killed first:
# Check OOM scores
cat /proc/<PID>/oom_score
# Check OOM adjustment (-1000 to 1000)
cat /proc/<PID>/oom_score_adj
# Negative = protected, Positive = more likely to be killed10.14 Inspecting Memory on Linux
Everything is exposed through /proc. Here's your toolkit:
# Overall memory info
cat /proc/meminfo
# MemTotal, MemFree, SwapTotal, etc.
# Process memory details
cat /proc/<PID>/status
# VmSize (virtual), VmRSS (resident), VmData, VmStk, VmExe
# Memory mappings
cat /proc/<PID>/maps
# Shows every virtual memory region with permissions
# Detailed per-mapping stats
cat /proc/<PID>/smaps
# Shows RSS, swap, and more for each mapping
# Page faults
ps -o min_flt,maj_flt -p <PID>10.15 Node.js and Memory
Node.js runs inside a process with its own virtual address space. Inside that process, V8 manages JavaScript memory, including the JavaScript heap.
A simplified model:
Node.js Process
│
├── V8
│ ├── JavaScript Heap
│ └── Garbage Collector
│
├── Native Memory
│
├── Stack(s)
│
├── Shared Libraries
│
└── Memory-mapped regions
This is why these concepts matter when diagnosing Node.js applications.
For example:
ps -o pid,rss,vsz,cmd -p <PID>can help you inspect process-level memory usage.
And inside Node:
console.log(process.memoryUsage());can show runtime memory statistics.
Keep in mind that V8 heap usage and total process memory are not the same thing.
A Node process can consume memory outside the JavaScript heap through:
- Native allocations
- Buffers
- Shared libraries
- Memory mappings
- Thread stacks
- Other runtime resources
Try It Yourself: Virtual vs Resident Memory
// Demonstrate demand paging
const buf = Buffer.alloc(100 * 1024 * 1024); // 100MB virtual
console.log("After alloc:", process.memoryUsage().rss / 1024 / 1024, "MB RSS");
// Touch every page
for (let i = 0; i < buf.length; i += 4096) {
buf[i] = 1;
}
console.log("After touch:", process.memoryUsage().rss / 1024 / 1024, "MB RSS");10.16 The Mental Model
Put everything together:
Process Virtual Memory Physical Memory
┌──────────────┐
│ Page 0 (Code)│ ───────────────┐
├──────────────┤ │
│ Page 1 (Data)│ ───────────┐ │
├──────────────┤ │ │
│ Page 2 (Heap)│ ───────┐ │ │
├──────────────┤ │ │ │
│Page 3 (Stack)│ ───┐ │ │ │
└──────────────┘ │ │ │ │
▼ ▼ ▼ ▼
┌────────────────────┐
│ Page Table │
├────────────────────┤
│ 0 → Frame 82 │
│ 1 → Frame 12 │
│ 2 → Frame 44 │
│ 3 → Frame 19 │
└────────────────────┘
│
▼
┌───────────────────────┐
│ Frame 12 (Data) │
├───────────────────────┤
│ Frame 19 (Stack) │
├───────────────────────┤
│ Frame 44 (Heap) │
├───────────────────────┤
│ Frame 82 (Code) │
└───────────────────────┘ Process
│
Virtual Address Space
│
┌────────────┼────────────┐
│ │ │
Code Heap Stack
│ │ │
└────────────┼────────────┘
│
Virtual Addresses
│
▼
MMU
│
Page Tables
│
▼
Physical Memory
│
┌──────────┴──────────┐
│ │
RAM Swap
Key principles to remember:
Processes use virtual addresses, not physical addresses directly.
The MMU and page tables translate virtual addresses to physical memory.
Memory is managed in pages (typically 4KB), while physical RAM is divided into frames.
The TLB caches recent translations memory.
Page faults are normal - minor faults are fast, major faults involve disk I/O.
Copy-on-write makes fork() efficient by sharing physical pages until modified.
Shared memory allows deliberate sharing between processes.
Memory-mapped files turn file I/O into memory access.
Swap extends RAM using disk, but excessive swapping causes thrashing.
The OOM killer terminates processes when both RAM and swap are exhausted.