ismile@portfolio:~$ cat notes/0001-compression.md
Compression
What is Compression?
Compression means reducing the size of data by encoding it using fewer bits (0s and 1s).
In simple words:
Compression reduces the amount of storage needed for a file.
A compressed file takes:
- Less disk space
- Less bandwidth during transfer
- Less time to send over the internet
Types of Compression
There are mainly two types of compression:
1. Lossless Compression
Compression without losing any information.
After decompressing, you get the exact original data back.
Example
If you compress a text file:
- The file size becomes smaller
- No words or characters are lost
- After decompression, the file remains exactly the same
Common Uses
- Text files
- Source code
- PDFs
- Databases
- ZIP files
Examples of Lossless Formats
- ZIP
- PNG
- GIF
- FLAC
Key Idea
Smaller size + No data loss
2. Lossy Compression
Compression, where some information is permanently removed to reduce the file size more aggressively.
The removed information is usually less noticeable to humans.
Example
In images or videos:
- Tiny color/details may be removed
- Human eyes may not easily notice the difference
- File size becomes much smaller
Common Uses
- Images
- Audio
- Videos
- Streaming platforms
Examples of Lossy Formats
- JPEG/JPG
- MP3
- MP4
- WebP
Key Idea
Much smaller size + Slight quality loss
Quick Comparison
| Feature | Lossless | Lossy |
|---|---|---|
| Data Loss | No | Yes |
| Original File Recovery | Exact | Not exact |
| File Size Reduction | Moderate | High |
| Best For | Text, code, documents | Images, audio, video |
| Quality Loss | None | Slight |
Real-Life Analogy
Lossless
Like folding clothes neatly into a suitcase.
- Same clothes
- Same quality
- Just organized efficiently
Lossy
Like summarizing a long movie into highlights.
- Some details are removed
- Main content remains
Important Note
Compression works because many files contain:
- Repeated patterns
- Redundant data
- Information humans may not notice easily
Compression algorithms detect and optimize these patterns.
Simple Definition (Short Note)
Compression is the process of reducing file size by encoding data using fewer bits.
- Lossless: No information is lost
- Lossy: Some information is removed for a higher compression ratio
Formats vs Algorithms (Important Distinction)
People often mix these up, so here's the clear difference:
The Basic Idea
Compression Algorithm = the actual method/logic used to shrink data (the "how")
File Format = a container/standard that may use one or more algorithms internally, plus some extra labeling info
Think of it like cooking:
Compression Algorithm = the recipe (the steps you follow to cook)
File Format = the packaged meal you get at the end (recipe + container + label)
A file format is often just an algorithm wrapped with extra structure (headers, metadata, file rules).
For Images (PNG, GIF)
Remember, the raw form of an image is just a plain grid of pixels (like BMP).
Now here's how that raw grid becomes a compressed format:
- Deflate = an algorithm (a method) that finds repeated patterns in the pixel grid and shrinks them
- PNG = takes those raw pixels, runs them through Deflate, then wraps the result with extra info (image size, color type, etc.) - this final wrapped package is called PNG
Raw pixels → [Deflate algorithm] → wrap with PNG info → PNG file
Similarly:
- LZW = a different algorithm, which works a bit differently than Deflate
- GIF = takes raw pixels, runs them through LZW, then wraps with GIF-specific info
Raw pixels → [LZW algorithm] → wrap with GIF info → GIF file
So PNG and GIF are like two different meals, but notice, they don't even use the same recipe (Deflate vs LZW). Different formats can use different algorithms.
For Audio (FLAC)
Remember, the raw form of audio is a plain sound recording (like WAV).
- FLAC's own algorithm = a special method designed just for compressing sound data
- FLAC (the format) = takes the raw WAV recording, runs it through this algorithm, then wraps it with audio info (sample rate, channels, etc.)
Raw sound (WAV) → [FLAC algorithm] → wrap with FLAC info → FLAC file
For General Files (ZIP)
- Deflate = the same algorithm used inside PNG is also used inside ZIP
- ZIP (the format) = takes any raw file, runs it through Deflate, then wraps it with a ZIP structure (file names, folder structure, etc.)
Any raw file → [Deflate algorithm] → wrap with ZIP structure → ZIP file
Examples
| Type | Name | What it is |
|---|---|---|
| Algorithm | Deflate | Combines LZ77 + Huffman coding |
| Algorithm | gzip | Wraps Deflate + adds header/checksum |
| Algorithm | Brotli (br) | Newer algorithm, better compression ratio than gzip |
| Format | ZIP | Container format - internally often uses Deflate |
| Format | PNG | Image format - internally uses Deflate |
| Format | GIF | Image format - uses LZW algorithm |
| Format | FLAC | Audio format - has its own custom lossless algorithm |
Why This Matters for HTTP Compression
This is exactly why gzip and deflate both appear as valid Accept-Encoding options in HTTP, they're algorithms/encoding methods meant for transferring raw data over the network, not file formats meant for long-term storage like ZIP or PNG.
| Context | Typical Choice | Why |
|---|---|---|
| Saving/sharing files | ZIP, PNG, FLAC (formats) | Includes structure, metadata, universal file support |
| Transferring data over HTTP | gzip, deflate, br (algorithms) | Lightweight, fast, applied on-the-fly, no need for full file structure |
HTTP Compression
HTTP compression is how browsers and servers use compression to make web responses smaller and faster to transfer.
How It Works (Step by Step)
-
Browser sends a request to the server, and includes a header:
Accept-Encoding: gzip, deflate, brThis tells the server: "Here are the compression algorithms I can understand."
-
Server checks that header and picks one of the supported algorithms (usually the best one available, like
brif listed). -
Server compresses the response body (HTML, CSS, JS, JSON, etc.) using that algorithm.
-
Server sends back the response with a header telling the browser what it used:
Content-Encoding: gzip -
Browser sees
Content-Encoding, decompresses the response automatically, and renders the page normally.
You never see this happening, it's all automatic between browser and server.
Common HTTP Compression Algorithms
| Algorithm | Notes |
|---|---|
| gzip | Oldest, most widely supported, good balance of speed/size |
| deflate | Similar to gzip, less commonly used today |
| br (Brotli) | Newer, made by Google, usually compresses better than gzip |
All three are lossless - text-based responses (HTML, CSS, JS, JSON) must decompress to the exact original content, or the page would break.
Why It Matters
- Reduces page load time
- Saves bandwidth for both server and user
- Especially useful for text-heavy responses (HTML/CSS/JS/JSON compress very well due to repeated patterns like tags, brackets, spaces)
Example Headers in Action
Request (from browser):
GET /api/data HTTP/1.1
Host: example.com
Accept-Encoding: gzip, deflate, br
Response (from server):
HTTP/1.1 200 OK
Content-Type: application/json
Content-Encoding: gzip
Content-Length: 1024
Here, the server chose gzip, and the original JSON might have been, say, 4 KB before compression now sent as ~1 KB.
Key Idea
Accept-Encoding= what the browser can handleContent-Encoding= what the server actually usedThis negotiation ensures compatibility, if a browser doesn't support
br, the server won't send Brotli-compressed data.
When Servers Don't Compress
Compression isn't always used. Sometimes the server just sends the file as-is, with no Content-Encoding at all. Let's see why.
The Basic Idea
Compression only helps when there's something "squeezable" left in the data - repeated patterns, redundant bits, etc.
If a file is already compressed, or too small to matter, squeezing it further wastes time for little or no benefit.
Case 1: Already-Compressed Files (JPEG, MP4, PNG, ZIP, etc.)
Remember, JPEG, MP4, and PNG are already compressed formats. Their redundant patterns have already been squeezed out once.
So when a server sends:
.jpg/.jpegimages.mp4/.webmvideos.mp3audio.png/.gifimages.zipfiles
It usually skips compression, because:
- Running gzip/br on them barely reduces size (sometimes even increases it slightly due to extra headers)
- It wastes CPU time on both server and browser for zero real benefit
Already compressed file → server sends it directly → no Content-Encoding header
Case 2: Very Small Responses
Compression itself isn't free; it adds:
- A small header/footer
- CPU time to compress (server) and decompress (browser)
For tiny responses (a few hundred bytes, like a small JSON reply or a short API response), the overhead of compression can be bigger than the savings.
So servers often set a minimum size threshold, for example, "only compress if response is bigger than ~1KB." Below that, they just send the raw response.
Very small response → compression overhead > savings → server skips it
Simple Table
| Situation | Compress? | Why |
|---|---|---|
| Large HTML/CSS/JS/JSON | ✅ Yes | Lots of repeated patterns to squeeze |
| Already-compressed image (JPEG, PNG, WebP) | ❌ No | Already squeezed, nothing left to save |
| Already-compressed video/audio (MP4, MP3) | ❌ No | Same reason, already compact |
| Tiny response (few bytes) | ❌ No | Compression overhead costs more than it saves |
| Large plain text/API response | ✅ Yes | Text compresses very well |
Minification vs Compression
These two are often confused because both make files "smaller," but they operate at different stages and use completely different techniques.
Core Distinction
Minification = removing unnecessary human-readable characters from source code (whitespace, comments, long variable names), but the code structure/syntax remains unchanged and still readable by the interpreter.
Compression = encoding the entire byte stream (already-minified or not) using an algorithm that exploits redundancy, the output is binary and not directly readable/executable until decompressed.
In short:
- Minification works before the file is compressed, at the source-code level
- Compression works after, at the byte/binary level, regardless of what the content is
Minification
Applies mainly to text-based source files: JS, CSS, HTML.
What it removes/changes:
- Whitespace, indentation, line breaks
- Comments
- Long/descriptive variable and function names → shortened to single letters
- Redundant syntax (e.g., unnecessary semicolons, quotes)
Example:
Before (readable JS):
function calculateTotalPrice(price, taxRate) {
// Calculate the total price including tax
const total = price + price * taxRate;
return total;
}After minification:
function calculateTotalPrice(a, b) {
const c = a + a * b;
return c;
}Key point: This is still valid, executable JavaScript. The browser/interpreter can run it exactly as-is, no "decompression" step needed. It's just been stripped of anything that was only useful for human readability.
Why Both Are Used Together
They're complementary, not competing techniques; that's why production web apps apply both, in this order:
Source code → Minify → Minified source → Compress (gzip/br) → Sent over network
↓
Browser decompresses
↓
Runs the minified (but still valid) code
- Minification reduces size by removing dead weight meant only for humans (comments, whitespace, naming)
- Compression reduces size further by exploiting statistical redundancy in the byte stream — including redundancy minification couldn't touch (e.g., repeated keywords like
function,return,const)