Roadmap

• Phase 1: Vectors – How AI stores data (like an Excel row)
  • What is a vector?
  • Magnitude (length)
  • Direction
  • Vector addition & subtraction
  • Scalar multiplication
  • Dot product ⭐ (the heart of AI comparisons)
  • Vector projection
  • Cosine similarity
• Phase 2: Matrices – How AI stores batches of data (like an Excel sheet)
  • Matrix operations
  • Matrix multiplication
  • Transpose
  • Identity matrix
  • Inverse matrix
• Phase 3: Linear Algebra – How AI transforms and squishes data
  • Linear transformations
  • Eigenvalues & eigenvectors
  • Basis
  • Rank
• Phase 4: Calculus – How AI learns from its mistakes
  • Derivatives
  • Partial derivatives
  • Gradients
  • Gradient descent
  • Chain rule
• Phase 5: Probability – How AI handles uncertainty and guesses
  • Probability
  • Bayes’ theorem
  • Mean, variance
  • Normal distribution
  • Maximum likelihood

Lesson 1: What is a Vector? (How AI Represents Information)

The Big Idea: A vector is just a list of numbers that describes something.

Imagine you’re building a system like ChatGPT. Humans understand words, cats, faces, and movies. Computers understand only numbers. So how do we make a computer “know” what a “cat” is?

We turn the cat into a list of numbers. That list is a vector.

This process of turning something real into a vector is called an embedding. You’ll hear “text embedding” or “image embedding” a lot in AI. (We’ll learn how embeddings are created later.)

The Graph Paper Analogy

Imagine you stand on a huge piece of graph paper at the point (0,0). You take 3 steps right and 4 steps up. The arrow from start to finish is a vector.

y
^
|
|        • (3,4)   <-- "3 right, 4 up"
|      /
|    /
|  /
|/
+-------------> x

A vector has two things:

• Magnitude – how long the arrow is.
• Direction – where the arrow points.

Why AI treats everything as a vector

Example: a person becomes a vector of their features:

Age = 25, Salary = 50000, Height = 175
Vector = [25, 50000, 175]

Dimensions Each value represents one feature. Three numbers → Three dimensions.

Computers don't understand “cat” or “movie”. They only understand numbers.

So AI converts everything into vectors:

Once something is a vector, you can measure how similar it is to another vector using simple arithmetic. That’s the whole game, and it’s why Lesson 2 (the dot product) matters so much.

Where Do These Numbers Come From?

Important: The vector for “cat” doesn’t come from a human writing down features. Initially, these numbers are random. Through a process called training (gradient descent, which we’ll learn later), the AI gradually adjusts them until they capture real meaning. So yes, the AI learns the numbers that make “cat” close to “kitten” and far from “spaceship”.

Magnitude (How Big Is the Vector?)

Every vector has two important properties: how far it goes (magnitude) and which way it points (direction).

Imagine walking: 2 steps east, 3 steps north. Your journey is the vector (2, 3). How far did you actually walk (the length of the arrow)?

Using the Pythagorean theorem:

Length = √(2²+3²) = √13 ≈ 3.61

That number is the magnitude. Notation: ||v||

Why AI Cares

Magnitude often represents how strong, how important, or how intense something is.

Example

Movie A = [2, 3, 1]
Movie B = [200, 300, 100]

Same preference pattern. Very different magnitude.

Direction

Two people walk northeast. One walks farther. Their magnitudes differ, but their directions are identical. AI often cares more about direction than distance.

Alice:  →  →
Bob:    →  →  →  →

Alice:  →  →
Bob:    ↑  ↑

Vector Addition

Vector addition combines movements. Walk east, then north – your final position is just both movements added together.

Morning walk  = (2, 1)
Afternoon walk = (1, 3)

Total movement = (3, 4)

Just add matching positions: [2,1] + [1,3] = [3,4]

AI Example

Combining features: Movie preference + User preference. Or Velocity + Acceleration.

Scalar Multiplication

Imagine zooming an image. Everything becomes twice as large, but nothing changes shape. Scalar multiplication stretches or shrinks a vector without changing its direction.

Multiply every value: [2,3] × 4 = [8,12]

Nothing changes direction. Everything becomes four times larger.

AI Example

Changing importance: Confidence × Feature, Learning Rate × Gradient, Attention Weight × Value Vector.

✅ Summary

• Vector = list of numbers; every number is a feature.
• AI turns everything into vectors (embeddings).
• Magnitude = length; direction = where it points.
• Addition combines movements; scalar multiplication stretches/shrinks.

⚠️ Common Beginner Mistakes

• Thinking of a vector as just a single number. It’s a whole list.
• Confusing “size” (magnitude) with “shape” (direction).

🧠 Quick Quiz

1. If you walk 3 steps east and 0 north, what is the vector?
2. What does the magnitude of that vector equal?
3. If you double that vector, what happens to its direction?

Answers: 1. [3,0]; 2. 3; 3. Direction stays the same.

📌 What’s Next?

Now that we can represent things as vectors, how do we compare them? That’s exactly what the dot product does.

Lesson 2: The Dot Product (The "Agreement" Score)

The Big Idea: The dot product measures how aligned two vectors are, while also being influenced by their size.

Imagine two people walking in a field.

• Walking the same direction → strongly aligned → dot product is large and positive
• Walking at a right angle to each other → no relationship → dot product is 0
• Walking opposite directions → dot product is negative

→  →          →  ↑          →  ←
large +       zero        negative

The dot product answers: “Do these two vectors agree with each other?” It appears everywhere: search engines, neural networks, transformers, recommendations.

⚠️ The Catch (Very Important): The dot product is magnified by size.

If a friend has the exact same taste in fruit as you, but they buy 10 times more of everything, their vector is huge. A raw dot product will score them as a much “better match” simply because of volume, not because their taste is any more similar.

To fix this, AI uses Cosine Similarity (Lesson 3), which shrinks all vectors to a standard length of 1 before comparing them.

The formula

For two 2D vectors A = (a₁, a₂) and B = (b₁, b₂):

Multiply matching positions, then add everything up.

Worked example:

A = (2, 3)
B = (4, 5)
 
2 × 4 = 8
3 × 5 = 15
8 + 15 = 23  → A·B = 23

Why 0 is special

A = (1, 0)
B = (0, 1)
1 × 0 + 0 × 1 = 0

A result of 0 means the vectors are perpendicular — completely unrelated in direction. Keep this in your head: dot product = 0 means "no relationship."

Code

// Javascript
 
function dotProduct(a, b) {
  let sum = 0;
  for (let i = 0; i < a.length; i++) {
    sum += a[i] * b[i];
  }
  return sum;
}
 
console.log(dotProduct([2, 3], [4, 5])); // 23

✅ Summary

• Dot product = how much two vectors point the same way (plus their size).
• Positive → same direction; zero → unrelated; negative → opposite.
• Formula: multiply matching positions, then sum.

⚠️ Common Beginner Mistakes

• Forgetting that larger magnitudes inflate the result.
• Thinking a negative dot product always means opposite (it could be an obtuse angle, not exactly 180°).

🧠 Quick Quiz

1. What is the dot product of [1, 2] and [2, 4]?
2. What does a dot product of 0 mean geometrically?
3. If two vectors point exactly the same way, is the dot product always positive?

Answers: 1. 1×2 + 2×4 = 10; 2. They are perpendicular; 3. Yes, and it will be positive (or zero if one vector is zero).

📌 What’s Next?

We need a way to compare only direction, ignoring size. That’s cosine similarity.

Lesson 3: Cosine Similarity (The "Direction" Match)

The Big Idea: Cosine similarity ignores how long the arrows are and looks only at the angle between them. It answers: "Are you pointing in the same direction as me?"

Imagine two people with identical movie taste. One has watched 100 movies, the other only 10. Their vectors are of different sizes, but the direction is the same. Cosine similarity says, “Their tastes are still identical.”

Its values range from:

• 1 → the same direction
• 0 → perpendicular (no alignment)
• −1 → exactly opposite directions

Formula

It’s the dot product divided by the product of the magnitudes. That division removes size.

Why AI lives and breathes this:

When you search Google or ask ChatGPT, your text becomes a vector. Cosine similarity is calculated between your query vector and millions of document vectors. The highest scores (closest to 1) are shown to you. This powers semantic search, recommendations, and memory in AI.

Example 1: Exact Same Direction

• A = (3, 4) (Length = 5)
• B = (6, 8) (Length = 10) (A doubled)

1. Dot Product = 50
2. Divide by lengths: 50 / (5 × 10) = 50 / 50 = 1

A score of 1.0 means "We point in the same direction!" (Very similar).

Example 2: Totally Different Direction

• A = (1, 0) (Points purely Right)
• B = (0, 1) (Points purely Up)

1. Dot Product = (1 × 0) + (0 × 1) = 0
2. Divide by lengths: 0 / (1 × 1) = 0

A score of 0 means "We are completely unrelated."

Code

// Javascript
 
function cosineSimilarity(a, b) {
  let dot = 0, magA = 0, magB = 0;
  for (let i = 0; i < a.length; i++) {
    dot += a[i] * b[i];
    magA += a[i] * a[i];
    magB += b[i] * b[i];
  }
  return dot / (Math.sqrt(magA) * Math.sqrt(magB));
}
console.log(cosineSimilarity([3, 4], [6, 8])); // 1

✅ Summary

• Cosine similarity = direction match, ignoring length.
• 1 = same, 0 = unrelated, -1 = opposite.
• It’s the dot product divided by both lengths.

⚠️ Common Beginner Mistakes

• Forgetting that identical vectors always give 1, even if they are scaled.
• Using cosine similarity when magnitude matters (e.g., comparing purchase amounts).

🧠 Quick Quiz

1. Cosine between [2,0] and [5,0]?
2. Cosine between [1,1] and [-1,1]?
3. Why is cosine similarity preferred over the dot product in semantic search?

Answers: 1. 1 (same direction); 2. 0 (they are perpendicular); 3. Because it ignores document length – a long document doesn’t automatically score higher than a short one if they talk about the same topic.

📌 What’s Next?

Now that we can compare directions, let’s ask: “How much of one vector points in a particular direction?” That’s vector projection.

Lesson 4: Vector Projection (Casting Shadows with Math)

The Core Question: Imagine pulling a heavy shopping cart. You pull the rope at an angle. Your friend asks: “Forget the angle, how much of your pull actually moves the cart straight forward?”

That’s exactly what vector projection answers: “How much of Vector A points in the direction of Vector B?”

The Shadow Analogy

Think of a stick and a flashlight.

• Stick = original vector.
• Shadow on the ground = projection onto the ground’s direction.
• Stick lying flat → shadow is full length. Projection = entire vector.
• Stick tilted → shadow shorter. Projection = part of the vector.
• Stick standing straight up → shadow is just a dot. Projection = zero.

Every vector can be broken into two pieces:

1. The projection (the part in the direction we care about).
2. The residual (everything else).

Example: A = (3, 4). If we only care about the Eastward direction (x-axis):

Projection = (3, 0) ← The “East-ness” of A

Residual = (0, 4) ← The “North-ness” of A

Notice they add back to A.

The Formula

The projection of A onto B is

It naturally breaks into three steps:

1. Measure alignment with A·B
2. Remove B’s length influence by dividing by B·B (which is ||B||²)
3. Turn the scalar back into a vector by multiplying by B

Sometimes we only need the length of the shadow (scalar projection). If we also need the direction, we multiply by B to get the vector projection.

Scalar Projection vs Vector Projection

Sometimes we only care about how long the shadow is. That is called the scalar projection. If we also want the direction, we multiply by B and get the vector projection.

Think of it like this:

Scalar Projection
↓

Length only

----------------------------

Vector Projection
↓

Length + Direction

Worked Example

Project A = (3, 4) onto B = (5, 0).

1. Dot product: A · B = (3 × 5) + (4 × 0) = 15
2. Length squared: B · B = (5 × 5) + (0 × 0) = 25
3. Scale factor: 15 / 25 = 0.6
4. Multiply: 0.6 × (5, 0) = (3, 0)

The projection is (3, 0). It removed the north part perfectly.

Why AI Cares

AI’s job is to find useful patterns hidden inside numbers. Projection helps isolate those patterns. Instead of looking at everything at once, AI can focus on just the part that matters.

• Focusing on one feature: A person vector [Height, Weight]. If we only care about height, project onto the “height” axis.
• Movie taste: Project a movie vector [Action, Comedy, Romance] onto the “Action” axis to see how action‑packed it is for an action fan.
• Reading handwriting: Project an image vector onto known digit patterns to ignore noise (pen thickness) and keep only the shape that defines a “3”.

✅ Summary

• Projection = how much of one vector lies in another’s direction.
• Splits a vector into a relevant part and the rest.
• Scalar projection gives just the length; vector projection gives the arrow.

⚠️ Common Beginner Mistakes

• Mixing up “project A onto B” with “project B onto A” – they’re different.
• Forgetting to divide by B·B; otherwise, the scale depends on B’s length.

🧠 Quick Quiz

1. What is the projection of [0, 5] onto the x‑axis [1, 0]?
2. If A and B are perpendicular, what is the projection of A onto B?
3. Why divide by B·B in the formula?

Answers: 1. [0,0]; 2. 0 (a zero vector); 3. To remove the influence of B’s length and keep only the direction effect.

📌 What’s Next?

Vectors describe one thing. To describe many things at once – like a whole dataset – we use matrices.

Lesson 5: What is a Matrix? (Storing Batches of Data)

The Big Idea: A vector describes one thing. A matrix describes many things at once; it's just a grid of vectors stacked together.

Remember the person vector:

Person = [25, 175, 50000]   (Age, Height, Salary)

That’s one person. AI learns from thousands, so we stack them into rows:

              Age   Height  Salary
Person 1  =  [ 25,   175,   50000 ]
Person 2  =  [ 31,   180,   62000 ]
Person 3  =  [ 19,   165,   28000 ]

That grid is a matrix – exactly like an Excel sheet:

• Rows = individual examples (people, images, sentences).
• Columns = features (age, height, salary).

Matrix Shape (Dimensions)

A matrix’s size is written as rows × columns.

The matrix above is 3 × 3 (3 people, 3 features).

1,000 people with 3 features → 1000 × 3 matrix.

Understanding “shape” is critical; most bugs in AI code are shape-mismatch errors.

Why AI Relies on Matrices

• One sentence → a vector
• A paragraph → a matrix (one row per sentence)
• A grayscale image → a matrix (pixel rows and columns)
• A batch of 32 images → a stack of matrices (tensors)

This is why GPUs are essential: they process huge matrices in parallel at incredible speeds.

Matrix Operations (Addition, Subtraction, Scalar Multiplication)

Good news: these work exactly like they did for vectors, position by position.

Addition / Subtraction

[1, 2] + [5, 1] = [6, 3]
[3, 4] + [0, 2] = [3, 6]

Rule: Matrices must have the exact same shape.

Scalar Multiplication

Multiply every single entry by the scalar

2  x  [1, 2]  =  [2, 4]
      [3, 4]  =  [6, 8]

AI Application

• Weights × Learning Rate to update a neural network.
• Image Matrix × Brightness Factor to alter pixel intensities.

Same idea as vectors: stretching or shrinking without changing the pattern.

✅ Summary

• Matrix = grid of vectors (rows = examples, columns = features).
• Shape = rows × columns.
• Addition/subtraction/scalar multiplication work position by position.

⚠️ Common Beginner Mistakes

• Forgetting that matrices must have identical shapes for addition.
• Confusing “3×3” (shape) with “9 elements” (total count).

🧠 Quick Quiz

1. If a dataset has 200 people and 5 features, what is the matrix shape?
2. Can you add a 2×3 matrix and a 3×2 matrix?
3. What happens to a matrix if you multiply it by a scalar > 1?

Answers: 1. 200×5; 2. No – shapes don’t match; 3. Every entry is multiplied; the pattern is preserved but stretched.

📌 What’s Next?

We can store data, but how do we combine information from two matrices? That’s matrix multiplication.

Lesson 6: Matrix Multiplication (Combining Information Grids)

The Big Idea: Matrix multiplication is taking every row of the first matrix and performing a dot product with every column of the second matrix. If you mastered the dot product, you already understand 90% of matrix multiplication.

Before we even think about how to multiply, we must answer: when can we multiply?

The Shape Rule (The Golden Rule)

You can only multiply A by B if:

A’s number of columns == B’s number of rows

A is (2 × 3)
B is (3 × 1)
     ↑match↑
Result is (2 × 1)   →  (A’s rows) × (B’s columns)

What happens when it fails?

A is (2 × 3) – 3 columns
B is (2 × 2) – 2 rows
3 ≠ 2  →  BROKEN. You can’t pair the third number with anything.

90% of runtime errors in deep learning are shape mismatches. Always check this first.

The Restaurant Analogy (now that we know shape is fine)

Matrix A (Customer orders):

              Burgers  Fries  Drinks
Customer 1  = [  2,      1,     1  ]
Customer 2  = [  1,      3,     2  ]

Matrix B (Menu prices):

Burgers = $5
Fries   = $2
Drinks  = $1

To get each customer's total bill, we perform a dot product between each row of A and the column of B:

Customer 1: (2×5) + (1×2) + (1×1) = 13
Customer 2: (1×5) + (3×2) + (2×1) = 13

Result = [13]
         [13]
     (shape 2×1)

Worked Example (2×2 by 2×2)

A = [1, 2]      B = [5, 6]
    [3, 4]          [7, 8]

• Top-left: (1 × 5) + (2 × 7) = 19
• Top-right: (1 × 6) + (2 × 8) = 22
• Bottom-left: (3 × 5) + (4 × 7) = 43
• Bottom-right: (3 × 6) + (4 × 8) = 50

Result = [19, 22]
         [43, 50]

⚠️ Order is Everything

Unlike regular numbers, A × B ≠ B × A.

See it for yourself:

A = [1, 2]      B = [0, 1]
    [0, 1]          [1, 0]

A × B = [2  1]      B × A = [3  4]
        [4  3]              [1  2]
 
A × B ≠ B × A

Totally different. In AI code, weights × input and input × weights are not interchangeable.

Why AI is Powered by This

Matrix multiplication is the most used operation in deep learning. When data flows through a neural network, every layer is essentially:

Output = (Input Matrix) × (Weight Matrix)

This happens billions of times per second.

Code

// Javascript
 
function matMul(A, B) {
  const rows = A.length,
    cols = B[0].length,
    inner = B.length;
  const result = Array.from({ length: rows }, () => Array(cols).fill(0));
 
  for (let i = 0; i < rows; i++) {
    for (let j = 0; j < cols; j++) {
      let sum = 0;
      for (let k = 0; k < inner; k++) {
        sum += A[i][k] * B[k][j];
      }
      result[i][j] = sum;
    }
  }
  return result;
}
 
console.log(
  matMul(
    [
      [1, 2],
      [3, 4],
    ],
    [
      [5, 6],
      [7, 8],
    ],
  ),
);
// [[19,22],[43,50]]

✅ Summary

• Multiply only when inner dimensions match.
• Result shape = rows of A × columns of B.
• Operation = dot product of rows of A with columns of B.
• Order matters: A×B ≠ B×A.

⚠️ Common Beginner Mistakes

• Forgetting to check shape compatibility first.
• Using element‑wise multiplication (like addition) instead of row‑column dot products.

🧠 Quick Quiz

1. Can you multiply a 3×2 matrix by a 2×4 matrix? What is the result shape?
2. In the restaurant example, if we add a third item, what happens to the shape of the price matrix?
3. Why is matrix multiplication so important in neural networks?

Answers: 1. Yes, result is 3×4; 2. It would become 3×1 (still compatible); 3. Because every layer computes a weighted sum of inputs, which is exactly a matrix multiplication.

📌 What’s Next?

Sometimes the data isn’t oriented the right way for multiplication. We need to flip the grid – that’s the transpose.

Lesson 7: Transpose (Flipping the Grid)

The Big Idea: The transpose, denoted Aᵀ, flips a matrix along its main diagonal. Every row becomes a column, and every column becomes a row.

The Notebook Analogy

Imagine your data written in a notebook, one row per person:

              Age  Height
Person 1  =  [ 25,  175 ]
Person 2  =  [ 31,  180 ]

Now turn the notebook sideways. Instead of "one row per person," you have "one row per feature":

          Person1  Person2
Age    =  [  25,     31  ]
Height =  [ 175,    180  ]

That’s the transpose.

Worked Example

A = [1, 2, 3]
    [4, 5, 6]

Aᵀ = [1, 4]
     [2, 5]
     [3, 6]

Rule: If A is shape (rows × columns), then Aᵀ is shape (columns × rows). A 2×3 matrix becomes a 3×2 matrix.

Why AI Cares

Often, two matrices have the right numbers but the wrong orientation. The transpose rotates the matrix so the shapes line up for multiplication. You’ll see .T or transpose() constantly in AI code. It’s rarely doing anything deep – it’s just “turning the notebook sideways” so the dot products work.

Code

// Javascript
 
function transpose(A) {
  return A[0].map((_, colIndex) => A.map((row) => row[colIndex]));
}
 
console.log(
  transpose([
    [1, 2, 3],
    [4, 5, 6],
  ]),
);
// [[1,4],[2,5],[3,6]]

✅ Summary

• Transpose flips rows ↔ columns.
• Shape becomes columns × rows.
• Main use: aligning dimensions for matrix multiplication.

⚠️ Common Beginner Mistakes

• Transposing twice gives back the original – but forgetting that and thinking the shape changed permanently.
• Assuming transpose changes the data values; it only rearranges.

🧠 Quick Quiz

1. What is the transpose of a 4×1 vector?
2. If A is 5×3, what is the shape of Aᵀ?
3. Why do we often compute Query × Keyᵀ in transformers?

Answers: 1. 1×4 row vector; 2. 3×5; 3. To align the dimensions so that each query can be compared with each key via dot products.

📌 What’s Next?

Before we learn to undo a transformation, we need to understand the “do nothing” transformation – the identity matrix.

Lesson 8: The Identity Matrix (The "Do Nothing" Operator)

The Big Idea: Why invent a matrix that does nothing? For the same reason, the number 1 exists: it’s the reference point for multiplication. Just as 5 × 1 = 5 leaves the number unchanged, multiplying by the identity matrix I leaves any matrix unchanged.

It’s a square grid of zeros, except for a diagonal line of ones:

I (3 × 3) = [1, 0, 0]
            [0, 1, 0]
            [0, 0, 1]

Worked Example

A = [4, 7]
    [2, 5]

I = [1, 0]
    [0, 1]

A × I = [4, 7]  (unchanged)
        [2, 5]

Why AI Cares

• It defines what “no transformation” looks like. This is crucial for checking if a transformation can be undone (inverse).
• Neural networks are sometimes initialized close to the identity, meaning they start by “doing nothing” and gradually learn useful distortions.
• It serves as the target for verifying that A × A⁻¹ = I.

Code

// Javascript
 
function identity(n) {
  return Array.from({ length: n }, (_, i) =>
    Array.from({ length: n }, (_, j) => (i === j ? 1 : 0)),
  );
}
 
console.log(identity(3));
// [[1,0,0],[0,1,0],[0,0,1]]

✅ Summary

• Identity matrix I = 1 on the diagonal, 0 elsewhere.
• Multiplying by I leaves a matrix unchanged.
• It’s the baseline for “no transformation” and for checking inverses.

⚠️ Common Beginner Mistakes

• Thinking the identity matrix is all ones (it’s only ones on the diagonal).
• Trying to use a non‑square identity with a non‑square matrix; identity must be square.

🧠 Quick Quiz

1. What is the 2×2 identity matrix?
2. If you multiply a 3×2 matrix by a 2×2 identity, what happens?
3. Why is the identity important for the concept of an inverse?

Answers: 1. [[1,0],[0,1]]; 2. The 3×2 matrix remains unchanged; 3. Because an inverse is defined by A × A⁻¹ = I.

📌 What’s Next?

If a matrix can be undone, we use the inverse matrix to do it.

Lesson 9: The Inverse Matrix (Undoing a Transformation)

The Big Idea: The inverse of a matrix A, written A⁻¹, completely undoes what A did. Multiply them, and you get the identity matrix I – back to “nothing changed.”

The Lock-and-Key Analogy

Matrix A      →  locks the door.
Inverse A⁻¹   →  the key that opens it.
A × A⁻¹ = I   →  back to original state.

Why AI Cares

Some classical methods solve for optimal weights directly using the inverse; no iterative guessing needed. If a transformation is invertible, it means no data was lost – a concept used in advanced architectures like normalizing flows. In deep learning, we usually don’t compute inverses (Gradient Descent is cheaper), but the idea of reversibility still matters for understanding data flow.

The Catch: When You Can’t Reverse (Determinants & Singular Matrices)

Some transformations destroy information, making them impossible to reverse. Imagine squashing a 3D object completely flat onto a table. Once flat, you can’t tell how tall it was. That squashing has no inverse.

A matrix without an inverse is called singular or non‑invertible.

The Determinant: a “Squash Detector”

For a 2×2 matrix:

A = [a, b]
    [c, d]

The determinant is ad − bc. It measures the area of the parallelogram formed by the rows.

• If the rows point in genuinely different directions, the area is non‑zero → invertible.

• If they point in the same (or opposite) direction, the area collapses to 0 → singular, no inverse.

  If the determinant = 0, you cannot divide by it, just as with numbers.

Worked Example (2×2 Inverse)

A = [4, 7]
    [2, 6]
 
Determinant = (4×6) − (7×2) = 10  (non‑zero, good)

The inverse formula for a 2×2 is:

A⁻¹ = 1/(ad−bc) × [ d, −b]
                  [−c,  a]

So:

A⁻¹ = 1/10 × [ 6, −7]  =  [ 0.6, −0.7]
             [−2,  4]     [−0.2,  0.4]

Check: A × A⁻¹ gives the identity.

Code

// Javascript
 
function inverse2x2(A) {
  const [[a, b], [c, d]] = A;
  const det = a * d - b * c;
  if (det === 0) throw "Singular matrix, no inverse.";
  const invDet = 1 / det;
  return [
    [ d * invDet, -b * invDet],
    [-c * invDet,  a * invDet]
  ];
}

✅ Summary

• A⁻¹ undoes A: A × A⁻¹ = I.
• Not all matrices have an inverse; those that don’t are singular.
• Determinant = 0 → singular (information lost).
• In AI, exact inverses are rare, but the concept underlies the understanding of loss of information.

⚠️ Common Beginner Mistakes

• Assuming every square matrix has an inverse (check the determinant first).
• Trying to find an inverse for non‑square matrices – not possible.

🧠 Quick Quiz

1. What is the inverse of a 2×2 matrix with determinant 0?
2. Why can’t a 2×3 matrix have an inverse?
3. If a neural network layer used an invertible weight matrix, what does that guarantee?

Answers: 1. Doesn’t exist; 2. Only square matrices can have an inverse; 3. No information is lost – the transformation can be reversed.

📌 What’s Next?

We now have the building blocks to understand how AI transforms entire spaces of data. In Phase 3, we’ll dive into linear transformations, eigenvalues, and basis – the mechanics behind feature engineering and dimensionality reduction.

Lesson 10: Linear Transformations (The Matrix Machine)

The Big Idea: A matrix isn’t just a grid of numbers – it’s a transformation machine. You feed it a vector, and it spits out a brand new vector in a predictable way.

Think of the matrix as a machine:

Input Vector
     │
     ▼
┌─────────────┐
│   Matrix A  │
└─────────────┘
     │
     ▼
Output Vector

Depending on the numbers inside the matrix, it can:

• 📏 Stretch a vector
• 📉 Shrink it
• 🔄 Rotate it
• 🪞 Reflect it
• ↗️ Shear it (slant it like a tilted bookshelf)

Example 1: Stretching (Double Size)

Matrix:

Input vector: $v=[3,4]$

Multiply row by row:

• First row: 2 × 3 + 0 × 4 = 6
• Second row: 0 × 3 + 2 × 4 = 8

Result: $[6,8]$

The direction is exactly the same (still pointing from the origin to that spot), but the length is doubled. The matrix stretched it.

Example 2: Shrinking (Half Size)

Matrix:

Input: $[6,8]$ → Output: $[3,4]$

Now it’s half as long. Same direction, different magnitude.

Example 3: Reflection (Flip Vertically)

Matrix:

Input: $[3,4]$ → Output: $[3,-4]$

The vector is flipped across the x‑axis, like a mirror reflection.

Why AI Cares

Imagine a sentence embedding "I love programming" represented as: [0.2, 0.8]

A neural network applies a weight matrix:

New embedding = Weight Matrix × Input embedding

The matrix transforms the embedding into a new representation.

For example:

• Layer 1 may learn grammar
• Layer 2 may learn meaning
• Layer 3 may learn sentiment
• The final layer may learn the answer to generate

The right sequence of transformations turns raw words into an understanding that can generate an answer.

Mental Model: Photoshop for Vectors

Think of editing a photo:

• Resize (stretch/shrink)
• Rotate
• Flip (reflect)
• Skew (shear)

Neural networks do the same operations, but on high‑dimensional vectors instead of pixels.

Code

// Javascript
 
function transform(matrix, vector) {
  return matrix.map((row) =>
    row.reduce((sum, val, i) => sum + val * vector[i], 0),
  );
}
 
const stretch = [
  [2, 0],
  [0, 2],
];
console.log(transform(stretch, [3, 4])); // [6, 8]

✅ Summary

• A matrix is a transformation that stretches, shrinks, rotates, reflects, or shears vectors.
• Multiplying matrix × vector applies that transformation.
• In AI, weight matrices transform data step by step through a network.

⚠️ Common Beginner Mistakes

• Thinking a translation (moving all points by +3) is a linear transformation – it moves the origin, so it’s not.
• Forgetting that the transformation applies to the entire space, not just your data.

🧠 Quick Quiz

1. If the matrix is [[1,0],[0,0]], what does it do to the vector [3,4]?
2. Is rotation a linear transformation? (Hint: does the origin move?)
3. Why is the identity matrix a “do‑nothing” transformation?

Answers: 1. It projects onto the x‑axis, giving [3,0] – the y component is destroyed. 2. Yes – rotation keeps the origin fixed and lines stay straight. 3. Because multiplying by I leaves the vector unchanged (like multiplying by 1).

Lesson 11: Eigenvalues & Eigenvectors (The Magic Directions)

The Big Idea: Most vectors get both rotated and stretched by a matrix. But some very special ones keep their direction – they only get stretched or shrunk. Those are eigenvectors. The amount they are stretched is the eigenvalue.

Imagine a rubber sheet with arrows drawn all over it. Pull the sheet:

• Most arrows change direction and length.
• A few arrows stay pointing exactly the same way – they just get longer or shorter. Those are the eigenvectors. Their scale factor is the eigenvalue.

The Simple Equation

For a matrix $A$, an eigenvector $v$ and eigenvalue $λ$ satisfy:

In words: Applying the matrix does nothing but multiply the vector by a number. Direction unchanged.

Worked Example

Matrix:

Take the vector:

Multiply:

The vector still points along the x-axis. It only became twice as long.

So:

• Eigenvector = (1,0)
• Eigenvalue = 2