Linear Algebra Formulas in LaTeX, Python and Rust

✅ Copy-ready code for linear algebra formulas in LaTeX, Python and Rust. For each formula, you'll see how to write it in LaTeX and how to calculate it in code. 😻

Every snippet runs on its own with no extra libraries, and the expected output is written in the comments. Use the tabs to switch between LaTeX, Python and Rust. To run a Rust snippet, save it as main.rs and run rustc main.rs, or paste it into the Rust Playground.

📚 Math Formulas in Code series

• Part 1: Algebra Formulas: exponents, logarithms, quadratics, lines and sequences
• Part 2: Linear Algebra Formulas (you are here): vectors, matrices and a neural network layer
• Part 3: Descriptive Statistics Formulas: mean, median, spread and correlation
• Part 4: Inferential Statistics Formulas: tests, confidence intervals, probability and regression
• Part 5: Calculus Formulas: derivatives, integrals and numerical methods

Just click the copy button in the top right corner of each code block!

Summary Overview

#	Formula	Example
1	Vector Addition and Scalar Multiplication	$[1,2]+[3,4]=[4,6]$
2	Dot Product	$[1,2,3]\cdot [4,5,6]=32$
3	Vector Norms	$\Vert [3,4]{\Vert }_2=5$, $\Vert [3,-4]{\Vert }_1=7$
4	Cosine Similarity	$[1,0]$ and $[1,1]\to 0.7071$
5	Matrix Addition	two $2\times 2$ matrices
6	Transpose	$2\times 3\to 3\times 2$
7	Matrix Multiplication	result $\left[\begin{array}{cc}19& 22\\ 43& 50\end{array}\right]$
8	Determinant of a 2 × 2 Matrix	$\left|\begin{array}{cc}1& 2\\ 3& 4\end{array}\right|=-2$
9	Linear Layer	$[3.5,6.5]$

1. Vector Addition and Scalar Multiplication

Vector Addition and Scalar Multiplication

Formula:

$$\mathbf{u}+\mathbf{v}=\left[\begin{array}{c}{u}_1+v_1\\ u_2+v_2\end{array}\right]c\mathbf{v}=\left[\begin{array}{c}c{v}_1\\ c{v}_2\end{array}\right]$$

In Python, a list stands in for a vector. In Rust, the code uses Vec<f64>.

Where it's used: combining and rescaling vectors, such as updating weights in gradient descent.

LaTeX

$$
\mathbf{u} + \mathbf{v} = \begin{bmatrix} u_1 + v_1 \\ u_2 + v_2 \end{bmatrix} \qquad c \, \mathbf{v} = \begin{bmatrix} c \, v_1 \\ c \, v_2 \end{bmatrix}
$$

Python

def add(u: list[float], v: list[float]) -> list[float]:
    return [a + b for a, b in zip(u, v)]

def scale(c: float, v: list[float]) -> list[float]:
    return [c * a for a in v]

print(add([1, 2], [3, 4]))  # [4, 6]
print(scale(2, [1, 2]))     # [2, 4]

Rust

fn add(u: &[f64], v: &[f64]) -> Vec<f64> {
    u.iter().zip(v).map(|(a, b)| a + b).collect()
}

fn scale(c: f64, v: &[f64]) -> Vec<f64> {
    v.iter().map(|a| c * a).collect()
}

fn main() {
    println!("{:?}", add(&[1.0, 2.0], &[3.0, 4.0])); // [4.0, 6.0]
    println!("{:?}", scale(2.0, &[1.0, 2.0])); // [2.0, 4.0]
}

2. Dot Product

Dot Product

Formula:

$$\mathbf{u}\cdot \mathbf{v}=\sum _{i=1}^n{u}_i{v}_i$$

Where it's used: the weighted sum inside every neuron, and measuring similarity.

LaTeX

$$
\mathbf{u} \cdot \mathbf{v} = \sum_{i=1}^{n} u_i v_i
$$

Python

def dot(u: list[float], v: list[float]) -> float:
    return sum(a * b for a, b in zip(u, v))

print(dot([1, 2, 3], [4, 5, 6]))  # 32

Rust

fn dot(u: &[f64], v: &[f64]) -> f64 {
    u.iter().zip(v).map(|(a, b)| a * b).sum()
}

fn main() {
    println!("{}", dot(&[1.0, 2.0, 3.0], &[4.0, 5.0, 6.0])); // 32
}

3. Vector Norms

Vector Norms

Formula:

$$\Vert \mathbf{x}{\Vert }_2=\sqrt{\sum _{i=1}^n{x}_i^2}\Vert \mathbf{x}{\Vert }_1=\sum _{i=1}^n|x_i|$$

Where it's used: the length of a vector, distances, and L1 and L2 regularisation.

LaTeX

$$
\lVert \mathbf{x} \rVert_2 = \sqrt{\sum_{i=1}^{n} x_i^2} \qquad \lVert \mathbf{x} \rVert_1 = \sum_{i=1}^{n} \lvert x_i \rvert
$$

Python

import math

def l2_norm(x: list[float]) -> float:
    return math.sqrt(sum(a * a for a in x))

def l1_norm(x: list[float]) -> float:
    return sum(abs(a) for a in x)

print(l2_norm([3, 4]))  # 5.0
print(l1_norm([3, -4]))  # 7

Rust

fn l2_norm(x: &[f64]) -> f64 {
    x.iter().map(|a| a * a).sum::<f64>().sqrt()
}

fn l1_norm(x: &[f64]) -> f64 {
    x.iter().map(|a| a.abs()).sum()
}

fn main() {
    println!("{}", l2_norm(&[3.0, 4.0])); // 5
    println!("{}", l1_norm(&[3.0, -4.0])); // 7
}

4. Cosine Similarity

Cosine Similarity

Formula:

$$\cos \theta =\frac{\mathbf{u}\cdot \mathbf{v}}{\Vert \mathbf{u}\Vert \Vert \mathbf{v}\Vert }$$

The result is 1 for vectors pointing the same way, 0 for perpendicular vectors, and -1 for opposite directions.

Where it's used: comparing word embeddings, documents and user preferences in recommender systems.

LaTeX

$$
\cos \theta = \frac{\mathbf{u} \cdot \mathbf{v}}{\lVert \mathbf{u} \rVert \, \lVert \mathbf{v} \rVert}
$$

Python

import math

def cosine_similarity(u: list[float], v: list[float]) -> float:
    dot = sum(a * b for a, b in zip(u, v))
    norm_u = math.sqrt(sum(a * a for a in u))
    norm_v = math.sqrt(sum(b * b for b in v))
    return dot / (norm_u * norm_v)

print(round(cosine_similarity([1, 0], [1, 1]), 4))  # 0.7071

Rust

fn cosine_similarity(u: &[f64], v: &[f64]) -> f64 {
    let dot: f64 = u.iter().zip(v).map(|(a, b)| a * b).sum();
    let norm_u = u.iter().map(|a| a * a).sum::<f64>().sqrt();
    let norm_v = v.iter().map(|b| b * b).sum::<f64>().sqrt();
    dot / (norm_u * norm_v)
}

fn main() {
    println!("{:.4}", cosine_similarity(&[1.0, 0.0], &[1.0, 1.0])); // 0.7071
}

5. Matrix Addition

Matrix Addition

Formula:

$$\left[\begin{array}{cc}1& 2\\ 3& 4\end{array}\right]+\left[\begin{array}{cc}5& 6\\ 7& 8\end{array}\right]=\left[\begin{array}{cc}6& 8\\ 10& 12\end{array}\right]$$

A matrix is stored as a list of rows. Both matrices must be the same size.

Where it's used: combining weight matrices and datasets.

LaTeX

$$
\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} + \begin{bmatrix} 5 & 6 \\ 7 & 8 \end{bmatrix} = \begin{bmatrix} 6 & 8 \\ 10 & 12 \end{bmatrix}
$$

Python

def mat_add(A: list[list[float]], B: list[list[float]]) -> list[list[float]]:
    return [[a + b for a, b in zip(row_a, row_b)] for row_a, row_b in zip(A, B)]

print(mat_add([[1, 2], [3, 4]], [[5, 6], [7, 8]]))  # [[6, 8], [10, 12]]

Rust

fn mat_add(a: &[Vec<f64>], b: &[Vec<f64>]) -> Vec<Vec<f64>> {
    a.iter()
        .zip(b)
        .map(|(ra, rb)| ra.iter().zip(rb).map(|(x, y)| x + y).collect())
        .collect()
}

fn main() {
    let a = vec![vec![1.0, 2.0], vec![3.0, 4.0]];
    let b = vec![vec![5.0, 6.0], vec![7.0, 8.0]];
    println!("{:?}", mat_add(&a, &b)); // [[6.0, 8.0], [10.0, 12.0]]
}

6. Transpose

Transpose

Formula:

$$A=\left[\begin{array}{ccc}1& 2& 3\\ 4& 5& 6\end{array}\right]\Rightarrow A^{\mathrm{\top }}=\left[\begin{array}{cc}1& 4\\ 2& 5\\ 3& 6\end{array}\right]$$

Rows become columns, so a $2\times 3$ matrix becomes $3\times 2$.

Where it's used: the normal equation, covariance matrices and backpropagation.

LaTeX

$$
A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix} \quad \Rightarrow \quad A^{\top} = \begin{bmatrix} 1 & 4 \\ 2 & 5 \\ 3 & 6 \end{bmatrix}
$$

Python

def transpose(A: list[list[float]]) -> list[list[float]]:
    return [list(col) for col in zip(*A)]

print(transpose([[1, 2, 3], [4, 5, 6]]))  # [[1, 4], [2, 5], [3, 6]]

Rust

fn transpose(a: &[Vec<f64>]) -> Vec<Vec<f64>> {
    (0..a[0].len())
        .map(|j| a.iter().map(|row| row[j]).collect())
        .collect()
}

fn main() {
    let a = vec![vec![1.0, 2.0, 3.0], vec![4.0, 5.0, 6.0]];
    println!("{:?}", transpose(&a)); // [[1.0, 4.0], [2.0, 5.0], [3.0, 6.0]]
}

7. Matrix Multiplication

Matrix Multiplication

Formula:

$$(AB{)}_{ij}=\sum _{k=1}^n{a}_{ik}{b}_{kj}\left[\begin{array}{cc}1& 2\\ 3& 4\end{array}\right]\left[\begin{array}{cc}5& 6\\ 7& 8\end{array}\right]=\left[\begin{array}{cc}19& 22\\ 43& 50\end{array}\right]$$

The number of columns of $A$ must equal the number of rows of $B$, so the code checks this first.

Where it's used: every layer of a neural network, and transforming data.

LaTeX

$$
(AB)_{ij} = \sum_{k=1}^{n} a_{ik} \, b_{kj} \qquad \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} \begin{bmatrix} 5 & 6 \\ 7 & 8 \end{bmatrix} = \begin{bmatrix} 19 & 22 \\ 43 & 50 \end{bmatrix}
$$

Python

def mat_mul(A: list[list[float]], B: list[list[float]]) -> list[list[float]]:
    assert len(A[0]) == len(B), "columns of A must equal rows of B"
    return [[sum(A[i][k] * B[k][j] for k in range(len(B)))
             for j in range(len(B[0]))]
            for i in range(len(A))]

print(mat_mul([[1, 2], [3, 4]], [[5, 6], [7, 8]]))  # [[19, 22], [43, 50]]

Rust

fn mat_mul(a: &[Vec<f64>], b: &[Vec<f64>]) -> Vec<Vec<f64>> {
    assert_eq!(a[0].len(), b.len(), "columns of A must equal rows of B");
    (0..a.len())
        .map(|i| {
            (0..b[0].len())
                .map(|j| (0..b.len()).map(|k| a[i][k] * b[k][j]).sum())
                .collect()
        })
        .collect()
}

fn main() {
    let a = vec![vec![1.0, 2.0], vec![3.0, 4.0]];
    let b = vec![vec![5.0, 6.0], vec![7.0, 8.0]];
    println!("{:?}", mat_mul(&a, &b)); // [[19.0, 22.0], [43.0, 50.0]]
}

8. Determinant of a 2 × 2 Matrix

Determinant of a 2 × 2 Matrix

Formula:

$$det(A)=\left|\begin{array}{cc}a& b\\ c& d\end{array}\right|=ad-bc$$

If the determinant is 0, the matrix has no inverse.

Where it's used: checking whether a system of equations has a unique solution.

LaTeX

$$
\det(A) = \begin{vmatrix} a & b \\ c & d \end{vmatrix} = ad - bc
$$

Python

def det2(A: list[list[float]]) -> float:
    (a, b), (c, d) = A
    return a * d - b * c

print(det2([[1, 2], [3, 4]]))  # -2

Rust

fn det2(m: [[f64; 2]; 2]) -> f64 {
    m[0][0] * m[1][1] - m[0][1] * m[1][0]
}

fn main() {
    println!("{}", det2([[1.0, 2.0], [3.0, 4.0]])); // -2
}

9. Linear Layer

Linear Layer

Formula:

$$\mathbf{y}=W\mathbf{x}+\mathbf{b}$$

This is a matrix-vector product followed by adding the bias vector.

Where it's used: a fully connected layer of a neural network.

LaTeX

$$
\mathbf{y} = W \mathbf{x} + \mathbf{b}
$$

Python

def linear(W: list[list[float]], x: list[float], b: list[float]) -> list[float]:
    return [sum(w * xi for w, xi in zip(row, x)) + bi for row, bi in zip(W, b)]

W = [[1, 2], [3, 4]]
print(linear(W, [1, 1], [0.5, -0.5]))  # [3.5, 6.5]

Rust

fn linear(w: &[Vec<f64>], x: &[f64], b: &[f64]) -> Vec<f64> {
    w.iter()
        .zip(b)
        .map(|(row, bi)| row.iter().zip(x).map(|(wi, xi)| wi * xi).sum::<f64>() + bi)
        .collect()
}

fn main() {
    let w = vec![vec![1.0, 2.0], vec![3.0, 4.0]];
    println!("{:?}", linear(&w, &[1.0, 1.0], &[0.5, -0.5])); // [3.5, 6.5]
}