LaTeX for Calculus

✅ Copy-ready LaTeX (MathJax) code for Calculus, organised from beginner to advanced, with where each formula is used in maths and in Machine Learning, AI and Data Science. 😻

New to LaTeX? Start with the LaTeX (MathJax) Basics for Greek letters, dots and spaces.

Just click the "copy code" button below 'markdown'!

Summary Overview

Level	Section	What you'll learn	Key commands
Beginner	1. Limits 1.1 Limit Notation 1.2 One-Sided Limits 1.3 Limits at Infinity 1.4 Continuity	what a function approaches	\lim, \to, \infty
Beginner	2. Derivatives 2.1 Derivative Notations 2.2 Definition of the Derivative 2.3 Differentiation Rules 2.4 Common Derivatives 2.5 Higher-Order Derivatives 2.6 Evaluating at a Point 2.7 Critical Points and the Second Derivative Test 2.8 Implicit Differentiation 2.9 L'Hôpital's Rule	rates of change and slopes	\frac{d}{dx}, ', `\left. \right
Intermediate	3. Integrals 3.1 Indefinite Integral 3.2 Definite Integral and the Fundamental Theorem 3.3 Common Integrals 3.4 Substitution 3.5 Integration by Parts 3.6 Improper Integrals 3.7 Riemann Sums 3.8 Simple Differential Equations	areas and accumulated change	\int, \, dx, \Big[ \Big]
Intermediate	4. Sequences and Series 4.1 Sequence and Series Notation 4.2 Geometric Series 4.3 Taylor Series	adding up infinitely many terms	\sum_{n=1}^{\infty}, n!
Advanced	5. Multivariable Calculus 5.1 Partial Derivatives 5.2 Gradient Vector 5.3 Multivariable Chain Rule 5.4 Jacobian Matrix 5.5 Hessian Matrix 5.6 Double Integrals 5.7 Lagrange Multipliers	functions with many inputs	\partial, \nabla, \iint
Applied	6. Calculus for Machine Learning 6.1 Gradient Descent 6.2 Chain Rule in Backpropagation 6.3 Derivative of the Sigmoid 6.4 Gradient of the Mean Squared Error	how models learn	\nabla_{\theta}, \hat{y}, \sigma'

1. Limits (Beginner)

1.1 Limit Notation

Limit: "\lim_{x \to a}"

Example:

$$\underset{x\to 2}{lim}(3x+1)=7$$

\to draws the arrow, and the condition goes under \lim using _.

Where it's used: describing the value a function gets close to. Limits are the foundation of derivatives and integrals.

$$
\lim_{x \to 2} (3x + 1) = 7
$$

1.2 One-Sided Limits

One-Sided Limits: "^{+}" and "^{-}"

Example:

$$\underset{x\to 0^{-}}{lim}\frac{1}{x}=-\mathrm{\infty }\underset{x\to 0^{+}}{lim}\frac{1}{x}=\mathrm{\infty }$$

0^{-} means approaching from the left, and 0^{+} means approaching from the right.

Where it's used: functions with jumps or vertical asymptotes, and piecewise functions.

$$
\lim_{x \to 0^{-}} \frac{1}{x} = -\infty \qquad \lim_{x \to 0^{+}} \frac{1}{x} = \infty
$$

1.3 Limits at Infinity

Limits at Infinity: "\infty"

Example:

$$\underset{x\to \mathrm{\infty }}{lim}\frac{1}{x}=0\underset{n\to \mathrm{\infty }}{lim}{(1+\frac{1}{n})}^n=e$$

Where it's used: long-run behaviour, horizontal asymptotes, and checking whether an algorithm converges.

$$
\lim_{x \to \infty} \frac{1}{x} = 0 \qquad \lim_{n \to \infty} \left( 1 + \frac{1}{n} \right)^{n} = e
$$

1.4 Continuity

Continuity

Example:

$$f\text{ is continuous at }a⟺\underset{x\to a}{lim}f(x)=f(a)$$

Where it's used: checking that a function has no breaks before you differentiate it.

$$
f \text{ is continuous at } a \iff \lim_{x \to a} f(x) = f(a)
$$

2. Derivatives (Beginner)

2.1 Derivative Notations

Four Ways to Write a Derivative

Example:

$$f^{\prime }(x)\frac{dy}{dx}\frac{d}{dx}f(x)\dot{x}{D}_{x}f$$

Prime notation f'(x) is short, Leibniz notation \frac{dy}{dx} shows what changes with what, the dot \dot{x} is used for time in physics, and D_x f appears in some textbooks.

Where it's used: every topic in calculus. Different books prefer different notations.

$$
f'(x) \qquad \frac{dy}{dx} \qquad \frac{d}{dx} f(x) \qquad \dot{x} \qquad D_x f
$$

2.2 Definition of the Derivative

Derivative from First Principles

Example:

$$f^{\prime }(x)=\underset{h\to 0}{lim}\frac{f(x+h)-f(x)}{h}$$

Where it's used: proving differentiation rules from first principles, a common exam question.

$$
f'(x) = \lim_{h \to 0} \frac{f(x + h) - f(x)}{h}
$$

2.3 Differentiation Rules

Power, Product, Quotient and Chain Rules

Example:

$$\begin{array}{rlrl}\frac{d}{dx}{x}^n& =n{x}^{n-1}& & \text{Power rule}\\ \frac{d}{dx}[fg]& =f^{\prime }g+f{g}^{\prime }& & \text{Product rule}\\ \frac{d}{dx}\left[\frac{f}{g}\right]& =\frac{f^{\prime }g-f{g}^{\prime }}{g^2}& & \text{Quotient rule}\\ \frac{d}{dx}f(g(x))& =f^{\prime }(g(x))\cdot g^{\prime }(x)& & \text{Chain rule}\end{array}$$

&& adds a second column, so the rule names line up on the right.

Where it's used: almost every derivative you will calculate. The chain rule is the heart of backpropagation.

$$
\begin{aligned}
\frac{d}{dx} x^n &= n x^{n-1} && \text{Power rule} \\
\frac{d}{dx} [f g] &= f' g + f g' && \text{Product rule} \\
\frac{d}{dx} \left[ \frac{f}{g} \right] &= \frac{f' g - f g'}{g^2} && \text{Quotient rule} \\
\frac{d}{dx} f(g(x)) &= f'(g(x)) \cdot g'(x) && \text{Chain rule}
\end{aligned}
$$

2.4 Common Derivatives

Common Derivatives

Example:

$$\begin{array}{rlrl}\frac{d}{dx}{e}^x& =e^x& \frac{d}{dx}\ln x& =\frac{1}{x}\\ \frac{d}{dx}\sin x& =\cos x& \frac{d}{dx}\cos x& =-\sin x\\ \frac{d}{dx}{a}^x& =a^x\ln a& \frac{d}{dx}\tan x& ={\sec }^{2}x\end{array}$$

A single & between pairs makes two columns of equations.

Where it's used: the building blocks of harder derivatives. $e^x$ and $\ln x$ appear in the sigmoid function and log-loss.

$$
\begin{aligned}
\frac{d}{dx} e^{x} &= e^{x} & \frac{d}{dx} \ln x &= \frac{1}{x} \\
\frac{d}{dx} \sin x &= \cos x & \frac{d}{dx} \cos x &= -\sin x \\
\frac{d}{dx} a^{x} &= a^{x} \ln a & \frac{d}{dx} \tan x &= \sec^2 x
\end{aligned}
$$

2.5 Higher-Order Derivatives

Second and Higher Derivatives

Example:

$$f^{″}(x)\frac{d^{2}y}{d{x}^2}{f}^{(n)}(x)$$

In Leibniz notation, the 2 goes on the $d$ at the top and on the $x$ at the bottom. Use brackets in f^{(n)} so it is not read as a power.

Where it's used: concavity, acceleration and Taylor series.

$$
f''(x) \qquad \frac{d^2 y}{dx^2} \qquad f^{(n)}(x)
$$

2.6 Evaluating at a Point

Evaluation Bar: "\left. \right|"

Example:

$${\frac{d}{dx}{x}^2|}_{x=2}=2(2)=4$$

\left. is an invisible left bracket, so \right|_{x = 2} can draw the evaluation bar on its own.

Where it's used: finding the slope at a specific point, such as for a tangent line.

$$
\left. \frac{d}{dx} x^2 \right|_{x = 2} = 2(2) = 4
$$

2.7 Critical Points and the Second Derivative Test

Second Derivative Test

Example:

$$f^{\prime }(c)=0\text{and}{f}^{″}(c)>0\Rightarrow f\text{ has a local minimum at }x=c$$

If $f^{″}(c)<0$ instead, it is a local maximum.

Where it's used: optimisation problems, such as finding the lowest point of a cost function.

$$
f'(c) = 0 \quad \text{and} \quad f''(c) > 0 \;\Rightarrow\; f \text{ has a local minimum at } x = c
$$

2.8 Implicit Differentiation

Implicit Differentiation

Example:

$$\begin{array}{rl}{x}^2+y^2& =1\\ 2x+2y\frac{dy}{dx}& =0\\ \frac{dy}{dx}& =-\frac{x}{y}\end{array}$$

Differentiate both sides with respect to $x$. Because $y$ depends on $x$, the chain rule adds $\frac{dy}{dx}$ to every $y$ term.

Where it's used: curves that are hard to write as $y=f(x)$, such as circles, and related rates problems.

$$
\begin{aligned}
x^2 + y^2 &= 1 \\
2x + 2y \frac{dy}{dx} &= 0 \\
\frac{dy}{dx} &= -\frac{x}{y}
\end{aligned}
$$

2.9 L'Hôpital's Rule

L'Hôpital's Rule

Example:

$$\underset{x\to a}{lim}\frac{f(x)}{g(x)}=\underset{x\to a}{lim}\frac{f^{\prime }(x)}{g^{\prime }(x)}$$$$\underset{x\to 0}{lim}\frac{\sin x}{x}=\underset{x\to 0}{lim}\frac{\cos x}{1}=1$$

Use it only when substituting directly gives $\frac{0}{0}$ or $\frac{\mathrm{\infty }}{\mathrm{\infty }}$.

Where it's used: limits that look undefined at first, a very common exam question.

$$
\lim_{x \to a} \frac{f(x)}{g(x)} = \lim_{x \to a} \frac{f'(x)}{g'(x)}
$$

$$
\lim_{x \to 0} \frac{\sin x}{x} = \lim_{x \to 0} \frac{\cos x}{1} = 1
$$

3. Integrals (Intermediate)

3.1 Indefinite Integral

Indefinite Integral: "\int"

Example:

$$\int x^{2}dx=\frac{x^3}{3}+C$$

Do not forget + C. The thin space \, before $dx$ is the usual style.

Where it's used: finding antiderivatives.

$$
\int x^2 \, dx = \frac{x^3}{3} + C
$$

3.2 Definite Integral and the Fundamental Theorem

Definite Integral: "\int_{a}^{b}"

Example:

$${\int }_a^{b}f(x)dx=[F(x){]}_a^b=F(b)-F(a)$$$${\int }_0^2{x}^{2}dx={\left[\frac{x^3}{3}\right]}_0^2=\frac{8}{3}$$

\Big[ \Big] or \left[ \right] draws the square brackets used for evaluating between the limits.

Where it's used: areas under curves, total change and probabilities.

$$
\int_{a}^{b} f(x) \, dx = \Big[ F(x) \Big]_{a}^{b} = F(b) - F(a)
$$

$$
\int_{0}^{2} x^2 \, dx = \left[ \frac{x^3}{3} \right]_{0}^{2} = \frac{8}{3}
$$

3.3 Common Integrals

Common Integrals

Example:

$$\begin{array}{rl}\int x^{n}dx& =\frac{x^{n+1}}{n+1}+C(n\ne -1)\\ \int \frac{1}{x}dx& =\ln |x|+C\\ \int e^{x}dx& =e^x+C\\ \int \sin xdx& =-\cos x+C\\ \int \cos xdx& =\sin x+C\end{array}$$

Where it's used: the building blocks of harder integrals.

$$
\begin{aligned}
\int x^n \, dx &= \frac{x^{n+1}}{n+1} + C \quad (n \neq -1) \\
\int \frac{1}{x} \, dx &= \ln \lvert x \rvert + C \\
\int e^{x} \, dx &= e^{x} + C \\
\int \sin x \, dx &= -\cos x + C \\
\int \cos x \, dx &= \sin x + C
\end{aligned}
$$

3.4 Substitution

Integration by Substitution

Example:

$$\begin{array}{rlrl}\int 2x\cos (x^2)dx& =\int \cos udu& & \text{let }u=x^2,du=2xdx\\ & =\sin u+C\\ & =\sin (x^2)+C\end{array}$$

Where it's used: integrals where one part is the derivative of another part.

$$
\begin{aligned}
\int 2x \cos(x^2) \, dx &= \int \cos u \, du && \text{let } u = x^2, \; du = 2x \, dx \\
&= \sin u + C \\
&= \sin(x^2) + C
\end{aligned}
$$

3.5 Integration by Parts

Integration by Parts

Example:

$$\int udv=uv-\int vdu$$$$\int x{e}^{x}dx=x{e}^x-e^x+C$$

In the example, $u=x$ and $dv=e^{x}dx$.

Where it's used: integrals of products, such as $x{e}^x$ or $x\ln x$.

$$
\int u \, dv = uv - \int v \, du
$$

$$
\int x e^{x} \, dx = x e^{x} - e^{x} + C
$$

3.6 Improper Integrals

Improper Integral

Example:

$${\int }_0^{\mathrm{\infty }}{e}^{-x}dx=\underset{b\to \mathrm{\infty }}{lim}{\int }_0^b{e}^{-x}dx=1$$

Where it's used: probability distributions over infinite ranges, such as the exponential and normal distributions.

$$
\int_{0}^{\infty} e^{-x} \, dx = \lim_{b \to \infty} \int_{0}^{b} e^{-x} \, dx = 1
$$

3.7 Riemann Sums

Riemann Sum

Example:

$${\int }_a^{b}f(x)dx=\underset{n\to \mathrm{\infty }}{lim}\sum _{i=1}^{n}f(x_i^{\ast })\mathrm{\Delta }x,\mathrm{\Delta }x=\frac{b-a}{n}$$

Where it's used: understanding an integral as the area of many thin rectangles, and numerical integration in code.

$$
\int_{a}^{b} f(x) \, dx = \lim_{n \to \infty} \sum_{i=1}^{n} f(x_i^{*}) \, \Delta x, \qquad \Delta x = \frac{b - a}{n}
$$

3.8 Simple Differential Equations

Separable Differential Equation

Example:

$$\begin{array}{rl}\frac{dy}{dt}& =ky\\ \int \frac{1}{y}dy& =\int kdt\\ \ln |y|& =kt+C\\ y(t)& =y_0{e}^{kt}\end{array}$$

Move each variable to its own side, then integrate both sides. $y_0$ is the starting value at $t=0$.

Where it's used: exponential growth and decay, such as population growth, radioactive decay and continuous compound interest.

$$
\begin{aligned}
\frac{dy}{dt} &= k y \\
\int \frac{1}{y} \, dy &= \int k \, dt \\
\ln \lvert y \rvert &= k t + C \\
y(t) &= y_0 \, e^{k t}
\end{aligned}
$$

4. Sequences and Series (Intermediate)

4.1 Sequence and Series Notation

Sequences and Series

Example:

$$\{a_n{\}}_{n=1}^{\mathrm{\infty }}\sum _{n=1}^{\mathrm{\infty }}{a}_n=a_1+a_2+a_3+\cdots$$

Use \{ \} for the curly brackets of a sequence.

Where it's used: describing lists of numbers and their sums.

$$
\{ a_n \}_{n=1}^{\infty} \qquad \sum_{n=1}^{\infty} a_n = a_1 + a_2 + a_3 + \cdots
$$

4.2 Geometric Series

Geometric Series

Example:

$$\sum _{n=0}^{\mathrm{\infty }}a{r}^n=\frac{a}{1-r},|r|<1$$

Where it's used: compound interest, and discounted rewards in reinforcement learning, where $r$ is the discount factor $\gamma$.

$$
\sum_{n=0}^{\infty} a r^{n} = \frac{a}{1 - r}, \qquad \lvert r \rvert < 1
$$

4.3 Taylor Series

Taylor Series

Example:

$$f(x)=\sum _{n=0}^{\mathrm{\infty }}\frac{f^{(n)}(a)}{n!}(x-a{)}^n$$$$e^x=1+x+\frac{x^2}{2!}+\frac{x^3}{3!}+\cdots$$

Where it's used: approximating functions with polynomials, and optimisation methods such as Newton's method.

$$
f(x) = \sum_{n=0}^{\infty} \frac{f^{(n)}(a)}{n!} (x - a)^{n}
$$

$$
e^{x} = 1 + x + \frac{x^2}{2!} + \frac{x^3}{3!} + \cdots
$$

5. Multivariable Calculus (Advanced)

5.1 Partial Derivatives

Partial Derivative: "\partial"

Example:

$$f(x,y)=x^{2}y\Rightarrow \frac{\partial f}{\partial x}=2xy,\frac{\partial f}{\partial y}=x^2$$

Differentiate with respect to one variable and treat the others as constants.

Where it's used: functions with many inputs. Backpropagation computes $\frac{\partial L}{\partial w}$ for every weight $w$.

$$
f(x, y) = x^2 y \quad \Rightarrow \quad \frac{\partial f}{\partial x} = 2xy, \quad \frac{\partial f}{\partial y} = x^2
$$

5.2 Gradient Vector

Gradient: "\nabla"

Example:

$$\mathrm{\nabla }f(x,y)=\left[\begin{array}{c}\frac{\partial f}{\partial x}\\ \frac{\partial f}{\partial y}\end{array}\right]=\left[\begin{array}{c}2xy\\ x^2\end{array}\right]$$

Where it's used: finding the direction of steepest increase. Gradient descent moves in the opposite direction.

$$
\nabla f(x, y) = \begin{bmatrix} \frac{\partial f}{\partial x} \\ \frac{\partial f}{\partial y} \end{bmatrix} = \begin{bmatrix} 2xy \\ x^2 \end{bmatrix}
$$

5.3 Multivariable Chain Rule

Multivariable Chain Rule

Example:

$$\frac{dz}{dt}=\frac{\partial z}{\partial x}\frac{dx}{dt}+\frac{\partial z}{\partial y}\frac{dy}{dt}$$

Here $z$ depends on $x$ and $y$, which both depend on $t$.

Where it's used: backpropagation when one value affects the loss through several paths.

$$
\frac{dz}{dt} = \frac{\partial z}{\partial x} \frac{dx}{dt} + \frac{\partial z}{\partial y} \frac{dy}{dt}
$$

5.4 Jacobian Matrix

Jacobian Matrix

Example:

$$J=\left[\begin{array}{ccc}\frac{\partial f_1}{\partial x_1}& \cdots & \frac{\partial f_1}{\partial x_n}\\ ⋮& \ddots & ⋮\\ \frac{\partial f_m}{\partial x_1}& \cdots & \frac{\partial f_m}{\partial x_n}\end{array}\right]$$

Where it's used: derivatives of functions with several outputs, such as the layers of a neural network.

$$
J = \begin{bmatrix} \frac{\partial f_1}{\partial x_1} & \cdots & \frac{\partial f_1}{\partial x_n} \\ \vdots & \ddots & \vdots \\ \frac{\partial f_m}{\partial x_1} & \cdots & \frac{\partial f_m}{\partial x_n} \end{bmatrix}
$$

5.5 Hessian Matrix

Hessian Matrix

Example:

$$H=\left[\begin{array}{cc}\frac{{\partial }^{2}f}{\partial x^2}& \frac{{\partial }^{2}f}{\partial x\partial y}\\ \frac{{\partial }^{2}f}{\partial y\partial x}& \frac{{\partial }^{2}f}{\partial y^2}\end{array}\right]$$

Where it's used: the second derivative test with several variables, Newton's method, and the curvature of a loss function.

$$
H = \begin{bmatrix} \frac{\partial^2 f}{\partial x^2} & \frac{\partial^2 f}{\partial x \, \partial y} \\ \frac{\partial^2 f}{\partial y \, \partial x} & \frac{\partial^2 f}{\partial y^2} \end{bmatrix}
$$

5.6 Double Integrals

Double Integral: "\iint"

Example:

$${\iint }_{D}f(x,y)dA={\int }_a^b{\int }_c^{d}f(x,y)dydx$$

\iint and \iiint give double and triple integral signs.

Where it's used: volumes, and joint probability distributions of two variables.

$$
\iint_{D} f(x, y) \, dA = \int_{a}^{b} \int_{c}^{d} f(x, y) \, dy \, dx
$$

5.7 Lagrange Multipliers

Lagrange Multipliers

Example:

$$\mathrm{\nabla }f(x,y)=\lambda \mathrm{\nabla }g(x,y),g(x,y)=c$$$$\mathcal{L}(x,y,\lambda )=f(x,y)-\lambda (g(x,y)-c)$$

$\lambda$ is the Lagrange multiplier. To solve, set every partial derivative of the Lagrangian $\mathcal{L}$ to zero.

Where it's used: optimisation with a constraint, such as support vector machines in Machine Learning.

$$
\nabla f(x, y) = \lambda \, \nabla g(x, y), \qquad g(x, y) = c
$$

$$
\mathcal{L}(x, y, \lambda) = f(x, y) - \lambda \big( g(x, y) - c \big)
$$

6. Calculus for Machine Learning (Applied)

6.1 Gradient Descent

Gradient Descent Update

Example:

$$\theta :=\theta -\alpha {\mathrm{\nabla }}_{\theta }J(\theta )$$

$\alpha$ is the learning rate and $J(\theta )$ is the cost function.

Where it's used: training almost every machine learning model.

$$
\theta := \theta - \alpha \, \nabla_{\theta} J(\theta)
$$

6.2 Chain Rule in Backpropagation

Backpropagation

Example:

$$\frac{\partial L}{\partial w}=\frac{\partial L}{\partial \hat{y}}\cdot \frac{\partial \hat{y}}{\partial z}\cdot \frac{\partial z}{\partial w}$$

Here $z=wx+b$ and $\hat{y}=\sigma (z)$. Each fraction is one step back through the network.

Where it's used: computing how each weight affects the loss in a neural network.

$$
\frac{\partial L}{\partial w} = \frac{\partial L}{\partial \hat{y}} \cdot \frac{\partial \hat{y}}{\partial z} \cdot \frac{\partial z}{\partial w}
$$

6.3 Derivative of the Sigmoid

Sigmoid Derivative

Example:

$$\sigma (z)=\frac{1}{1+e^{-z}}\Rightarrow {\sigma }^{\prime }(z)=\sigma (z)(1-\sigma (z))$$

Where it's used: logistic regression and neural networks that use the sigmoid activation.

$$
\sigma(z) = \frac{1}{1 + e^{-z}} \quad \Rightarrow \quad \sigma'(z) = \sigma(z) \big( 1 - \sigma(z) \big)
$$

6.4 Gradient of the Mean Squared Error

MSE Gradient

Example:

$$J(\mathbf{w})=\frac{1}{n}\sum _{i=1}^n({\hat{y}}_i-y_i{)}^2\Rightarrow \frac{\partial J}{\partial w_j}=\frac{2}{n}\sum _{i=1}^n({\hat{y}}_i-y_i)x_{ij}$$

Here ${\hat{y}}_i={\mathbf{w}}^{\mathrm{\top }}{\mathbf{x}}_i+b$, and $x_{ij}$ is feature $j$ of example $i$.

Where it's used: training linear regression with gradient descent.

$$
J(\mathbf{w}) = \frac{1}{n} \sum_{i=1}^{n} (\hat{y}_i - y_i)^2 \;\Rightarrow\; \frac{\partial J}{\partial w_j} = \frac{2}{n} \sum_{i=1}^{n} (\hat{y}_i - y_i) \, x_{ij}
$$