Env

import numpy as np
from matplotlib_inline import backend_inline
from d2l import torch as d2l

Given a function \( f: \mathbb{R}^n \to \mathbb{R} \)
if the derivative of \( f \) at \( \mathbf{x} \) exists, then the derivative is given by:
\( f'(\mathbf{x}) = \lim_{\mathbf{h} \to 0} \frac{f(\mathbf{x} + \mathbf{h}) - f(\mathbf{x}) }{\mathbf{h}} \)

  e.g.

def f(x):
    return 3 * x ** 2 - 4 * x

def numerical_lim(f, x, h):
    return (f(x + h) - f(x)) / h

  e.g.'s derivative
When \( h \to 0 \),
\(u' = f'(1) = \lim_{\mathbf{h} \to 0}\frac{f(1 + \mathbf{h}) - f(1) }{\mathbf{h}} = 2\)

h = 0.1
for i in range(5):
    print(f'h={h:.5f}, numerical limit={numerical_lim(f, 1, h):.5f}')
    h *= 0.1

h=0.10000, numerical limit=2.30000
h=0.01000, numerical limit=2.03000
h=0.00100, numerical limit=2.00300
h=0.00010, numerical limit=2.00030
h=0.00001, numerical limit=2.00003

Given \( \mathbf{y} = f(\mathbf{x}) \)
\(f'(\mathbf{x}) = \mathbf{y}' = \frac{dy}{d\mathbf{x}} = \frac{df}{d\mathbf{x}} = \frac{d}{d\mathbf{x}}f(\mathbf{x}) = Df(\mathbf{x}) = D_{\mathbf{x}}f(\mathbf{x})\)

Given Differentiable Functions \(f\) and \(g\)