Brownian Motion

A Jittering Particle

In 1827 the botanist Robert Brown watched tiny particles from pollen grains jitter endlessly in water. In 1905 Einstein explained why: the particle is hit, from random directions, by molecules far too small to see.

Raise the energy, make the particle heavier, then reveal the molecules. The strip below the box tracks only the particle's horizontal position.

$t$

Time elapsed since we started watching.

$\Delta x$

How far the particle has moved sideways (left or right) from where it started. Positive means right, negative means left. The Greek letter Δ ("delta") just means "change in".

$\Delta x^2$

That distance multiplied by itself. Squaring makes left and right moves both count as positive, so they can't cancel each other out.

$\langle \;\cdot\; \rangle$

The average over many particles, or over many repeats of the same experiment.

$D$

The diffusion coefficient: one number that says how fast the particle spreads out. A hotter fluid (the Energy slider) makes it larger.

In plain words: on average, the squared distance grows in step with time, so the distance itself grows only like the square root of time. To get twice as far from the start, the particle needs four times as long. A car at steady speed is different: double the time and it covers double the distance. The particle is slow because its random kicks keep partly undoing each other.

Energy is the fluid's temperature: how fast the molecules move. Mass ratio is how many times heavier the particle is than one molecule. A heavier particle is more sluggish, so each kick nudges it less.

No single collision matters. What matters is the sum of countless tiny kicks, each from a random direction and unrelated to the others. That sum produces the square-root-of-time spreading above. The jagged line in the strip is the object the rest of this chapter makes precise: Brownian motion.

import numpy as np
rng = np.random.default_rng(0)
particles, kicks_per_second, seconds = 2000, 1000, 4
kick = 0.01                                          # size of one molecular kick
x = np.cumsum(rng.choice([-kick, kick], size=(particles, kicks_per_second * seconds)), axis=1)
for t in (1, 2, 4):
    msd = np.mean(x[:, t * kicks_per_second - 1] ** 2)   # mean squared displacement
    print(f"t = {t} s: <dx^2> = {msd:.4f}")              # doubles when t doubles: 2Dt

t = 1 s: <dx^2> = 0.0977
t = 2 s: <dx^2> = 0.1949
t = 4 s: <dx^2> = 0.3985

1.1From Random Walk to Brownian Motion

Replace the molecules with a coin. Flip it: heads, step up by 1; tails, step down by 1. This is a random walk. Now squeeze n flips into one unit of time and shrink every step, so the path stays on screen however many flips we make.

How much should each step shrink? After n flips the walk is typically about $\sqrt n$ steps from where it started, not n, because heads and tails partly cancel. So we divide by $\sqrt n$. The typical distance then stays about 1 whatever n is. Divide by n instead and the path flattens to a straight line at 0. Don't divide at all and it flies off the screen.

Drag n up. The staircases turn into rough curves, and the places where the walks end up (the bars on the right) pile into a bell shape.

$n$

Number of coin flips per unit of time. The slider sets it.

$S_k$

The walk's position after $k$ flips: the running total of +1s and −1s. For example, heads, heads, tails gives $S_3 = 1 + 1 - 1 = 1$.

$\lfloor nt \rfloor$

How many whole flips have happened by time $t$. The brackets mean "round down". With $n = 10$ and $t = 0.37$ that is $\lfloor 3.7 \rfloor = 3$ flips.

$\frac{1}{\sqrt n}$

The shrink factor applied to every step, explained above.

$W^{(n)}_t$

The shrunken walk's height at time $t$: one point on the orange path.

$\longrightarrow$

"Gets closer and closer to, as $n$ grows without limit."

$W_t$

Brownian motion: the random path the walks settle into. The letter W honours Norbert Wiener, who first put it on a rigorous footing.

$W_1 \sim N(0,1)$

"Where the path ends at time 1 follows the standard bell curve." The $\sim$ reads "is distributed as". $N(0,1)$ is the normal (bell-shaped) distribution centred at 0, with average squared distance from 0 equal to 1.

In plain words: take a coin-flip walk, make the flips faster and the steps smaller in exactly the right balance ($\sqrt n$), and the jagged staircase turns into one specific random curve, Brownian motion. At any moment, its height is a bell-curve random number. The bell shape is no coincidence: adding up many small, independent random pieces always produces it. This is the central limit theorem.

The mean is the average ending point. The variance is the average squared distance from that mean, the same "average of squares" as in Einstein's formula above.

Shrink the steps by $1/\sqrt n$ and the random walk turns into Brownian motion. Mathematicians call this Donsker's theorem. The coin doesn't matter: any fair random steps (ones that average out to zero) give Brownian motion too, at most stretched vertically. That is why Brownian motion turns up everywhere, from pollen grains to stock prices.

import numpy as np
rng = np.random.default_rng(1)
walks = 10_000
for n in (10, 100, 10_000):
    S_n = 2 * rng.binomial(n, 0.5, size=walks) - n   # sum of n coin flips (+1 or -1)
    W1 = S_n / np.sqrt(n)                              # shrink every step by 1/sqrt(n)
    print(f"n = {n:>6}: mean {W1.mean():+.3f}, variance {W1.var():.3f}")   # -> 0 and 1

n =     10: mean +0.004, variance 1.006
n =    100: mean +0.016, variance 0.989
n =  10000: mean +0.011, variance 1.025

1.2Defining Properties

Section 1.1 built Brownian motion as a limit. Mathematicians usually go the other way and define it by a short list of rules. The most important rules are about increments: how much the path rises (or falls) between two moments in time.

The chart on the left shows 100 of 500 simulated paths, with one in orange. Each shaded band is a stretch of time. For every path, the scatter plot on the right puts one dot at (its rise over A, its rise over B). Stretch a band by dragging its edge and watch the cloud widen in that direction. Then slide one band on top of the other and watch the cloud tilt.

$s,\ t$

Two moments in time, with $s$ earlier than $t$. On the chart they are the left and right edges of a band.

$W_t - W_s$

The increment: where the path is at time $t$ minus where it was at time $s$. It is positive if the path rose over that stretch, negative if it fell.

$\sim N(0,\; t-s)$

"Follows a bell curve centred at 0, with variance $t - s$." Variance is the average squared size of the rise, so a band twice as long gives rises whose squares are twice as large on average.

independent

Knowing one of the two rises tells you nothing about the other. In the scatter plot, the cloud has no tilt.

correlation

A number from −1 to 1 measuring how much two quantities tend to move together. 0 means no tilt, 1 means a perfect upward line. Printed below the chart.

dashed ellipse

Where theory says about 95% of the dots should land. Its width and height come from the variances, its tilt from the correlation.

In plain words: Brownian motion has no memory. How the path moves during one stretch of time says nothing about how it moves during another, separate stretch. The only thing that controls how much it moves is how long you wait: the average squared rise equals the elapsed time, the same rule as Einstein's formula above. Together with two small housekeeping rules (the path starts at 0, and it has no jumps), that is the complete definition.

Drag an edge of a band to resize it, or its middle to slide it. Keyboard: Tab to the chart, press A or B to pick an interval, ← → to slide it, ↑ ↓ to resize it (hold Shift for bigger steps). The orange dot on the right is the orange path.

Longer interval, bigger spread: the variance of the rise equals the interval's length. Separate intervals give an untilted cloud, because the path has no memory. Overlap them and the cloud tilts, since both rises now share the overlapping piece of path. In fact the correlation equals the overlap divided by $\sqrt{|A|\cdot|B|}$, where $|A|$ and $|B|$ are the two bands' lengths.

import numpy as np
rng = np.random.default_rng(2)
paths, steps = 5000, 100
dW = rng.normal(0, np.sqrt(1 / steps), (paths, steps))
W = np.concatenate([np.zeros((paths, 1)), np.cumsum(dW, axis=1)], axis=1)
rise = lambda s, t: W[:, round(t * steps)] - W[:, round(s * steps)]
A, B = rise(0.15, 0.35), rise(0.55, 0.85)            # two separate intervals
print(f"var A {A.var():.3f} (0.20)  var B {B.var():.3f} (0.30)  corr {np.corrcoef(A, B)[0, 1]:+.3f} (0)")
C = rise(0.15, 0.45)                                   # overlaps A by 0.20
print(f"corr(A, C) {np.corrcoef(A, C)[0, 1]:.3f} (theory {0.2 / np.sqrt(0.2 * 0.3):.3f})")

var A 0.200 (0.20)  var B 0.309 (0.30)  corr -0.010 (0)
corr(A, C) 0.816 (theory 0.816)

1.3Rough Paths

Ordinary calculus rests on one idea: zoom in far enough on a smooth curve and it looks like a straight line. The steepness of that line is the curve's derivative, its rate of change at a single instant. Speed, for example, is the derivative of position.

Drag the zoom slider, or press Play. Both panels zoom toward the same moment, $t = 0.5$ (the thin vertical line), up to a million times. Each panel stretches vertically so the curve always fills the frame. Watch which one straightens out.

$h$

The width of the time window you are looking at. Zooming in makes $h$ smaller. At ×1,000, $h = 0.001$.

$W_{t+h} - W_t$

How much the path rises across the window: the increment from section 1.2.

$\lvert \;\cdot\; \rvert$

The size of a number, ignoring its sign. So $\lvert -3 \rvert = 3$.

$\approx \sqrt h$

"Is typically about $\sqrt h$". Section 1.2 showed the squared rise averages $h$, so the rise itself is typically about $\sqrt h$. For a small window that is much bigger than $h$: with $h = 0.01$ the rise is about $0.1$, ten times the window's width.

$\Longrightarrow$

"Therefore".

$\frac{W_{t+h} - W_t}{h}$

The slope across the window: rise divided by run, the steepness of the straight line joining the window's two ends. For a smooth curve it settles down to the derivative as $h$ shrinks. The panels show the same idea as "height ÷ width".

$\to \infty$

"Grows without limit."

In plain words: in a small window of width $h$, a Brownian path moves by about $\sqrt h$, which is far more than $h$. So its slope, roughly $\sqrt h / h = 1/\sqrt h$, gets steeper the closer you look instead of settling down. Zoom in 100 times and the slope grows about 10 times. Zoom in 10,000 times and it grows about 100 times. The smooth curve's "height ÷ width" settles to one number (about 5.8 here, its derivative at $t = 0.5$). The Brownian path's keeps climbing.

The smooth curve turns into a straight line. The Brownian path looks just as rough at ×1,000,000 as at ×1: every zoom level looks statistically the same, a property called self-similarity. Its slope never settles, so it has no derivative at any point (Paley, Wiener and Zygmund proved this in 1933). Ordinary calculus is built on derivatives, so it cannot handle Brownian motion. The rest of this site builds the calculus that can: Itô calculus. The first clue is in Chapter 2: squared changes, which were the problem here, turn out to behave perfectly.

import numpy as np
rng = np.random.default_rng(3)
f = lambda t: 0.8 * np.sin(2 * np.pi * 1.3 * t) + 0.4 * np.sin(2 * np.pi * 3.1 * t + 1)
for h in (1e-2, 1e-4, 1e-6):
    smooth = abs(f(0.5 + h) - f(0.5)) / h
    brown = np.mean(np.abs(rng.normal(0, np.sqrt(h), 10_000))) / h   # typical |rise| / h
    print(f"h = {h:.0e}: smooth slope {smooth:.2f}, Brownian slope {brown:,.0f} (0.8/sqrt(h) = {0.8 / np.sqrt(h):,.0f})")

h = 1e-02: smooth slope 4.85, Brownian slope 8 (0.8/sqrt(h) = 8)
h = 1e-04: smooth slope 5.81, Brownian slope 79 (0.8/sqrt(h) = 80)
h = 1e-06: smooth slope 5.82, Brownian slope 787 (0.8/sqrt(h) = 800)