HMWK 6

Homework VI — Online Algorithms for Mean and Variance

HMWK 6 — Deriving and Implementing Online Algorithms for Mean and Variance

Written on November 2025

This article presents concise derivations of the recurrence relationships for the arithmetic mean and variance, followed by minimal, working JavaScript implementations of online algorithms that update these statistics incrementally. Unlike traditional batch methods, online algorithms are efficient in time and memory and offer better numerical stability.

1. Recurrence for the Arithmetic Mean

Let \(x_1, x_2, \dots, x_n\) be observations and let the running mean after \(n\) samples be \(\displaystyle \mu_n=\frac{1}{n}\sum_{i=1}^n x_i\).

Using \(S_n=S_{n-1}+x_n\) and \(S_{n-1}=(n-1)\mu_{n-1}\):

\[ \mu_n=\frac{(n-1)\mu_{n-1}+x_n}{n} = \mu_{n-1} + \frac{x_n-\mu_{n-1}}{n}. \]

This update needs only the previous mean, the new value, and \(n\).

2. Recurrence for the Variance (Welford’s Method)

Define the centered sum of squares \(\displaystyle M2_n=\sum_{i=1}^{n}(x_i-\mu_n)^2\). Let \(\delta=x_n-\mu_{n-1}\) and update the mean \(\mu_n=\mu_{n-1}+\delta/n\). Then:

\[ M2_n = M2_{n-1} + \delta\,(x_n-\mu_n). \]

The variances follow as \(\displaystyle \sigma^2_n=\frac{M2_n}{n}\) (population) and \(\displaystyle s^2_n=\frac{M2_n}{n-1}\) (sample, \(n\ge2\)).

This avoids the unstable identity \( \operatorname{Var}(X)=E[X^2]-E[X]^2 \) that suffers from catastrophic cancellation.

3. Online Mean Calculator

Type a number and press Add Value. The mean is updated via \( \mu\leftarrow \mu + (x-\mu)/n \).


Count (n): 0

Mean (μ): 0.00000

4. Online Variance Calculator

This calculator implements Welford’s algorithm using the \(M2\) accumulator.


Count (n): 0

Mean (μ): 0.00000

Sample Variance (s²): 0.00000

Sample Std Dev (s): 0.00000

5. (Optional) Numerical & Computational Advantages

Online updates minimize catastrophic cancellation and error propagation by avoiding differences of large, close quantities. They need only one pass and \(O(1)\) memory, improving robustness and scalability for streams and large datasets.

© 2025 Homework VI — Statistics Assignment