Law of large numbers

The weak law

The weak law of large numbers states that the sample average converges in probability towards the expected value

That is to say that for any positive number ε,

(Proof)

Interpreting the convergence in probability, weak law essentially states that the average of many observations will eventually be close to the mean within any nonzero margin specified, no matter how small.

This version is called the weak law because convergence in probability is weak convergence of random variables.

A consequence of the weak LLN is the asymptotic equipartition property.

[edit] The strong law

The strong law of large numbers states that the sample average converges almost surely to the expected value

That is,

The proof is more complex than that of the weak law. This law justifies the intuitive interpretation of the expected value of a random variable as the "long-term average when sampling repeatedly".

This version is called the strong law because almost sure convergence is strong convergence of random variables. The strong law implies the weak.