Expected Value Pathology

(You can watch the video version of this post on YouTube.)

Now you know what an expected value is: it’s the sum over all possible outcomes of the probability times the value (or an integral if the space continuous). There are actually some fun ways to use expected values in algorithms, but we’ll talk about that some other time.

So expectation is linear, right, and this doesn’t assume independence or anything - we just have $\mathbb{E}[X+Y]=\mathbb{E}[X]+\mathbb{E}[Y]$, no conditions whatsoever.

Well – and this is the pathology from the title – not every random variable has an expected value. A lot of theorems that use expected values are technically qualified with “if it exists” or “if it’s finite.” Assuming finite variance in particular is a bit of thing, even in the Central Limit Theorem.

This is maybe not really something that would have come up for you unless you were at the maths department (I don’t remember it from studying computer science), but it actually follows quite straightforwardly from the definition: not all series and integrals are finite or well-defined, so if you make a weird enough random variable, you can cause some real shenanigans. And it doesn’t even need to be extremely weird: it actually comes up. Over a finite domain you’re good, of course, but check out the Cauchy distribution, for example, which is remarkably reasonable for not having an expectation or a variance.