As a quick reminder for those who may have missed the beginning, Uncertainty Wednesday is a series about, well, uncertainty. We are in the middle of limits on observations. To understand why these matter and how they relate to uncertainty, you should go back to the initial post in the series which lays out a framework. So far we have covered the following limitations on observations: foundational, resolution and measurement error. Today we will cover the cost of observation.

One reaction to the limitations of resolution and measurement error is to suggest building a better instrument. But for a given technology a better instrument will always be more expensive. The important constant here is “given technology.” For instance, in the post about measurement error, I referred to the averaging of observations in the early days of telescopes. For a given telescope technology building a better instrument is more expensive. In the wonderful book The Age of Wonder there is a detailed description of William Herschel’s quest for a larger telescope (which would have better resolution). Ultimately it was the King of England who financed the famous 40-foot telescope.

Over time of course technology progresses and today you can build a telescope that far exceeds Herschel’s for a fraction of the price due to better lenses and mirrors. But then again with today’s technology if you want even better resolution and lower error than that cost still goes up. For instance, the Hubble Space Telescope has cost around $10 Billion (about $1.5 Billion for initial launch, the rest for ongoing operation and maintenance.

Now you might argue that cost isn’t really a separate limitation. That cost really just determines the resolution and measurement error that you can afford. But that’s not the case, because there is a different component to cost as well: how many observations you can make!

Take measurement of temperature as an example. If you want observations of temperature over time in one location, the longer the time the higher cost. If you want parallel observations of temperature in multiple locations, the more locations the higher the cost. And those two are multiplicative, meaning observation of temperature over time in multiple locations it gets expensive quickly.

In all real world applications there is some kind of budget constraint, which will limit the number of observations you can make. Because the number of observations is limited there is additional uncertainty that goes above and beyond the measurement error and resolution of each individual observation.

This is a limitation on observations that turns out to be super important especially in many business contexts. Consider quality control as an example. Each inspection (observation) costs money. You can reduce uncertainty about quality by making more inspections but the cost of your quality control will go up. This is what has given rise to a lot of work on so-called statistical quality control. More on that in a future post.  

Now last week I asked a question whether you could spot the important reasoning misstep when Adolphe Quetelet started applying averages from the measurement of stars to the measurement of people. Here is the answer: the variation in observation of the stars was the result of measurement error (change in observation, reality unchanged — meaning the same star is observed). The variation in the height and chest size of humans on the other hand is the result of changes in reality — each human body is in fact a separate entity! This is why understanding the framework that clearly distinguishes between observations, explanations an reality is so important.