Philosophy Mondays: Human-AI Collaboration
Today's Philosophy Monday is an important interlude. I want to reveal that I have not been writing the posts in this series entirely by myself. Instead I have been working with Claude, not just for the graphic illustrations, but also for the text. My method has been to write a rough draft and then ask Claude for improvement suggestions. I will expand this collaboration to other intelligences going forward, including open source models such as Llama and DeepSeek. I will also explore other moda...
Intent-based Collaboration Environments
AI Native IDEs for Code, Engineering, Science
Web3/Crypto: Why Bother?
One thing that keeps surprising me is how quite a few people see absolutely nothing redeeming in web3 (née crypto). Maybe this is their genuine belief. Maybe it is a reaction to the extreme boosterism of some proponents who present web3 as bringing about a libertarian nirvana. From early on I have tried to provide a more rounded perspective, pointing to both the good and the bad that can come from it as in my talks at the Blockstack Summits. Today, however, I want to attempt to provide a coge...
Philosophy Mondays: Human-AI Collaboration
Today's Philosophy Monday is an important interlude. I want to reveal that I have not been writing the posts in this series entirely by myself. Instead I have been working with Claude, not just for the graphic illustrations, but also for the text. My method has been to write a rough draft and then ask Claude for improvement suggestions. I will expand this collaboration to other intelligences going forward, including open source models such as Llama and DeepSeek. I will also explore other moda...
Intent-based Collaboration Environments
AI Native IDEs for Code, Engineering, Science
Web3/Crypto: Why Bother?
One thing that keeps surprising me is how quite a few people see absolutely nothing redeeming in web3 (née crypto). Maybe this is their genuine belief. Maybe it is a reaction to the extreme boosterism of some proponents who present web3 as bringing about a libertarian nirvana. From early on I have tried to provide a more rounded perspective, pointing to both the good and the bad that can come from it as in my talks at the Blockstack Summits. Today, however, I want to attempt to provide a coge...
>400 subscribers
>400 subscribers
Share Dialog
Share Dialog
In last week’s Uncertainty Wednesday, I introduced the expected value EV of a random variable X. We saw that EV(X) is not a measure of uncertainty. The hypothetical investments I had described all had the same expected value of 0. It is trivial, given a random variable with EV(X) = μ to construct X’ so that EV(X’) = 0. That’s in fact how I constructed the first investment. I started with $0 with 99% probability and $100 with 1% probability, which has an EV of
EV = 0.99 * 0 + 0.01 * 100 = 1
and then I simply subtracted 1 from each possible outcome to get -$1 with 99% probability and $99 with 1% probability.
What we are looking for instead in order to measure uncertainty, is a number that captures how spread out the values are around the expected value. The obvious approach to this would be to form the weighted sum of the distances from the expected value as follows:
AAD(X) = ∑ P(X = x) * |x - EV(X)|
where | | denotes absolute value (meaning the magnitude without the sign). This metric is known as the Average Absolute Deviation (btw, instead of the shorthand P(x) I am now writing P(X = x) to show more clearly that it is the probability of the random variable X taking on the value x).
AAD is a one measure of dispersion around the expected value but it is not the most commonly used one. That instead is what is known as Variance, which is defined as follows
VAR(X) = sum Prob(X = x) * (x - EV(X))^2
Or expressed in words: the probability weighted sum of the squared distances of possible outcomes from the expected value. It turns out that for a variety of reasons using the square instead of the absolute value has some useful properties and also interesting physical interpretations (we may get to those at some later point).
Let’s take a look at both of these metrics for the random variables from our investment examples
Variance
Investment 1: 0.99 * (-1 - 0)^2 + 0.01 * (99 - 0)^2 = 0.99 * 1 + 0.01 * 9,801 = 99
Investment 2: 0.99 * (-100 - 0)^2 + 0.01 * (9,900 - 0)^2 = 0.99 * 10,000 + 0.01 * 98,010,000 = 990,000
Investment 3: 0.99 * (-10,000 - 0)^2 + 0.01 * (990,000 - 0)^2 = 0.99 * 100,000,000 + 0.01 * 980,100,000,000 = 9,900,000,000
Average Absolute Deviation
Investment 1: 0.99 * |-1 - 0| + 0.01 * |99 - 0| = 0.99 * 1 + 0.01 * 99 = 1.98
Investment 2: 0.99 * |-100 - 0| + 0.01 * |9,900 - 0| = 0.99 * 100 + 0.01 * 9,900 = 198
Investment 3: 0.99 * |-10,000 - 0| + 0.01 * |990,000 - 0| = 0.99 * 10,000 + 0.01 * 990,000 = 19,800
You might have previously noticed that Investment 2 is simply Investment 1 scaled by a factor of 100 (and ditto Investment 3 is 100x Investment 2). We see that AAD, as per its definition, follows that same linear scaling whereas variance grows in the square, meaning the variance of Investment 2 is 100^2 = 10,000x the variance of Investment 1.
Both of these are measures that pick up the values of the random variable as separate from the structure of the underlying probabilities. If that doesn’t make sense to you, go back and read the initial post about measuring uncertainty and then go back to the posts about entropy. The three hypothetical investments each have the same entropy as they share the same probabilities. But AAD and Variance pick up the difference in payouts between the investments.
In last week’s Uncertainty Wednesday, I introduced the expected value EV of a random variable X. We saw that EV(X) is not a measure of uncertainty. The hypothetical investments I had described all had the same expected value of 0. It is trivial, given a random variable with EV(X) = μ to construct X’ so that EV(X’) = 0. That’s in fact how I constructed the first investment. I started with $0 with 99% probability and $100 with 1% probability, which has an EV of
EV = 0.99 * 0 + 0.01 * 100 = 1
and then I simply subtracted 1 from each possible outcome to get -$1 with 99% probability and $99 with 1% probability.
What we are looking for instead in order to measure uncertainty, is a number that captures how spread out the values are around the expected value. The obvious approach to this would be to form the weighted sum of the distances from the expected value as follows:
AAD(X) = ∑ P(X = x) * |x - EV(X)|
where | | denotes absolute value (meaning the magnitude without the sign). This metric is known as the Average Absolute Deviation (btw, instead of the shorthand P(x) I am now writing P(X = x) to show more clearly that it is the probability of the random variable X taking on the value x).
AAD is a one measure of dispersion around the expected value but it is not the most commonly used one. That instead is what is known as Variance, which is defined as follows
VAR(X) = sum Prob(X = x) * (x - EV(X))^2
Or expressed in words: the probability weighted sum of the squared distances of possible outcomes from the expected value. It turns out that for a variety of reasons using the square instead of the absolute value has some useful properties and also interesting physical interpretations (we may get to those at some later point).
Let’s take a look at both of these metrics for the random variables from our investment examples
Variance
Investment 1: 0.99 * (-1 - 0)^2 + 0.01 * (99 - 0)^2 = 0.99 * 1 + 0.01 * 9,801 = 99
Investment 2: 0.99 * (-100 - 0)^2 + 0.01 * (9,900 - 0)^2 = 0.99 * 10,000 + 0.01 * 98,010,000 = 990,000
Investment 3: 0.99 * (-10,000 - 0)^2 + 0.01 * (990,000 - 0)^2 = 0.99 * 100,000,000 + 0.01 * 980,100,000,000 = 9,900,000,000
Average Absolute Deviation
Investment 1: 0.99 * |-1 - 0| + 0.01 * |99 - 0| = 0.99 * 1 + 0.01 * 99 = 1.98
Investment 2: 0.99 * |-100 - 0| + 0.01 * |9,900 - 0| = 0.99 * 100 + 0.01 * 9,900 = 198
Investment 3: 0.99 * |-10,000 - 0| + 0.01 * |990,000 - 0| = 0.99 * 10,000 + 0.01 * 990,000 = 19,800
You might have previously noticed that Investment 2 is simply Investment 1 scaled by a factor of 100 (and ditto Investment 3 is 100x Investment 2). We see that AAD, as per its definition, follows that same linear scaling whereas variance grows in the square, meaning the variance of Investment 2 is 100^2 = 10,000x the variance of Investment 1.
Both of these are measures that pick up the values of the random variable as separate from the structure of the underlying probabilities. If that doesn’t make sense to you, go back and read the initial post about measuring uncertainty and then go back to the posts about entropy. The three hypothetical investments each have the same entropy as they share the same probabilities. But AAD and Variance pick up the difference in payouts between the investments.
No comments yet