I use a site called PredictionBook to make predictions, say how likely I think they are to occur, and then later judge whether I was right or wrong. Over many predictions, this improves my calibration on knowing what will happen on the future.

The reason this is super useful is that pretty much all humans are overconfident in their predictions. As one blogger put it:

Nearly everyone is very very very overconfident. We know this from experiments where people answer true/false trivia questions, then are asked to state how confident they are in their answer. If people’s confidence was well-calibrated, someone who said they were 99% confident (ie only 1% chance they’re wrong) would get the question wrong only 1% of the time. In fact, people who say they are 99% confident get the question wrong about 20% of the time.

So if I say I think my team will ship Feature X before March with 80% confidence, I want that to happen 80% of the time, not 55% of the time. If I think we will be 95% likely to hit our sales targets, then I'd like us to miss only 5% of the time. I want to be well-calibrated. This is really important not just for business strategy, but also for day-to-day decisions. If I'm "pretty sure" that I'll have enough time to hit the gym before my meeting, then I don't want to miss my meetings because my "pretty sure" is a human's normally overconfident way of judging a "slightly probable" situation.

But it's hard to realize that when you say something is 99% likely, it only happens 80% of the time, unless you are keep tracking of your predictions. So, I practice using PredictionBook. Over time, I have become less confident at the high end of certainty and more confident at the low end:

You can read this chart as saying I've made 37 predictions I thought were 50/50, and they were about 51/49, so pretty close. At 60% I'm overconfident–it's still a coinflip. So I read this and say, "Okay, I feel like this thing is a little more probable, but... it's probably not." At 70% I'm dead on across 124 predictions. 80% needs work. And I'm no longer nearly as bad at 90% confidence as I used to be.

What kind of things do I predict? I keep all my predictions private (the public predictions on PredictionBook are all pretty eccentric), so it's all pretty scattered. Here are some random predictions I made:


1. We will hire Bob. ( 20% confidence )
2. I'll have been to Mars by 2055. ( 16% confidence )
3. I'll decide it was a good idea to go for China now after a year. ( 60% confidence )
4. Chloe's ankle/foot are just sprained or better, not broken. ( 40% confidence )
5. This next logic shifter attempt will work at establishing serial communication between the robot and the Pi. ( 40% confidence
6. Max's infant swimming lessons will be successful (he's happy and capable of surviving in the water). ( 80% confidence )
7. We will go with this Berry Street childcare we're checking out. ( 50% confidence )
8. Chloe will finish Worm. ( 75% confidence )
9. My humans code won't be in the top 10 when I get back home on Wednesday. ( 80% confidence )
10. Max will be reading dozens of simple words via sounding them out by himself by his 3rd birthday. ( 70% confidence )

So yeah, it's kind of fun. If you want to get better at strategy, productivity, shipping stuff, or just general life effectiveness, you can geek out like me and start overcoming your natural human overconfidence bias with PredictionBook.