Tetlock: How the Scoring Worked

In Tetlock's experiment on prediction, subjects would predict 'yes, the price of oil will go above $X in January.' Or 'No; it won't.' For that prediction, they would have to assign a weight, a confidence level. If you predicted 'yes' and said 60% sure, and it turned out to be a 'yes,' you would score well but not fantastically well because you had hedged your bet. If the reality turned out to be 'no,' you would lose some points but not a lot. By contrast, if you said 'yes' with 90% certainty, you would score Big Points. And so you would have to track your prediction, your confidence and the result, scored in this way. Try this! When you make a prediction, assign a confidence level and later go back and check how you did… It is an amazing discipline.