Module 3 / Lesson 6.5
Aggregation Systems
In the previous lesson, we looked at individual data points. Here, we explore how to synthesize multiple points of data to arrive at one number, a quantitative prediction of risk.
The "One Number" Concept: Aggregation systems aim to take a variety of predictions and evidence from different data sources and distill them into a single probability or prediction. In the previous Lesson 6.4, you learned about different approaches people might use to predict an outcome. It will be common to have multiple sources of information (maybe various predictions each from a difference source of historical data and additional various predictions each from a difference prospective model). For example, you might have asteroid predictions from looking at the fossil record, from rocks, from looking to the sky using multiple different types of telescopes. How do you combine all of these different predictions from different sources of information into a single prediction? How do you weight the different predictions? Here, you will learn about different tools for aggregating information from multiple sources, and the professions of individuals who do this kind of aggregation.
Learning: Watch the expert briefings on superforecasters and prediction markets.
- Reflection: Analyze how aggregation systems, like betting markets and specialized teams, transform messy, diverse data sets into a single "one number" probability. Consider why certain systems might be more or less effective when predicting existential risks versus standard financial risks.
Collective Intelligence Tools (surveys and prediction markets) vs Individual Intelligence Tools (superforecasters and actuaries)
- 6.5.1 Superforecasters: Superforecasters are individuals who use probabilistic thinking to aggregate information. They are experts in evaluating the quality of different types of information and in aggregating that information, even when they may not know anything about the specific thing they are trying to predict.
- 6.5.2 Prediction Markets: Betting platforms where the market price reflects the collective probability of an event occurring. A prediction market is a market where people buy the right to earn $1 if a certain event happens. For example, will a democrat win the 2028 presidential election? What is the probability of this outcome? In a prediction market, you can buy the right to $1 if a democrat wins the 2028 presidential election, perhaps for a price of $0.52. If you thought that the probability of a democrat winning was 52%, this would be a break-even bet. 52% of the time, you would win the bet and earn $1, so your expected payout would be $0.52 ($1 52% of the time and $0 48% of the time) on a bet that cost $0.52. A person who thought the probability was above 52% would want to buy this contract and would drive its price up, and a person who thought the probability was below 52% would want to sell this contract and would drive its price down. This process of buying and selling should continue until the price settles. In this way, the prices in prediction markets encode the probability that the market participants ascribe to the outcome.
- Challenges: One challenge of a prediction market is that market participants may try to manipulate the world to make their prediction market position profitable. (Link here to funny story.)
- Another limitation is in the setting of existential risk. A prediction market cannot usefully predict an existential event, since there is no way to collect on a correct bet in the prediction market after an existential event since market participants will not be alive to pay or get paid.