Brier Score
The Brier score measures the accuracy of probabilistic forecasts as the mean squared error between each predicted probability and the actual 0-or-1 outcome. Lower is better: 0 is a perfect forecast and higher scores indicate worse calibration.
See Brier Score in real trade signals
Tradewink uses brier score as part of its AI signal pipeline. Get signals with full analysis — free to start.
Explained Simply
For a single yes/no forecast, the Brier score is (predicted probability − outcome)², where the outcome is 1 if the event happened and 0 if it did not. Averaging that across many forecasts gives one number between 0 and 1. It is the standard way to check whether a forecaster who says '70%' is actually right about 70% of the time — good calibration keeps the score low. Because it rewards both accuracy and honest probabilities, the Brier score is widely used to compare forecasting models and to detect over- or under-confidence.
How to Calculate a Brier Score
Take each forecast's predicted probability, subtract the realized outcome (1 for happened, 0 for not), and square the difference. If you forecast 0.80 and the event happens, the score for that forecast is (0.80 − 1)² = 0.04. If it does not happen, the score is (0.80 − 0)² = 0.64.
Average those per-forecast scores across all your predictions to get the overall Brier score. Confident forecasts are rewarded when right and penalized heavily when wrong.
Brier Score and Calibration
A low Brier score means predicted probabilities line up with reality over the long run — the essence of calibration. It exposes overconfidence: a model that says '95%' but is right only 70% of the time will score poorly.
This differs from simple accuracy, which only checks whether the more-likely side happened. Two models can share the same accuracy while one is far better calibrated, and the Brier score is what separates them.
Frequently Asked Questions
What is a good Brier score?
Lower is better, with 0 being perfect. Context matters: for evenly-matched coin-flip events, always predicting 50% yields a Brier score of 0.25, so a useful forecaster should score below that baseline.
How is a Brier score different from accuracy?
Accuracy only checks whether the predicted direction was right. The Brier score also rewards well-calibrated probabilities, penalizing a forecaster who is directionally correct but wildly over- or under-confident in the numbers.
Why does calibration matter for prediction markets?
Prediction-market prices are themselves probabilities. A calibrated forecast, tracked with a Brier score, helps you judge whether a contract's price looks fair, too high, or too low relative to the true odds.
How Tradewink Uses Brier Score
Tradewink Predictions tracks the Brier score of its probability estimates to keep them calibrated — so that a stated 70% chance corresponds, over many events, to roughly a 70% hit rate. This is an internal accuracy and model-quality check for educational transparency, not a performance guarantee.
Save a signal preview for later
Get a concise AI signal example in your inbox, then build a watchlist when you are ready. No spam, unsubscribe anytime.
Related Terms
Learn More
Is Kalshi Profitable? What the Data Actually Shows
An honest, data-backed look at whether you can make money on Kalshi — what the research shows, why most bettors lose, who actually profits, and what a disciplined positive-EV approach requires.
Tradewink Predictions: Autonomous AI Prediction Market Trading
Learn how Tradewink Predictions uses multi-model AI to trade binary event markets on Kalshi autonomously — from probability estimation and Kelly criterion sizing to settlement arbitrage.
Kalshi Trading Strategies That Actually Have an Edge
The Kalshi trading strategies with a real, evidence-based edge — model divergence, favorite-longshot bias, settlement timing, and fractional Kelly sizing — plus an honest look at why most bettors still lose.
See Brier Score in real trade signals
Tradewink uses brier score as part of its AI signal pipeline. Get daily trade ideas with full analysis — free to start.