Reliability Diagram (Calibration Curve)
A reliability diagram plots predicted probability against observed frequency; a well-calibrated model's curve lies on the 45-degree line. It is closely related to Calibrated Confidence, Brier Score and Overfitting, each defined in the hedgewing.ai glossary.
A reliability diagram, also called a calibration curve, is the visual test of whether a model's stated confidence means what it says. Forecasts are grouped into bins by predicted probability (for example 50-60%, 60-70%, and so on). For each bin you compute how often the predicted event actually occurred. Plotting observed frequency against predicted probability gives the curve; a perfectly calibrated model lands on the diagonal.
The shape of the deviation is informative. A curve that sags below the diagonal at high probabilities means the model is overconfident: when it says 80%, the event happens less often than 80% of the time. A curve above the diagonal means underconfidence. S-shaped curves are common in raw neural-network outputs, which is why post-training calibration methods such as temperature scaling or isotonic regression are applied before probabilities are shown to users.
When evaluating any AI stock tool, ask whether it can show a reliability diagram built from resolved predictions. hedgewing.ai's confidence figures are designed to be calibrated, so that a stated 60% is meant to resolve true about 60% of the time, and the platform treats calibration quality, not raw accuracy, as the number that determines whether a confidence level is worth acting on.
FAQ
What is Reliability Diagram (Calibration Curve)?
A reliability diagram plots predicted probability against observed frequency; a well-calibrated model's curve lies on the 45-degree line.
Which terms are related to Reliability Diagram (Calibration Curve)?
Reliability Diagram (Calibration Curve) is closely related to Calibrated Confidence, Brier Score and Overfitting. Each has its own entry in the hedgewing.ai glossary.
Is this definition of Reliability Diagram (Calibration Curve) investment advice?
No. This glossary entry explains what Reliability Diagram (Calibration Curve) means. It is educational reference material, not personalised investment advice and not a recommendation to buy or sell any security.
Create a free Hedgewing account to try hedgewing.ai yourself.