How to Evaluate Any AI Stock Tool: A 10-Point Checklist

2026-11-12 ยท hedgewing.ai Research

The AI stock-tool market has grown faster than the standards for judging it. Most products show a backtested equity curve, a hit-rate figure, and a few testimonials, and most buyers cannot tell whether any of it would survive contact with live markets. The checklist below is the one we would want applied to hedgewing.ai, and it works equally on any competitor.

1. Is the evidence out of sample?

Ask whether the published results come from data the model was tuned on. In-sample results are not evidence of predictive power. Look for walk-forward testing, a held-out period, or best of all a timestamped log of resolved live predictions. A tool that labels its backtests honestly as in-sample and illustrative is being more truthful than one that calls the same thing a track record. See /glossary/out-of-sample-testing.

2. Are the probabilities calibrated?

If the tool reports confidence, ask whether a 70% call resolves true about 70% of the time, and whether the provider can show a reliability diagram. Confidence that never moves, or that is always above 80%, is decoration.

3. Can you see disagreement?

Single-number outputs hide uncertainty. Tools that expose the components, whether that is a per-model vote, factor sub-scores, or a forecast range, let you judge how fragile the headline is.

4. Is the methodology documented?

You should be able to learn what architectures are used, roughly what inputs they consume, how often they retrain, and how leakage is prevented. 'Proprietary AI' with no further detail is a reason to discount every other claim the tool makes.

5. Does the universe have survivorship bias?

Backtests built on today's index members exclude every company that failed. Ask how the historical universe was constructed. See /glossary/survivorship-bias.

6. Are costs included?

A short-horizon strategy that ignores spreads, slippage, and taxes can show a positive edge that is negative in practice. Any return figure should state whether it is gross or net of realistic costs.

7. Are return claims specific or vague?

'Average returns of 300%' without a period, a benchmark, a universe, and a dividend treatment is not a claim you can check. Credible providers either state all four or do not make the claim.

8. What does it say it cannot do?

Honest tools name their limits: coverage, horizon, data latency, event risk. A product page with no limitations section is describing a tool that has not been stress-tested, or a marketing team that would rather you not ask.

9. Is the pricing legible?

Credit systems, add-ons, and introductory rates that renew at multiples of the entry price are not disqualifying, but they should be visible before you sign up. Compare the renewal price, not the promotion.

10. Does it separate research from advice?

A research tool reports what its models see. An adviser tells you what to do with your money and is regulated accordingly. Tools that blur the line, with 'guaranteed' language or personalised buy lists from an unregistered entity, deserve suspicion.

Applying it to hedgewing.ai

For transparency: hedgewing.ai documents its four-model ensemble and its engineered feature set, reports calibrated confidence with a per-model breakdown, labels its on-demand backtests as in-sample and illustrative, makes no return claims, covers US equities only, and is not a registered adviser or broker. Where it falls short of the checklist today is the length of its resolved live-prediction history, which is still accumulating. Judge it, and every alternative, on the same ten questions. This article is educational and is not investment advice.

Create a free Hedgewing account to try hedgewing.ai yourself.