How to Verify an AI Stock Picker's Track Record
Every AI stock picker advertises a win rate. Almost none let you check it. The gap between a marketing number and a verifiable one is where most of the industry lives — and learning to tell them apart is the single most valuable skill when you're deciding whether a tool is worth paying for.
The four questions a real track record can answer
A track record you can trust survives four questions. If a picker can't answer all four with public evidence, treat the headline win rate as marketing, not measurement:
- Were the calls timestamped before the outcome? A prediction logged after the fact isn't a prediction. Look for calls dated in advance, ideally on an immutable public record.
- Is every call counted? The oldest trick in the book is quietly dropping the losers. A credible ledger shows wins and losses with the same prominence, and the count of settled calls should keep rising, never shrink.
- Is the scoring mechanical? “Win” has to mean one fixed, published rule applied to a real closing price — not a human deciding after the fact whether a fuzzy call “basically worked.”
- Can you audit an individual pick? If you can click a single ticker and see the exact call, the date, and how it settled, the aggregate number is checkable. If you can only see the aggregate, it isn't.
Where the numbers get inflated
Once you know what a clean track record looks like, the common distortions are easy to spot:
- Survivorship bias — losers are pruned from the list, so the surviving picks look prescient. The tell is a track record that only ever shows winners, or one where the total number of tracked calls doesn't add up over time.
- Cherry-picked windows — “92% since March” quietly starts the clock after a bad stretch. Insist on the full history, not a favourable slice.
- Moving goalposts — a “win” that's redefined per trade (“we were directionally right”) can't be falsified. One rule, stated up front, applied to every call.
- Paper vs. real pricing — backtests on frictionless hypothetical fills beat anything that has to survive a real bid-ask spread and IV crush. Forward, out-of-sample results scored against actual closes are worth far more than a backtest.
What a verifiable ledger looks like in practice
This is exactly why tickerseer publishes its prediction track record as an open ledger rather than a headline stat. Every options idea our two independent analysis engines publish is logged the week it's made, then scored mechanically against how the stock actually closed around expiration — wins and losses alike, nothing removed. You can open any single ticker to see the original call and how it settled, so the aggregate win rate is something you can audit rather than take on faith. That's the standard to hold every AI stock picker to: not “trust our number,” but “here is every call, check it yourself.”
When you're comparing tools, start from the track record and work backwards — a picker confident in its results makes them auditable; one that hides the ledger is telling you something too.