The Volatility Illusion
— Not a Good Signal, Just a Stock That Whipsaws
If a signal has a high hit rate, is the signal good, or did it just pick a stock that naturally moves a lot? Real measured data shows the latter effect dominates overwhelmingly.
Real Measurement — a Hit Rate That Splits on Volatility Alone
We split the entire universe into 4 quartiles by ATR (average true range) and re-measured each bucket's base rate (the rate of reaching +15% within 20 trading days). We didn't look at any signal at all.
Sample: 63,912 historical backfilled episodes (15,978 per bucket) · as of 2026-08-14.
| Volatility Bucket | Base Rate (+15% Reached) | Meaning |
|---|---|---|
| Q1 (lowest) | 4.0% | About 1 in 25 stocks |
| Q2 | 17.8% | |
| Q3 | 35.3% | |
| Q4 (highest) | 51.7% | About 1 in 2 stocks |
Same target (+15%), but the hit rate spreads from 4.0% to 51.7% — more than a 12x gap. And we didn't look at a single signal.
The Illusion This Creates
Now say some signal has the property of showing up often on high-volatility stocks. A volume spike, a gap up, a large breakout — that describes most surge-related signals.
Its hit rate then naturally comes out high. But that's simply the signal picking out stocks that already move a lot on their own — not the signal predicting the future. It's like reading "more people carry umbrellas on rainy days" as "umbrellas cause rain."
How to Filter This Out — Stratification
The fix is to compare within the same volatility bucket.
- Split the universe by volatility (e.g., into 4 quartiles).
- Within each bucket, compare the hit rate for stocks with the signal versus without it.
- Only accept a signal as real if it shows a consistent edge across every bucket.
If the lift survives even in the low-volatility bucket, that signal is more likely carrying real information, not just acting as a volatility proxy. Conversely, if it only shows an edge in the high-volatility bucket, that's just another way of saying "it picked high-volatility stocks."
Why This Trap Is So Common
- It produces a number that looks good — without stratifying, the hit rate comes out high, so there's no incentive to bother splitting it.
- The sample shrinks — splitting into 4 quartiles cuts each bucket's sample to 1/4, making it harder to reach statistical significance.
- Volatility isn't visible — just looking at a chart, it's hard to feel the volatility difference between two stocks.