The Trap in a Hit Rate's Denominator
— In-Progress Stocks Inflate the Win Rate
A hit rate is hits ÷ the denominator. But if you get who's included in the denominator wrong, the number comes out inflated even when every calculation afterward is correct. This article publishes a mistake DawnScan actually made and how we corrected it, with the real numbers, exactly as they happened.
What Happens When You Put "Not Finished Yet" Into the Denominator
Say a scanner picks a stock. Whether it wins or loses gets decided 20 trading days later. But there's a stock that's only 5 days in so far. Should it go into the scorecard?
"Include it if it's already hit the target, and judge it later if it hasn't" — that sounds reasonable at first. This is the trap. Among stocks picked during the same period, only the winners enter the denominator first, while the ones that haven't risen yet get excluded for being "not judged yet."
In clinical statistics, this structure is called immortal time bias. A sample that only enters the tally by surviving until the outcome is known has, by that very condition, already required success.
A Mistake DawnScan Actually Made
On 2026-08-11, we found the following rule in this site's hit-rate tallying code.
The middle condition was the problem. At the time of the correction (2026-08-11), the 210 in-progress picks being tracked broke down like this.
| In-Progress Pick | Count | Enters Denominator? |
|---|---|---|
| Already hit +15% | 40 | Yes |
| Not there yet | 170 | Excluded |
Of the 210 in-progress picks, only the 40 winners went into the scorecard, while the 170 that hadn't risen yet were excluded. After fixing the rule so that only picks whose 20 trading days had actually finished got counted, the numbers changed like this.
| Metric | Before the Fix | After the Fix |
|---|---|---|
| Cumulative hit rate | 39.8% | 30.7% |
| Volatility-matched lift (fixed +15% target) | 11.1%p | 2.9%p |
The two numbers above are the values as of the 2026-08-11 correction. We're leaving them as-is, not as the current figures, to show "what changed, and by how much."
Current values (as of 2026-08-14): of 259 matured picks, 73 hit the target → 28.2% (95% interval ±5.4%p). There are still 211 picks in progress, and of those, the 58 that have already reached the target and the 153 that haven't both stay out of the denominator — that's the whole point of the fix. You can check the latest value yourself at /track.
9.1%p disappeared. And the claim that "the scanner beats a random pick from the same volatility bucket" also lost statistical significance under this standard. The current numbers are published under the corrected standard on the hit-rate page.
Why a Sanity Check on the Math Doesn't Catch This
What's scary about this bug is that every calculation was correct. The confidence interval was computed exactly right, the multiple-testing correction was applied properly, and it reproduces down to the decimal point. No amount of re-adding the numbers reveals the error.
That's why, when you look at a performance statistic, you should check the population definition before the result number. Similar traps include survivorship bias (dropping failed stocks from the sample), multiple testing (picking the one that got lucky out of many trials), and overfitting.
3 Questions to Ask When Looking at Someone Else's Track Record
When you run into a "○○% hit rate" from a trading room, a YouTube channel, or a blog, asking these three questions filters out most of them.
-
How do you handle in-progress stocks?
If the answer is "we're still watching them," those stocks are likely excluded from the denominator. Check whether just the winners among them are already counted in the results. -
What's the denominator? Does it match the total number of picks made?
If 100 picks were made but the denominator is 60, ask where the other 40 went. -
Do you only count stocks whose window has ended?
Counting a mid-window target hit isn't a problem by itself. The key question is whether failures from the same period get counted too.
How DawnScan Handles This
- Only picks whose 20 trading days have ended go into the denominator. Even if a pick hits its target along the way, it isn't tallied before the window ends — because a same-period failure isn't in the denominator yet either.
- A delisting is treated as final and goes into the denominator regardless of elapsed time. Most get tallied as a miss, lowering the hit rate, but excluding them would create survivorship bias.
- If the same stock gets picked repeatedly, it's counted only 1x within a 21-day window (see repeat signals).
- This rule, and the fact that we previously labeled it incorrectly, are written directly into the body of the hit-rate page.
The reason we publish a correction that pushes the number down is simple. A number built by picking a favorable standard is advertising, not verification. What this site is trying to measure is the base rate, not an impression.
Frequently Asked Questions
Does this mean in-progress stocks aren't shown at all?
Doesn't that delay how recent performance gets reflected?
Doesn't a stock that hits the target early get shortchanged?
Can this bias show up in a backtest too?
How is this different from survivorship bias?
Related Reading
- Survivorship Bias — dropping failed stocks makes the record look better on its own
- Base Rate — without a comparison baseline, a hit rate means nothing
- Lift and Confidence Intervals — can you actually trust that number?
- Multiple Testing — test enough times and chance starts looking significant
- The Volatility Illusion — not a stock that rises well, a stock that moves a lot
Written and verified by: Dawn (about) — an individual developer, not a financial-industry professional. The figures in this article are real measured values from DawnScan's own tally, and the correction history is published together on the hit-rate page. Not investment solicitation (disclaimer).