We Tested 50 Signals and Adopted Zero
— Ending the Search
This isn't a post about a signal that works. It's a conclusion that 1 year 4 months turned up nothing -- and every number behind that conclusion, published as-is.
The last hypothesis — a market-regime gate, also invalid
Our most recent hypothesis was a gate that would only issue picks while the market
was in a "risk-on" regime. Using universe_snapshot.regime_state
(market-wide, keyed by date) we split picks by regime and compared each against the
base rate of the same regime's universe (measured 2026-09-05, source:
analyze_regime_picks.py, read-only).
| Regime | n (picks) | Actual hit rate | Expected (same-regime universe) | Lift | 95% CI | BH significant |
|---|---|---|---|---|---|---|
| risk_on | 159 | 28.9% | 26.1% | +2.8pp | 22.4-36.4% | N |
| neutral | 58 | 44.8% | 36.9% | +7.9pp | 32.7-57.5% | N |
| risk_off | 0 days observed -- never appeared even once across all 55 trading days | |||||
In both regimes, the 95% confidence interval contains the expected hit rate (the same-regime universe's base rate), and neither passes BH (FDR) correction. This result alone doesn't settle the cause -- as shown below, the risk-on sample (159) is underpowered to detect this lift. "We haven't found evidence of improvement in the current sample" is the accurate statement.
risk_off was never even observed
Even if we set a rule to pause picks during a "risk_off" regime, over the last 55 trading days not a single day was classified as risk_off. That means this rule would never once have triggered during this period. Even if the regime gate had turned out to be statistically valid, the risk_off clause would have been dead code that never did anything in practice.
"We don't know because there's no data" vs. "there is data, but we didn't find evidence of improvement"
Proving the risk-on regime's +2.8pp lift at 80% statistical power would require roughly 4,000 picks, but we only have 159. At our current accumulation rate (about 4.6 picks/day), reaching 4,000 would take 3 years 5 months -- practically undetectable on any useful timeframe. ★For this specific lift, that means "we lack the power to detect it," not "it doesn't exist."
What was rejected before this
On the way to the regime gate, hypotheses from several independent sources were rejected one after another.
| Hypothesis | Outcome |
|---|---|
| Earnings (EPS/revenue surprise) | Rejected -- revenue growth turned out to be a fifth confirmed volatility proxy (it looked like a signal, but was really just firing on high-volatility stocks) |
| Insider buying (Form 4) | Rejected -- CEO/CFO open-market purchases performed worst of all the directional signals we tested (-2.5pp) |
| 8-K filings | Rejected before starting -- a preliminary scarcity check showed the sample could never be large enough |
| ML model lift (2.29x) | Rejected -- it was volatility arithmetic. Once volatility-normalized, it reversed to 0.63x |
| ATR cap (T3) | Not adopted -- confirmed the gate's robustness, but added no extra benefit |
| Label preview (P12) | Held permanently -- didn't meet the adoption bar |
So why do we keep running the scanner?
We're not going to dodge this question. Once you control for volatility, every signal we've tested is statistically indistinguishable from chance. And yet we still calculate, every day, which stocks meet pre-surge conditions (volatility contraction, quiet accumulation, relative strength, and so on) and publish that list. These are two separate activities -- calculating and publishing conditions is not the same thing as proving those conditions beat the base rate.
We haven't proven the latter, and we don't hide that. The hit rate and confidence interval are published as-is on the Hit Rate page and the Base Rate page. We won't offer the unearned comfort of "but it's still useful as a reference." The scanner keeps running; what you make of its results is up to you.