I Found a Signal. Here's Exactly How I'll Know If It's Real.
Pre-registering the test before the result, so you can hold me to it.
Most quant writers show you the signal after it works. I’m showing you mine before I know if it does.
That’s not modesty. It’s the only honest way to do this.
Six weeks ago I started building a machine learning system to find exploitable gaps in equity markets. I started in large caps, hit a wall. Institutional quants have more data, more compute, and more capital than I’ll ever have in that space. I pivoted to mid-caps, where analyst coverage is thin and expectations lag reality the longest. That backstory is coming next week. This post is about what I found last week and what it will take to convince me it’s real.
The Signal
My system scores stocks on expected returns using a composite of signals. The main one I’d been tracking, call it net alpha, started deteriorating in late June. Information Coefficient went negative and stayed there. The model was failing in real time.
While diagnosing it, I found something unexpected. A secondary signal called gap score, which measures the divergence between where expectations are and where price momentum is heading, was doing the opposite. When net alpha IC went to −0.243, gap score IC came in at +0.236 at the 20-day horizon and +0.240 at the 10-day horizon.
The mirror image. Almost exactly.
That contrast is either a genuine regime-conditional edge, a signal that shines precisely when the crowd’s momentum model breaks, or it’s a small-sample artifact that will dissolve as more data accumulates. Right now I have 6 matured dates at the 20-day horizon and 16 at the 10-day horizon. That’s a promising point estimate, not a settled fact. The overlap-adjusted t-statistic, which is the real test of statistical significance, currently reads nan at both horizons. It needs more independent, non-overlapping windows before it prints a real number.
So I’m not claiming I found alpha. I’m claiming I found a signal worth testing, and I’m publishing the test criteria before the result so you can hold me to it.
The Bar
I’ll call gap score a real, deployable signal when ALL of these conditions hold:
Direction persists. Gap score 20D IC stays clearly positive (at or above +0.10) as the sample grows past the current 6 clustered mid-June dates into genuinely independent observations.
It’s statistically real. The overlap-adjusted t-statistic reaches 1.96 or above at the 10-day horizon minimum. This is the gate. When the system prints a real number at or above 1.96, that’s the signal. Until then, it’s a hypothesis.
It beats the incumbent. Incremental-t at or above 1.64 versus raw alpha. Since gap score is the mirror of raw alpha, this should follow naturally if condition 2 is met, but I’m requiring it explicitly.
It holds on the real book. Engine 2’s shadow ledger shows positive live forward IC over 15 or more accumulated dates on the actual 225-name trading universe. This is the out-of-sample confirmation that keeps the first three conditions honest.
When conditions 1 through 3 are met, my system’s validator will print INTEGRATE, a literal green light built into the code, not a judgment call I make under pressure. Condition 4 is my additional out-of-sample check before I size up.
The Honest Timeline
Daily accumulation of independent observations resumed July 17. On current trajectory, I expect the 10-day significance test to resolve around late August to September. That’s a forecast, not a system-confirmed date. It could slip to September or October depending on whether the per-date IC scatter cooperates and the signal continues holding as the sample grows.
The 20-day significance test will take longer, likely October to November.
I’m not waiting silently for the green light. Every week between now and that resolution, I’ll document what the system is showing: the IC readings, the t-stat climbing from nan toward 1.96, the shadow ledger accumulating live observations. You’ll see the test in real time, not just the verdict.
The Caveat
Gap score looks good right now partly because momentum is reversing. It’s the mirror of a failing momentum model. That makes it regime-conditional. It’s unproven across a full market cycle. If momentum rotates back, gap score may underperform. I don’t know yet. That uncertainty is the whole point of the test.
Publishing “I found alpha” would be overclaiming. What I’m actually saying is: I found a signal that performs well exactly when the crowd’s momentum model breaks, I have a pre-registered set of conditions that will tell me whether it’s real, and I’m documenting the test publicly so you can watch it pass or fail.
That’s what The Expectations Gap is. Not a newsletter that shows you winning trades. A live test of whether a solo quant operator can find and validate a real edge in mid-cap equities, in public, with real capital on the line, before the result is known.
Subscribe if that sounds more interesting than the alternative.
“Why I Left Large Caps and What I’m Building Instead” goes out next week.
— Rick