I thought 78% accuracy meant I had something

One of my models was right 78% of the time. It was supposed to classify what kind of trading day was developing, and I reported the number as a useful result.

Then a reviewer asked to see what it was actually predicting.

The same category. Every time.

That category made up 78% of the sample. I could have skipped building the model, guessed the most common answer every day, and reported the same accuracy.

There was some information in its probability estimates, but the headline I'd used didn't show an improvement over the simplest possible guess. I'd checked whether the number was correct without checking carefully enough what it meant.

That was one correction in a research project that now fills 122 numbered notebook sections. I started it because I wanted to understand how certain traders read the market. Months later, I have five setups still being evaluated, a lot of failed experiments, and several conclusions I've had to walk back.

I think the process is worth sharing, especially if you've ever looked at a chart, noticed something that seemed obvious, and wondered whether you could trade it.

What I was trying to figure out

I kept coming back to a trader whose published plans and real-time commentary seemed consistently good. I'd watch price reach a level he'd marked and react, and wonder how he was getting there.

Could I write down enough of his process to test it?

I started with his plans, levels, and commentary. There was plenty to study, but I couldn't reconstruct all the decisions. A plan tells you what someone chose to write down. It doesn't necessarily tell you everything they noticed, or why they passed on one trade and took another.

Eventually I moved toward questions I could test directly. Does reaching yesterday's high tell me anything about the next hour? Does a fast move usually continue or reverse? If a trade starts losing money, does that tell me it's likely to keep losing?

Most of the work focused on SPY, the fund that tracks the S&P 500, with its futures counterpart, ES, alongside it. The price dataset goes back to 2002 at one-minute resolution, though the studies use different portions of it. Some of the joint datasets are much shorter.

I used AI to help write code, run tests, and organize the results. That let me investigate far more questions than I could have manually. It also meant I needed to keep track of every attempt. With enough freedom to change an idea, it wasn't hard to find a version that looked good on old data.

So I wrote down the rules before testing them, kept later periods aside, included trading costs, and saved the failures too. If I changed a rule after seeing its result, that counted as another experiment.

Six models, six losses to simpler rules

I expected machine learning to help find distinctions that a fixed trading rule would miss. Maybe two entries look similar, but the recent price action makes one more likely to work. That seemed like a reasonable job for a model.

Across five comparisons and a separate model for managing open trades, the simpler rules won all six times on data held back from development.

Some of the models were regressions. Others used gradient boosting or classified individual trading opportunities. The alternatives were often basic: take the qualifying signal, hold it for a set amount of time, and exit.

I still think the question was worth asking. The answer was that my models didn't add enough useful information to beat those rules. Building something more elaborate hadn't earned me a better result.

The level tests were disappointing in a similar way. Across 34,206 touches of familiar reference prices, including prior-day and opening-range levels, the average moves over the tested 30- to 60-minute horizons were all within about 0.02% of zero.

That didn't reproduce the clean reactions I'd noticed on charts. It also didn't test every way a trader might use those levels. It tested whether touching them, under the study's rules, was enough to generate the subsequent move I was looking for.

One test did help me understand why a trader's explanation and a statistical result might disagree.

The idea involved specific Fibonacci distances from the opening price. Price reaching one level was supposed to reverse toward another. I tested 22,340 events, then compared the proposed levels with nearby alternatives and evenly spaced grids.

The exact numbers didn't help. Price generally continued rather than making the proposed reversal.

But when I looked at the whole day, sessions that completed those back-and-forth trips behaved differently from sessions that kept moving in one direction. That gave me another question to investigate. The trader might have been noticing a real difference between types of days without the specific levels being what explained it.

I found that more interesting than simply marking the idea wrong and moving on.

A useful forecast that didn't improve the trading

Another study asked how much we could know about the rest of the day by 10:30 in the morning.

The models could predict some of the variation in the remaining price range. They didn't show comparable success predicting the remaining return. In plain terms, they were better at estimating how much room prices might cover than where prices would finish relative to where they were now.

That seemed useful. If I expected more movement, perhaps I could use that information to decide when to trade or how much to risk.

I described it as a sizing signal before testing that next step.

When challenged on it, I applied a fixed version of the forecast as a filter to an existing strategy's trades. The apparent benefit changed sign between the validation period and the later test period.

The forecast still contained information. The particular trading use I'd tested hadn't demonstrated a consistent benefit. I corrected the report because I'd treated those as if they were the same result.

The review also caught a claim about whether VIX, an options-based measure of expected market volatility, leads SPY during the day. I had called the hypothesis rejected on a joint dataset covering only 21 sessions.

There were thousands of minute bars, but only 21 days. Closely related observations from the same session don't give you thousands of independent tests. The appropriate conclusion was that I didn't have enough evidence.

I had rules intended to prevent these mistakes and still made them. Having the notebook wasn't a substitute for someone looking closely at what I'd claimed.

What was left

Five intraday setups remain in the research book. Most look for movement to continue under specific conditions, including a fast push, a breakout on an active day, or a new high reached without much back-and-forth movement.

Several came from questions where I'd expected the opposite. I was looking for evidence that a move was getting tired, and the tested conditions instead pointed toward continuation.

One setup had a reported win rate of about 58%, with roughly 0.05% net per occurrence. That's small enough that execution and costs matter. A win rate alone wouldn't tell you whether it was worth trading anyway; the size of the wins and losses matters too.

These setups passed the research checks used to select them. They still need forward evaluation. I don't have a demonstrated live track record for the five-signal system.

I did run the tools together during one August paper-trading session. There were three discretionary trades, and the total came to negative $1.02 at the simulated sizing used.

That session let me inspect the workflow while prices were arriving in real time. It wasn't a clean test of the five systematic setups: I was making discretionary decisions using several inputs. Those trades need to stay separate from the system's forward record.

Someone else's strategy complicated my conclusions

I also rebuilt a published intraday SPY momentum strategy using independent data and code. It reported a Sharpe ratio of 1.33 over seventeen years. My replication produced 1.55 over the same period. Sharpe measures return relative to its variability, so this was a useful comparison, though it wasn't enough by itself to verify every detail of the replication.

After the paper's sample ended in April 2024, the replication's Sharpe fell to 0.77. The later results included a strong 2024, a flat 2025, and a slightly negative 2026 through the available data.

Performance had weakened. I couldn't establish from that sequence whether publication caused it, whether market conditions changed, or whether something else explained it.

Its short trades, which profit when prices fall, were also contributing to the later results. That mattered because the short setups I'd developed hadn't survived my own tests.

There were differences in how the strategies worked. Several of my failed ideas bet against rising prices. The published strategy followed downward momentum under volatility conditions and exited when price crossed back through a defined band.

That also qualified my exit findings. I'd tested 51 rules for cutting trades after they moved against me, and none improved on the scheduled exits in those setups. It would have been easy to turn that into “stops don't work.” But this other strategy used a different exit mechanism as part of a different entry system.

My tests gave me a reason to reject particular rules, not a reason to stop investigating every strategy that used the same word. They also didn't settle how much loss someone should be willing to carry.

The next tempting result

More recently, I found twelve sessions where price broke below the premarket low before breaking above the premarket high. All twelve closed red.

Twelve out of twelve is the kind of result that makes me want to keep digging. There's a plausible explanation too: sellers push through the overnight low first and maintain control.

But I'd found it while searching through patterns. Twelve examples and a convincing explanation don't tell me how it will perform on the next twelve.

It's now one of six patterns being tracked on new sessions. Each has a fixed definition and criteria for keeping or retiring it. The observations used to discover it don't count toward passing the next test.

That waiting is a part AI can't remove. It can help me explore another hundred variations today, but tomorrow's market data still has to arrive tomorrow. If I keep changing the rules while watching the new results, I won't learn whether the original pattern held up.

Where this leaves me

I wanted to understand what those traders were seeing. I haven't reproduced a discretionary trader's whole process, and I haven't established that the current system can make money live.

I have narrowed down what I'm working on. When another level or exit idea catches my attention, I can check whether I've already tried it, what the comparison was, and why it failed. Sometimes the new version is meaningfully different. Sometimes I'm asking the same question again because I didn't like the answer.

Using AI made the research much faster. The corrections made me more careful about the conclusions. I still need to choose the questions, inspect the tests, and notice when a result isn't answering what I originally asked.

For now, I'm continuing the forward records for the five setups and six newer patterns, with discretionary trades kept separate. I'm also watching how the replicated strategy performs as more data comes in.

I'm particularly interested in whether the remaining ideas survive a period where I leave their definitions alone. That's the next result I want to write about.

Technical results and scope

These are the figures behind the examples above. They come from separate studies with different samples and should not be read as one combined backtest. Basis points are used below: 100 basis points equals 1%.

The five retained setups are first strength, a 30bp momentum thrust over 15 minutes held for an hour, a confirmed trend flip entered three minutes late, a first upward break of the two-hour opening range conditioned on a premarket volatility forecast, and an efficient new high where price moves ahead of its volume-weighted average. These descriptions omit implementation details and aren't executable specifications.

The research protocol includes prewritten hypotheses and thresholds, chronological holdouts, recording holdout reuse, session-level aggregation for statistical tests, era breakdowns, five-year half-life weighting with effective sample sizes, and baseline and stressed trading costs. Search corrections and the exact sample construction need to be checked in each experiment's record. A list of procedures isn't evidence that every conclusion is sound, as the corrections above show.

The framework, code, and experiment records are in TradingDesk, excluding licensed market data and fitted coefficients. These findings describe an ongoing research project, not a verified live investment track record.