I tested my discretionary strategy against randomized markets (Part of the edge survived).

On top of automated stuff, I also trade a discretionary strategy based on structure, zones, and divergences. Last weekend, I finally tested it properly, and I wanted to share the process because I think this kind of validation is something a lot of traders don't do. **Step 1: getting Claude to understand my strategy** I had previously recorded videos explaining my discretionary method, so I took the raw transcripts from six of them and gave them to Claude Code. It understood the overall logic surprisingly well and wrote the code for it. It even picked up rules that I had only mentioned once in passing. **Step 2: checking that it didn't make things up** Before trusting any results, I asked it to export a CSV of everything it detected (zones, structure, invalidation, state changes). I then checked the output against my charts line by line. The detections matched what I would have marked manually: same zones, same breaks, same points where a zone became invalid. That was the point where I felt comfortable actually testing the strategy. **Step 3: creating fake markets** Then diit suggested something I believe most people never think of: null tests. The idea was to take all the historical candles I was testing, keep each candle exactly as it was, including its body and wicks, but shuffle the candles into a random order and rebuild the chart. So visually, you still get something that looks like a market. But the chronological relationships have been destroyed. What happened before no longer contains information about what happens next. Claude gave me an analogy I liked: "It's the sugar pill in a medical trial. If a drug cures 60% of patients, that's great... unless a random sugar pill also cures 60%." **Step 4: testing my entries against the randomized markets** I ran my entry rules on the shuffled markets, with up to 50 different shuffles per test, and compared those results with the real markets I trade: mainly NQ and BTC. The exact same exit logic was used in both cases. Across every timeframe I tested, my setups produced roughly 5 to 8 percentage points more winning trades on the real market than on the shuffled versions.The largest test simulated around 18,000 trades. So at least according to this test, the entries were detecting something that existed in the real sequence of market prices and wasn't reproduced simply by rearranging candles randomly. That was a relief. **Step 5: testing everything else** This is where things became much less comfortable. One common claim among "market structure" traders is that after a break of structure, price continues in that direction around 60% of the time. I tested it. Then I tested the exact same thing on the shuffled markets. The result was basically identical. So that 60% continuation rate seems to come largely from the way the levels and breaks are defined rather than from some special predictive market behavior. That was probably the most interesting result for me. Then I tested trade management (breakeven stops, taking partial profits, etc.) All of them reduced expectancy. The version with the highest expectancy was basically the dumbest one: a set-and-forget approach, and leave it alone. I also tested whether the edge transferred to other markets. It didn't transfer nearly as well to gold or S&P futures. Proof that an edge can apparently be much more instrument-specific than most people assume. Then I tested the strategy on the 20 biggest stocks. The real-market version was profitable and initially looked pretty convincing. But then I ran the same test on the shuffled versions of those stocks. The randomized version made roughly three times more. In other words, what initially looked like an edge was actually worse than the null control. I also went through a few side quests during the session. For example: \- Do support/resistance zones actually weaken after being retested multiple times? In my tests: no. \- Is there an edge in taking mean-reversion trades after price sweeps a zone? \- How much of apparent "market structure" performance survives randomization? \- Which parts of the strategy remain useful when you separate entries from trade management? **What I took away from it** The biggest thing was that several ideas I had considered part of my edge apparently weren't. My entries seem to contain real information on the markets I actually trade. But some of the "market structure" statistics I believed in also appear in randomized markets. And my trade management was actively reducing expectancy. The whole exercise made me realize how easy it is to backtest something, see a positive result, and conclude that you've discovered market behavior when the exact same result might appear in a control dataset where the temporal structure has been destroyed. I'm curious how people here approach this. Happy to discuss my backtesting session.