Im currently building a trading bot farm using Claude Opus 4.8, and i need some advice...

So i have the farm mostly built, and i am currently in the strategy testing phase. ive now tested 20 strategies on ES/MES/SPX and have had zero be promoted to my paper trading trial, which would run for about 1-2 months to ensure that it wasnt a curve-fitted bot. ive had several profitable bots, however my Farm killed them due to underperformance compared to the underlying, fees and spreads ate all the profits, or only performed well in a specific market regime. im 2 weeks into this project, and havnt really made any progress for over a week. here is a brief workflow rundown of how my bot runs it trials...

1. Data foundation — Databento history → back-adjusted continuous futures (roll on liquidity, additive/Panama method); integrity checks (bad-tick/gap/timezone); options marks cleaned via a point-in-time symbology join. Clean, realistic, reproducible inputs — garbage in = fake edge.

2. Strategy contract — every bot is a pure function (bar/chain → order intents); it never sees size, the clock, or the broker. The identical code runs backtest and live — "what you prove is what you run."

3. Backtest engine — no-lookahead replay clock, pessimistic fills, real MES/SPX costs, ruin floor + margin-liquidation check. One honest single-run P&L.

4. Walk-forward + sealed holdout — rolling train/test windows; the final \~1yr is locked in code, scored exactly once. Out-of-sample discipline you can't iterate against.

5. Robustness sweep + regime breakdown — re-run across many window sizes; label each window bull/bear/high-vol/low-vol. Kills edges that only work at one lucky setting or in one regime.

6. Monte Carlo + Kelly sizing — block-bootstrap with an injected fat-tail shock; size off the lower confidence bound of the edge, not the point estimate. Ruin-aware sizing.

7. Validation gate (promote only if ALL): (a) out-of-sample edge statistically significant, deflated for the number of bots ever tested (Šidák); (b) robust across windows + regimes; (c) beats buy-and-hold risk-adjusted, after costs; (d) survives the sealed holdout. One strict promote/reject rule.

8. Research journal — append-only SQLite; every backtest auto-logs as a trial, supplying the honest denominator for the multiple-testing deflation above + tracking stage attrition. No forgotten failures inflating the hit rate.

Two asset classes, one pipeline: a futures engine and a parallel options engine (multi-leg structures, chain-aware fills that cross the real spread) both feed the same gate + journal.

Deliberately NOT built yet: live execution, forward paper-trading (the intended real final gate), any self-improvement/auto-tuning, and a UI. A pass today ends at "clears the gate → would go to paper."

Can i get advice on how i should continue to approach this project? im struggling to find strategies that aren't already oversaturated to the point the edge no longer exists, so where should i be looking to find profitable strategy? any and all advice / criticism is greatly appreciated! also feel free to ask questions for anything unanswered that would be helpful for you to help. *this project is for personal use, im not selling this or giving it away, so if you have an existing strategy for any market that is profitable that you'd like to share, id be extremely grateful if you send it in my DMs, and i promise that the strategy wont be marketed, and wont ruin your edge.*

44 comments held Reddit says 44 on reddit ↗

u/LegendOfTheNoob 5 2 days ago

Keep researching and testing ideas. Once you get your basic data infrastructure together, you'll spend more time researching and backtesting. Part of that pipeline is reading and feeding ideas. I spent a few months iterating my infrastructure (data layer and rules layer). Once that got to a point I liked and trusted, I started cranking ideas through it at volume and scale. Edge can be tough to find and very conditional. Most things fail my testing and evaluation. The few things which were ultimately promoted to live trading are profiting, will stay flat/no trade as needed, and are so far less profitable than my backtests... but they are working so far. In some cases a low number of trades, in others, thousands of trades. The vast majority of my time is spent reading and dreaming up strategies and ideas.
u/Trikotic OP 1 2 days ago

thats where im stuck at, just feeding it more ideas. im also trying to build a 'connections' framework for my AI to be able to connect, research, build, and test ideas automatically. im hoping it be able to look through the data and find synergies in old backtests to build new and improved strategies.
u/Brilliant-Dig3435 1 1 day ago

Just be careful with the type of information you feed it; otherwise, they will probably start to fit based on your trading engine and learn to crack it over time. For my swarm, 9-10 cycles was a good optimum in the past. But I can tell you that this is gonna be another very deep rabbit hole to dive into
u/Psychological-Try882 7 2 days ago

20 is rookie numbers. Try 500+ ,but yeah Claude is good.
u/kush_patil 69 2 days ago

Honestly, zero out of 20 passing that pipeline doesn’t sound like failure to me. With costs, walk-forward, regime checks and a sealed holdout, most ideas probably *should* die. The one thing I’d be careful with is repeatedly generating strategies until something passes. Even with a sealed holdout, the whole research process starts becoming a search over thousands of variations. I’d rather have 100 boring failures properly logged than one great-looking backtest I can’t explain.
u/[deleted] 1 2 days ago

[removed] — already gone when the archive first saw it
u/AutoModerator moderator 0 2 days ago

Your post was removed under **Rule 2 (high-quality questions only)**. Generic “which data vendor should I use?” posts usually lack the detail needed for meaningful discussion. **Commonly used market data providers:** - **Yfinance** - **Massive.com** - **Databento** - **FMP** If you repost, please include details such as: - asset classes and markets - symbols or venues - historical vs real-time - granularity and depth - licensing or redistribution needs - latency expectations - budget constraints *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/algotrading) if you have any questions or concerns.*
u/Trikotic OP 1 2 days ago

i didnt even ask that question???
u/rduser 1 1 day ago

get rekt
u/MerlinTrashMan 14 2 days ago

This guy algos
u/EvilPencil 1 1 day ago

The search process itself can become a source of lookahead bias.
u/CompetitiveStoic 4 2 days ago

Solid pipeline. A couple of recommendations: - Don't focus too much on beating buy-and-hold. Buy and hold Sharpe is anyway a low bar, but your strategy may be short skewed, complementing buy-and-hold. - Don't penalize parameter sweeps in development by deflating for each combination swept. Save a search, and deflate for the search, not each test under the search. - Add a human override to every gate. Every structured pipeline like this has a considerable probability to reject a true Sharpe 1.0 strategy, so you should be able to take risks as business owner. Incubate with small contract sizes if you "believe" in the strategy but the pipeline says no. And yes, ideas will be your bottleneck. Good luck!
u/Trikotic OP 3 2 days ago

this is great advice, thank you. the manual overide was the original setup i had, but Claude was like "nah, youre going to f it up" lol.
u/Jtex1414 1 2 days ago

I made my entire platform and iterated my strategy with claude. I like to use Fable (high) when theorycrafting, reviewing backtests, planning out builds, etc. For doing the more repetitive stuff, and building the things (that fable designed), I usually use opus (high). Use Cowork with the desktop app.... I've come to like chatgpt's sun model as well, and have been using that (with codex) more too.
u/Trikotic OP 1 2 days ago

good to know. so use Fable 5 for strategy researching and backtests reviewing, and improving, while use opus for app/ui building and infrastructure building?
u/Jtex1414 1 1 day ago

Yes, think of fable as the researcher/designer, and opus as the builder/worker. Fable has higher costs, so you don’t want to use it for everything.
u/s_lw0 3 2 days ago

i would not treat the 20 failures as no progress. the useful thing is whether the rejects are dying for the same reason. if fees and spread kill most of them then i would stop searching entries for a bit and test holding period liquidity and order type assumptions first. if they only work in one regime then the next search should be regime specific instead of trying to find one general strategy. the farm sounds solid but i would add a reject taxonomy before adding more strategy volume because otherwise 500 failed tests just tells you less than it could
u/Trikotic OP 2 2 days ago

i do have a reject taxonomy, but not as good of what you described. i have it give reasons for its rejection and an overview of its performance, but that taxonomy doesnt really go anywhere except in its knowledge base, so building a series of regime-specific strategies as a selectable arsenal is a really good idea for greater diversification and robustness. thank you
u/shironekoooo 1 2 days ago

I would be carefull about p hacking even if you test your strategies from your gaunlet a trash strategy can still pass. I would reccomend you look at bootstrapping and hansen spa test
u/Roharcyn1 1 2 days ago

I am not sure if this is the right approach and just started building mine. But was going to leverage vectorbt for initial parameters tuning and testing before moving towards a back test engine.
u/SadPhone8067 12 2 days ago

You’re assuming you can’t use bots that only work in a specific regime….you can. If you have three bots that all work in different regimes then all you gotta do is find out how to classify the regime and only allow the bots that work in that regime to trade.
u/Trikotic OP 5 2 days ago

another commenter pointed this out, and im actually currently working on this exact thing. a regime-specialized arsenal. this is actually been the biggest breakthrough since i got it built. now, i just got to figure out how to identify regime change. got any advice for identifying regime change?
u/Arty_Puls 1 1 day ago

Identifying regime change is literally what people spend their lifetimes on. That's the money maker lol. There's really no way to truly recognize a regime change. You can use volume based ATR to try to gauge if volume is picking up while bouncing off a support, or maybe you use bollinger bands. Higher/Lower time frame signals. It's a rabbit hole man. Good luck. I couldn't find too much worth a damn. Instead of worrying about timing the exact market structure shift, find a formula that captures the majority, but is able to identify when it's not in a true market structure change and cut losses early. That will be the real deal maker
u/fourohfournotfound 1 1 day ago

statistical jump models, a few types of hmm. or to hell with that and let the market do with by just using vix. vix is better than most regime methods I know.
u/roztok_potok 1 2 days ago

I doubt you can find single asset strategy. Like one dedicated for ES/NQ. This is very very hard to find.
u/Trikotic OP 0 2 days ago

well, im building a farm for strategies, im going to have many strategies on many markets, but right now im focusing on ES/SPX so that way i can justify spending more money on other market data.
u/roztok_potok 1 1 day ago

Read my post again.
u/Bonkers24-7 4 2 days ago

I wouldn’t treat 0/20 passing as failure by itself. With a strict enough validation gate, most ideas probably should die. The thing I’d check before generating more bots is why they failed. I’d bucket each rejection into something like: no gross edge, edge disappears after fees/spread, regime concentration, sizing/liquidation issue, live/backtest mismatch, or not enough sample. If all 20 fail for different reasons, that’s probably just the graveyard doing its job. But if most of them die for the same reason, then the useful discovery may be in the failure pattern, not in the next strategy idea. Do you know whether most of the failures were “no edge at all” failures, or “looked okay until costs/fills/regime split” failures?
u/Trikotic OP 2 2 days ago

i do have a reject taxonomy record. here it is: Failure mode │ Count │ Share │ │ regime\_dependent │ 7 │ 44% │ │ cost\_killed │ 7 │ 44% │ │ beats\_passive │ 6 │ 38% │ │ underpowered │ 6 │ 38% │ │ no\_edge │ 1 │ 6% │ │ holdout\_decay │ 1 │ 6% │
u/Bonkers24-7 1 2 days ago

That taxonomy is actually more useful than the 0/20 number. The one thing I’d separate is primary failure reason vs secondary tags, because the overlap changes the conclusion. For example, if a strategy is both regime_dependent and cost_killed, I’d want to know whether costs killed a real edge, or whether the “edge” only existed in one high-turnover regime where friction was underestimated. I’d probably turn this into a root-cause table: primary fail reason secondary fail tags gross expectancy before costs net expectancy after costs best/worst regime sample size whether it beat passive after costs whether paper/live behavior matched the backtest Then I’d look for clusters. If 7 are cost-killed for the same reason, that’s not seven unrelated failures — that’s one execution/economics assumption telling you something. Same with regime dependence. The useful question might be: “what failure mode keeps repeating across unrelated ideas?” If the same failure keeps showing up, the bottleneck may be the validation framework or market assumptions more than the next bot idea.
u/Trikotic OP 2 2 days ago

this is a good improvement step. ill add this.
u/Personal_Ad9395 1 2 days ago

Try hmm for regime
u/Trikotic OP 1 2 days ago

hmm?
u/Personal_Ad9395 1 1 day ago

Be careful while turning it , if it's very much prone to overfitting .
u/Known_Grocery4434 1 1 day ago

I stopped reading when it said "Farm". You need to get just one and that's hard enough
u/ConfidentEmergency33 1 1 day ago

Good luck on think journey. You are almost there. Just take a break and come back. You are closer than you think!!! DM me if you are loosing hope.
u/No_Administration444 1 1 day ago

Hey I’m on a similar journey maybe we can compare notes see where we each are
u/rduser 1 1 day ago

sorry to be bearer of bad news but algo trading is a scam. if you're not an HFT shop with at least 500 million in assets you're not going to make money period.
u/Independent-Boot-706 1 1 day ago

I m doing the same thing but I have tried more than 200 different strategies and so far only 4 have passed the grade. Bot has been paper trading for 2 weeks now and is looking good on small sample. Make sure any strategy that passes runs through the gauntlet of shuffle test, PBO and DSR test before calling it passing. I am trying to find other strategies that can complement the 4 strategies so I can increase trade count as it only averages around 5 trades a month.
u/Trikotic OP 1 1 day ago

interesting concepts. i have a pretty heavy anti-curve-fitting system already including 1-2 months of paper trading after passing before promoting to live, but this should be a good addition to increase robustness.
u/Independent-Boot-706 1 1 day ago

Also make sure the fill logic is realistic in the backtest that includes slippage
u/[deleted] 1 1 day ago

[removed] — already gone when the archive first saw it
u/AutoModerator moderator 1 1 day ago

Your post was removed under **Rule 2 (high-quality questions only)**. Generic “which data vendor should I use?” posts usually lack the detail needed for meaningful discussion. **Commonly used market data providers:** - **Yfinance** - **Massive.com** - **Databento** - **FMP** If you repost, please include details such as: - asset classes and markets - symbols or venues - historical vs real-time - granularity and depth - licensing or redistribution needs - latency expectations - budget constraints *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/algotrading) if you have any questions or concerns.*
u/dombrogia 1 19 hours ago

Think about why they failed — not at a micro or config / param level but from a macro market perspective. Understand the market (or at least try to make sense of all the noise) and have a plan against that. I’ve felt the same working with LLMs on algos — IME the need a ton of direction and it doesn’t really matter the model you use it matters the guidance you give it. They wont give you answers but they’ll help write you solution and polish some things you’d half bake