What are the common mistakes in systematic models and infra that people make?

Building a backtesting engine and designing strategies are two very different things. If you wanna build a house, you shouldn't be designing nail guns. You should design a house.

How do I know if I'm doing one vs the other? Even when I'm looking at what others see as a nail gun, but from my POV it looks like a really well done house.

Avoiding leakage when training ml models

That's a reasonable place to start. Just build a good enough MVP so that you can start testing strategies. If you're a software engineer it's easy to get lost in designing the backtesting engine and make zero progress doing the thing that actually makes you money - designing strategies and running them live. Some people here spend 1900 hours designing the backtester and 100 hours designing strategies when it should be the other way around. Take something live to the paper server quickly so that you can troubleshoot your backtester.

Generally agree, but having a good backtester is part of the problem. The backtester and strategy building goes hand in hand. You need to think about both equally

Yes but bro is not gonna know what's a good backtester until he starts designing strategies or seeing how they run live, so as he makes new discoveries he needs to continually improve and build on the fly. It's a chicken-or-egg issue. He needs the experience to build the tooling.

backtesting engine is a good start but not enough - if you want a more minimum but easy to use backtesting engine, check out seeer ai, they may already have what you need.

The biggest one I keep seeing is treating the backtest like it is the exam, not the practice test. A decent system can still get mauled by boring stuff: survivorship bias, fees modeled too politely, fills that assume the market is a vending machine, and parameter tuning until the equity curve looks like it had a marketing department. My usual sanity checks are: - walk-forward or at least anchored OOS splits - costs and slippage worse than you expect - fewer knobs than your ego wants - paper/live shadow period before real size - logging every reject, partial fill, timeout, and manual override The model is usually only half the system. The other half is plumbing, and plumbing is where PnL goes to become a mysterious smell.

the cost stuff you listed is the part everyone reads about. the part that actually got me was accounting, not costs. my engine was marking open positions at their eventual exit price when it computed the equity curve. no future data in the signal, nothing wrong in the strategy logic, and the results were roughly double reality. it survived months of review because every individual trade was correct. the test that caught it: freeze the run as of a date in the middle of the sample, write the equity value down, then rerun the whole thing and check that date again. if the past changes when you add more future, something is reading forward. second one, corporate actions. i had ghost rows left behind after a split fix, so a chart showed a pre-split price sitting next to post-split prices and everything downstream inherited it quietly. if you are building your own store, put a check in that recomputes a few known split dates and screams if the ratio does not match. on your 1945 ambition, i would rather have 2005 to now with correct point in time fundamentals than 1945 with restated ones. old fundamental data is almost always as-restated, and that is a look-ahead you cannot see.

The bugs that usually survive the obvious fee/slippage checklist are timing and state bugs. I would make these explicit in the engine: \- Point-in-time universe membership and identifiers, including delistings, mergers, spinoffs, special dividends and ticker reuse. Adjusted prices are useful for research, but fills and cash flows must still use tradable prices plus the actual corporate-action event. \- Data availability time versus bar timestamp. A signal computed from today's close cannot fill at that same close unless you model a real auction order submitted before the cutoff. Also test holidays, half-days, DST, halts and stale quotes. \- One portfolio clock. Cross-sectional ranks must use the same information set for every symbol; missing bars must not silently leak a later observation or change the universe after ranking. \- A real order state machine: submitted, acknowledged, partial, cancelled, rejected, expired. Add participation limits, spread/impact that depends on liquidity, and deterministic handling when several strategies compete for the same cash or position. \- Short-specific state: point-in-time borrow availability, borrow rate, locate failures, recalls and forced buy-ins. A constant borrow fee misses the cases that matter most. \- Cash and financing: settlement lag, interest on cash/debit, margin changes and portfolio-level constraints. Taxes are often better kept as a separate investor-specific reporting layer unless the strategy itself is tax-aware. For validation, keep an immutable run manifest (data snapshot/version, universe definition, parameters, code commit, calendar and cost model), then build tiny known-answer fixtures for splits, delistings, missing bars, partial fills and margin rejection. Those tests catch more than adding another decade of history. I would trust a 1990-present point-in-time test with those invariants over a 1945 series whose membership and event history cannot be reproduced.

Look, from my experience — I've built a fair number of strategies, built portfolios, put together everything to go to production — the most typical mistakes, when it comes to technical infrastructure, are in how it's built. That is, if you're using a framework that's already built, like MetaTrader, and let's say you're using a strategy builder, there aren't that many errors, because you just build the strategy, take the code, and connect it to live. But if you're talking more professional, more hedge-fund level, where you use custom languages or builds instead of platforms because you need exhaustive control, that's where there are thousands of things that can go wrong and come back to bite you. Now, if you're talking about the research workflow, that's a completely different thing. Research is a protocol you have to follow, one you need to internalize and carry out so you can actually trust what you deploy. For example: find the hypothesis, test the hypothesis, stress-test it for robustness, run walk-forward windows to see whether the strategy stays stable across the years. Do you have a solid protocol for parameter selection? A solid protocol for selecting which strategies go live, and for how long? How much time do you spend in paper trading, how much in live? What do you do during paper trading, what do you do during live? What do you track? What rules do you have for cutting losses — do you have rest rules, or just a stop loss or a drawdown level that, once you hit it, you stop trading and start over? All of this comes down heavily to the experience you already have, or the experience you pick up along the way, until something works.

Almost all of them are versions of fooling yourself: overfitting to one regime, look-ahead and survivorship bias in the data, and backtesting without realistic costs and slippage. Most edges die the second you add fills and fees, so I'd build the cost model before the strategy, not after.