Paper 2 Live (What mistakes did your trading bot make that you didn’t expect?)

Fills by far are the trickiest thing when going from backtest to paper to live. I trade a lot of options strategies and filling at mid almost never happens (especially on less liquid underlyings). Took me a a lot of iterations (and time) to refine my fill model. Initially (and naively) started with mid fills, which is obviously way to optimistic and led to a lot of wasted time on strategies that looked much better in tests than they would actually fare live. Then moved to a flat % based spread slippage model which is better but still arbitrary and misses nuances of fast moving markets and differing regimes. These days it’s probabilistic, basically odds of a fill based on where I’m resting and how long I’m sitting there. It’s calibrated using trade and quote data. That one change did more for a backtest matching live than any other single engine change. It also helped cut down on order churn since I can quantify how much to move my order to increase fill % rather than arbitrarily walking it down until it fills. Nobody warns you about this either but the strategy logic almost never breaks, the plumbing does. Broker APIs will reject an order with a useless error code, ack one then go quiet, or drop the session between your open and your close. Rule I run now: the broker’s view of my positions is the only truth. Reconcile on every restart and on a timer while running, and assume my own state won’t survive a crash. Because eventually it didn’t.

I remember commenting once upon a time that I wasn't concerned with microstructure because I wasn't trading much faster than intraday. Today I would tell past me, microstructure absolutely matters, but it won't matter until you sort out all of the other issues.

IBKR paper gateway literally crashes for the most stupid thing but the live one works perfectly. So I spent months optimizing it for those crashes to then have to erase around 500 lines of code

Two from crypto side, both invisible on paper. Partial fills. On kraken they arrive as separate events sharing one order id. Our bot treated each event as the whole order, so after a partial fill it was managing a position size that did not exist. Found it when a stop closed more than we actually held. Fix - keep the running filled total in your own state per order id, and treat every fill event as an increment, not a snapshot. Paper never shows this because paper fills are always clean and full. Second - backtest bugs that only live trading exposed. We started on open source backtest libs and found too many bugs there, mostly lookahead, plus zero flexibility, so we ended up writing the engine from scratch - backtest and live now run through the same code, with a side by side log(execution log): every backtest entry next to the fill the bot actually took, exchange timestamps on both. When entries drifted apart, the log pointed exactly where the test was lying, and each bug shows up as a different time and price of entry. Without that diff we would have blamed slippage for all of it. Looking back, strategy logic never broke once. All the damage came from the bot having a wrong picture of its own position, or the test having a picture of the market that was too clean.