How I set up my pricer and execution engine for Kalshi’s 15min BTC markets

In my last post, I covered my bout with pricing the 15min BTC prediction market as an option \ [https://www.reddit.com/r/algotrading/s/ML4vQP2Ly1 \]. A few people asked how the system is actually put together, so here’s the architecture and why I landed on it. **Why not one script?** From the get-go, I knew I did not want a single script running everything. One bad data point or one connectivity drop shouldn’t take the whole system down with it. Likewise, one null exception shouldn’t crash the whole process. Once you split it up, you’re looking at a pipeline with at least 9 distinct steps: market discovery, market price ingestion, market data ingestion, data aggregation, pricer, gates, execution, position management, and logging to a database in parallel. That raises the first problem: how do you keep the state of each step in sync? **Problem 1: Message-based pipeline on Redis** I looked at a few options and settled on a message-based design using Redis as the transport layer. Every step is an input/output mailbox. It waits for messages on one or more input streams, processes them, and publishes its result. Each step gets its own independent behavior. For example, market discovery listens to the exchange for market openings and closings. Since Kalshi’s 15min markets are constantly rolling over, it publishes a state update on every event. The pricer waits until all the inputs it needs are present and fresh, checks timestamps for staleness, and only then runs and publishes a price for execution to evaluate. **The logger sits beside the pipeline, not inside it** The logger and database writer is one more subscriber, running in parallel to the whole pipeline. It listens to the same streams, writes everything to the database, and never sits in the path between a price and an order. If it falls behind or dies, trading carries on untouched. https://preview.redd.it/lcnzwtk5ohuh1.png… A nice side effect is that once the data is in the database, presenting it as a live chart and a dashboard is straightforward. It’s just reads on top of a table. hyhy.app dashboard with live streaming prices **Problem 2: Scaling beyond one of everything** Having a clean pipeline for one market and one asset is only half the job. The next question was how to extend it to different assets, markets, pricing models, and execution strategies without rewriting each step every time. I ended up with a factory-worker setup. Each step stays the same, and I spin up multiple instances with different parameters. The best example is spot price ingestion. Instead of one script subscribed to 5 exchanges, I run 5 instances, each pointed at a different exchange. Same idea for the pricer: one instance per asset (BTC, ETH), each subscribed to its own inputs. **What I got out of it** * **Horizontal scaling.** Adding an exchange or an asset means starting another instance, not touching the code. * **Fast recovery from failure.** Because each process is small and independent, one crash only takes out one step. A simple cron job that health-checks each process and restarts it is trivial to set up, and recovery is quick. Compare that to untangling a single monolith that died halfway through something. * **Isolation.** A bad feed from one exchange only affects the instance listening to it. **The downsides** None of this is free: * **Memory.** Running the same process many times over adds up.  * **Latency.** Every hop through Redis adds time. Having everything local helps. * **Stale quotes.** If a consumer falls behind, old messages can reach the pricer. That’s exactly why every consumer checks timestamps before acting. * **Reasoning about state.** With many independent processes, answering “what state is the system in right now?” is harder than with a single script. I think the tradeoff is worth it, but I wouldn’t claim it’s the right call for everyone. **Open to questions** Each stage has its own set of engineering decisions. Which one would you want me to go deeper on?