i checked how many stocks from the old s&p 500 you can still download, it's bad

most backtests start the same way, get a list of stocks, download prices from yfinance, run the strategy, the problem is the list, if you grab today's s&p 500 and test on 2010-2024 you're testing on companies that survived, everyone knows this in theory, i wanted the actual number so i rebuilt the index membership as it stood on two past dates using the public changelog (every add and drop is dated), then tried to download each member from yfinance today \- s&p 500 as of jan 2010: 506 tickers (dual-class listings make it not exactly 500), downloadable today 376, invisible 130, or 25.7% \- s&p 500 as of jan 2015: 502 tickers, downloadable today 401, invisible 101, or 20.1% one in four members of the most heavily covered index on earth, gone from the standard free source, and not just small names, the missing list includes allergan, celgene, cerner, alexion, broadcom's old ticker, baker hughes, CA technologies i ran the same measurement on india (top-500 as of 2015) and got 24%, basically the same number, the intuition that "us data is clean and emerging market data is messy" doesn't survive contact, yfinance loses the disappeared everywhere what differs is how they disappeared, every big us name i checked above left by acquisition, india's list is heavier on delistings and suspensions, and that difference matters for which way your backtest lies, a delisting takes the stock toward zero, an acquisition usually pays a premium, so survivor-only india data almost certainly flatters you, while for the us the sign of the bias is genuinely unclear because a lot of the invisible names exited UP, i've measured the coverage hole, not yet the return effect, that's the next piece caveats so nobody overtrusts this, membership rebuilt from the public changelog which i haven't independently audited, the probe ran in late july, and ticker recycling cuts the other way, some "visible" tickers are a different company wearing a dead company's symbol (S was sprint, now it's sentinelone), so my invisibility numbers are if anything too low practical takeaway, if your universe comes from "whatever yfinance serves when i ask for current constituents" your 10-year backtest is missing \~20-25% of the names that were actually there, and no walk-forward scheme fixes a hole in the data itself happy to share the reconstruction method if anyone wants to reproduce it

22 comments held Reddit says 0 on reddit ↗

u/noble_beggar 1 1 day ago

The direction point is the interesting one. If most of the missing US names left by acquisition at a premium, your backtest is understating returns, not flattering them. Opposite of what everyone assumes survivorship bias does. Ticker recycling seems worse than the invisible ones too. A missing ticker is a visible hole, a recycled one just quietly splices two companies into one price series and nothing flags it.
u/[deleted] 1 1 day ago

[removed] — already gone when the archive first saw it
u/MemeMan64209 1 1 day ago

Gotta use CUSIP or ISIN. Using the Ticker is dangerous.
u/B4SSF4C3 1 1 day ago

Even those have issues. SEDOLs tend to work best as far as public IDs go. My org has built internal ID systems to deal with continuity problems, with mapping to public and data provider ids. And STILL we’ll have the occasional issue with US/CA dual listings.
u/Finance__broski OP 1 14 hours ago

this thread is quietly the real lesson, tickers fail, then cusip/isin mostly work, then sedols work better, and an org that does this for a living STILL maintains a hand-curated dated id map and still hits edge cases, identity over time is the actual hard problem in equity data, prices are the easy part
u/stew1922 1 1 day ago

The ticker recycling is tricky. If you have the space for it, it’s almost better to pay for a month subscription from a data aggregater and download everything. Then use FIGI and CIK identifiers to filter recycled tickers. Even then sometimes they don’t carry a FIGI, and the CIK is misleading. So you have to align the dates the tickers were active in their “era” so you don’t combine two companies.
u/Bigunsy 1 23 hours ago

Norgate data has survivership bios free data, they give you what symbols were in the index at that time and have delisted symbol data. You have to pay but its worth it.
u/JonnyTwoHands79 1 20 hours ago

This is the route I plan on going. Have you had good luck avoiding survivorship bias with this? Related question - what criterion do you use to select the tickers to trade from the universe? Volume, Volatility, other metrics? I'm about to build this into my pipeline and could use some advice if you have it.
u/steelheaddan 1 18 hours ago

It’s not about having good luck, it’s about having good data. You can’t test a strategy with the current ‘Winners’ of the current sp500. You have to include what was in the actual sp500 during each day of the backtest period. Norgate is very good. If you only test your strategy on current sp500 members you won’t have an accurate backtest. You are literally looking into the future of what maintained its standing. You have to test all current and previous members of the sp500 universe to get an accurate backtest for what was the sp500 at the time, else it’s already severely overfit. First lesson of algo trading is backtesting what was in the index during the date you are testing not the current sp500 as it stands today. Same hold true for things that everyone knows such as the US stock market killed it vs other markets for decades. And that NDX beat SPX for 20 years. These known facts especially if you don’t account for past losers that dropped out, leads to severely overfit strategies - some even are simply subconscious decisions such as choosing the index you run your strategy on since you already know the outcome.
u/Finance__broski OP 1 14 hours ago

this is it exactly, and the index-choice point is underrated, picking ndx to backtest because ndx won IS the bias, just committed one level up
u/Finance__broski OP 1 14 hours ago

can't help with signal picks, but on the universe itself, three hygiene rules before any criteria, membership as of the test date not today, a liquidity floor so your fills are real (median daily value traded, not just volume), and a minimum history requirement applied point-in-time, the third one is sneaky, "at least 200 days of data" quietly deletes recent ipos from your past universes too
u/Bigunsy 1 13 hours ago

i have one strategy that works for me. Its a momentum based one that rotates once a month. My process is at the end of each month, get all the tickers in both the listed equities and de-listed (de-listed symbols are postpended with the date they were de-listed to disginuish between re-used symbols) equities database on northgate and then for each ticker check if its in S&P500 index at that time. then find the top X with the highest momentum. Sell any that are no longer in the top X and replace with the new ones. Then i have some portfolio sizing logic that decides what share of the capital each stock is assigned. I backtested that using norgate back to 2000 and have been running live since 2020.
u/Finance__broski OP 1 14 hours ago

yeah norgate is the right answer if you're paying, this measurement is really about the free path most people actually start on, the gap between the two is bigger than most beginners think
u/[deleted] 1 22 hours ago

[removed] — already gone when the archive first saw it
u/Automatic-Essay2175 1 21 hours ago

That’s why I pay for data
u/CalTechie-55 1 20 hours ago

How does this result in a backtesting error? If you're trading the s&p500 itself or one of its ETFs, its components don't matter. If you're trading individual stocks, the prices of the s&p will be irrelevant.
u/Finance__broski OP 1 14 hours ago

you're right that spy backtests are fine, the index provider handled the corpses for you, this bites the moment you pick stocks FROM the universe, which is most strategies here, "buy the 20 cheapest names in the s&p each month" needs to know what was in the s&p that month and what happened to the ones that later vanished, if your data only has survivors your screen was secretly picking from companies certified to still exist in 2026, that certificate wasn't available on trade day
u/MandingoPants 1 20 hours ago

can you not tie ticker to EIN or some bs like that?
u/Finance__broski OP 1 14 hours ago

EIN maps to the company not the listing, so it survives ticker changes but can't tell share classes or listings apart, and mergers make it messy, it's a decent extra key but the dated membership list is still doing the real work
u/idgafbroski 1 20 hours ago

Is there even value in backtesting such old market history? Happy to hear arguments against, but my feeling is that alpha is short lived and regime dependent. Something that worked 10 years ago is probably irrelevant today.
u/Finance__broski OP 1 14 hours ago

the hole isn't just in old data tho, even a 3 year backtest is missing 5-8% of names already, stocks die every month, so you don't escape it by staying recent and honestly the regime thing is kind of the argument FOR long history, with 3 years you literally can't tell if you found alpha or just one regime, you need the old data to check that, but the check only works if the old data still has the losers in it
u/skurrtis 1 12 hours ago

Don’t waste time developing a solution for this when sharadar nasdaq data is so cheap. You realistically only need to pay for it for a month