Skip to content
Archive
← r/algotrading
6
88%
u/Grand-Fly-6090 1 day ago Research Papers

Using bar charts as inputs?

Here is a machine learning paper where they are using bar charts as inputs to a CNN model. I have more questions than answers when I read this paper. Are any of you using bar chart images as inputs to models? Seems like an awkward way to convey price history data. Kim, Taewook, and Ha Young Kim. "Forecasting stock prices with a feature fusion LSTM-CNN model using different representations of the same data." *PloS one* 14.2 (2019): e0212320. https://preview.redd.it/etf15d7kxmuh1.png…
27 comments held Reddit says 20 on reddit ↗
  1. u/GurglingGarfish 6 1 day ago
    No. I thought about it once, but it’s just a visual representation of your historic OHLCV, so why not just feed the numbers in themselves?
  2. u/Grand-Fly-6090 OP 1 1 day ago
    Agreed. Also, I'm not sure how the CNN model interprets the scale of the plots.
  3. u/BeyondGITSandBots 3 1 day ago
    This. I cannot agree more. If you think about what your data really is: Why use 64x64 (or 128x128 or whatever low resolution that necessarily has to be equal or higher than your raw data series in terms of time steps and price bins) data points and an overly complicated slow to train model instead of clean low dimensional raw data (just a few raw series and possible transformations) that underlies the image? It's really just like taking an unnecessarily high-resolution/high-dimensional representation with lots of added waste and empty data. Think of analyzing a high-res image of a screen showing "1+1". Can you represent what should be a few bytes in machine-interpretable clean data in noisy, messy, bloated and mostly empty/sparse 3MB data? Yes, you can. But why would you? If you really want, use at temporal RNN, CNN or other more state-of-the-art time series networks (that nowadays allow for internal multi-timefram convolution and feature interactions, see sktime, nixtla and other time series frameworks, using models like e.g. NBEATS, PatchTST, etc.) that take the clean low-dimensional time series as input. I tested it and in my opnion it did not yield any advantage over simple and blazingly fast boosted tree methods. If data is ultra noisy and signals extremely ultra-sparse, NNs are simply a bad choice. And yes, NN models are notoriously difficult to interpret, while regression or even boosted tree models can give you highly interpretable feature weights and interactions. As u/jawanda said: UNDERSTAND YOUR STRATEGY, then use the simplest data and model to try you hypothesis. Mean-reversion, trend, etc. Basic logic and strategy first -> test with simplest model (stat test, lin/log reg, minimal meaningful derivations/indicators on cleanly and properly standardized/scaled/log-or-nonlog price or returns data). Tl:dr Logic + basics >> fancy bells and wistles models There's no way around understanding the basics of data science, statistics and your data at hand. Throwing tons of data at a fancy model hoping it will automagically find an edge and print money for you simply doesn't work. No free lunch here. Ps: that's also how I started out, naive, thinking a fancy model might print gold for me 🤣 but I also knew "it's probably not that simple, but in the worst case I don't make any money but will learn something along the way." And I did, I learnt a lot, and that's very valuable. Plus, after 5 years working on it on the side, I think I did find some profitable strategies that I'm gonna implement and paper trade soon, taking some time off after my PhD to give it a try :)
  4. u/CKtalon 0 1 day ago
    Counterpoint, consider instruments that are all meant to follow a particular index, say S&P. There's SPY, /ES, /MES, /NES, etc. All of them should have the same shape visually, but due to very minute liquidity differences, their raw price values can be off by a few ticks (the spread moves, but no trade happens). Yet, in the sub-minute/hour/day level, they still behave the same way. Numerically, all these instruments are then 'different', though with very high similarity. An image blurs these non-consequential differences and could possibly make them identical due to the pixel resolution blurring out these minor tick differences.
  5. u/BeyondGITSandBots 0 1 day ago
    I feel like I should not even answer your "counterpoint", as it makes no sense and should be self-explanatory, and I don't mean to be condescending here; please pardon my explicit and direct words but I feel you miss basic mathematical principles here, based on what you claim or suggest in your "counterpoint". I think it would have been better to frame your "counterpoint" as a question instead. What you suggested does not justify creating an image of your data. If you need to filter noise, there are several very simple mathematical operations that achieve exactly this while being much more computationally efficient (i.e. several orders of magnitudes faster) and in terms of storage size (try and see what happens if you save your data as millions of images, compared to rounding your fp64 arrays to fp8). Why not just: round, bin, apply a low-pass filter (i.e. moving average), or take the average of the assets you mentioned? Edit: I meant fp64, not int64
  6. u/Grand-Fly-6090 OP 2 1 day ago
    Thank you!
  7. u/Civil_Blackberry_225 0 1 day ago
    With this you don't have to get the standardization right
  8. u/jawanda 12 1 day ago
    I've experimented with lots of different ways of abstracting the ohlcv data and feeding it to different types of nn's to see if it can find patterns that are otherwise missed. My favorite was to classify each bar chart image based on what I know happens in the subsequent bars and then train an image recognition model on the cohort that would've preceded a massive climb. Needless to say there was no predictive value there that I could find. I think the idea is intriguing though because we want to believe in a black box that magically predicts direction so bad.... If it's so convoluted that I don't even know why the agent is doing what it's doing , maybe I'll stumble on a gold mine that defies all logic, just like the markets! No, sadly, my best strategies are incredibly logical and easy to understand at a basic level. Doesn't make them easier to stumble upon though.
  9. u/[deleted] 1 1 day ago

    [removed] — already gone when the archive first saw it

  10. u/DiscussionSudden1177 3 1 day ago
    that last line really hits, the simplest strategies are never simple to find
  11. u/BeyondGITSandBots 2 1 day ago
    Let me answer your question with a few counter questions, instead of simply answering it for you. Figure it out and learn some something based on your insights ;) * What's the data? And what does the image do/show? Is that a useful reprentation for us human? Is it a good representation of the data for computers? * What data in the image are you interested in? * Do you care about the lines? * Do you care about the white space? * What's the most efficient way of representing that data? * Do you need the image to feed the data to a CNN?
  12. u/[deleted] 1 1 day ago

    [removed] — already gone when the archive first saw it

  13. u/AutoModerator moderator 1 1 day ago
    Your post was removed under **Rule 2 (high-quality questions only)**. Generic “which data vendor should I use?” posts usually lack the detail needed for meaningful discussion. **Commonly used market data providers:** - **Yfinance** - **Massive.com** - **Databento** - **FMP** If you repost, please include details such as: - asset classes and markets - symbols or venues - historical vs real-time - granularity and depth - licensing or redistribution needs - latency expectations - budget constraints *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/algotrading) if you have any questions or concerns.*
  14. u/blu_mOOn_2020 1 1 day ago
    Claude: at any given space and time, no edge exists, all hover around 50% plus or minus. So any algos that have consistent, PF1.5+0edge is as rare as a spotting money in the streets. After that I turned off my algo and went back to manual trading for now 😔
  15. u/Grand-Fly-6090 OP 2 1 day ago
    You would think if you can do it manually that you could code it. Speaking from zero experience.
  16. u/blu_mOOn_2020 1 1 day ago
    No, algos play by fixed data rules. So many algos and systematic trading (I hear 80-90% of trading volume is done systematically that takes away any materially known edges), and it's now back to a human has an edge regime (for the most patient disciplined risk managed traders).
  17. u/blu_mOOn_2020 1 1 day ago
    Claude did say the only edge now exists with the most uncomfortable trades...ie that takes time (intraday 3hrs on avg), that takes large volatility abortion (meaning drawdown risks happen more often before you win). Source is Claude on NQ futures.
  18. u/Scott_Malkinsons 2 1 day ago
    Brother. You need to quit believing AI. AI is great for things like "how do I use a chainsaw" where everyone on Reddit agrees and probably knows how to successfully do the task. But trading, where most people lose, just trained AI to be a frigging dumbass when it comes to trading advice. It's literally the blind leading the blind because the AI was trained from a bunch of losers. If you want actual advice, you want the "secrets", read academic research papers on trading. SSRN and such have quite a few. They're boring AF but that's where the actual information is; looking at proven systems used over decades. Not whatever BS Claude is telling you.
  19. u/blu_mOOn_2020 2 1 day ago
    Yea I knew Claude is speaking data, but missing context. Thanks for the human insight...in age where AI begins to overwhelm, I'm actually looking forward to human edge wins too!
  20. u/Scott_Malkinsons 1 1 day ago
    Not always, or at least not easily. Take something as simple as a MA crossover. You, as a human trader, would look at the cross, take the trade and know not to keep doing it over and over. A computer, with just "trade the MA cross" as the entire instruction set would keep trading every time it crosses. You'd need additional logic to prevent it. But what logic? Don't trade more than once in 60 minutes? What if price keeps crossing entry for a couple hours? Don't trade more than once a day? Might miss opportunities. And this is like the absolute simplest example.
  21. u/blu_mOOn_2020 1 1 day ago
    Yes, edge now requires many filters and layers as simple strategies are being overly systematic trading to a point of edge erosion. How long have you being at this before you found one that worked for you? Just wondering as it gives me some glittering hope
  22. u/Scott_Malkinsons 2 1 day ago
    No edge exists, when you're taking every signal. That's why the money is in the filter, not the strategy. When you're manually trading, you have a discretionary filter that you never programmed into an algo. If you take 200 symbols and you do say an ORB on all of them you'll probably end up 50/50. If you filter it so only the top few volatile symbols are being traded, those with momentum to actually move, then you can have edge (usually the long tail is the bet, you'd trail the SL so your max loss is -1R and your positives, once in a while, are absurd).
  23. u/blu_mOOn_2020 1 1 day ago
    Yes I agree with your take that filters makes the difference!! Humans by design are such filters that AI and algos can never duplicate...humans vulnerability is emotions, risk management, and of course we can't sit at the computer 24/7
  24. u/[deleted] 1 1 day ago

    [removed] — already gone when the archive first saw it

  25. u/Klutzy-Sea-4857 4 1 day ago
    It is a terrible approach. You are literally encoding numerical data into pixels just to decode it back. Massive information loss for no reason. Raw OHLCV into the model is cleaner and faster.
  26. u/Grand-Fly-6090 OP 2 1 day ago
    Agreed.
  27. u/Scott_Malkinsons 2 1 day ago
    Yeah I use bar charts, but the image you've shown isn't bar; it's **LINE**bar and F-**LINE**bar. Here's the brutal truth. You can look at the same image a thousand different ways, it's still the same image. You are not going to "choose a chart visual" and go from non-profitable to profitable. You think it's odd? Just don't use it.