I benchmarked DensePack 2.0 and Ponytail 5.0.1 on their reproducible tasks. DensePack beats Ponytail and saves more on all (4/4) of DensePack's tasks and 28/39 of Ponytail's tasks.

[removed]

3 comments held Reddit says 1 on reddit ↗

u/AutoModerator moderator 1 3 hours ago

Your post will be reviewed shortly. (ALL posts are processed like this. Please wait a few minutes....) *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/ClaudeAI) if you have any questions or concerns.*
u/vrnvorona 1 2 hours ago

Are there any difficult benchmarks to verify agents behave correctly, or it's same info OCRed basically to save tokens?
u/rowdy_gridlock 1 2 hours ago

interesting that densepack just straight up won on its own tasks. like yeah that's expected but the 28/39 on ponytail's home turf is what catches my eye. that's not just cherry picking benchmarks, that's actually working better across the board the cost comparisons in the infographic make a pretty clear case too, especially when you look at the dollar amounts per run stacking up over time