To accelerate long-context reasoning, TENCENT Hunyuan has introduced the Stem sparse attention algorithm, re-examining block-level sparsity from the perspective of "causal information flow".
With two key innovations - Token Position Decay (TPD) and Output-Aware Metric (OAM) - the approach achieves accuracy close to dense attention using only 25% of the computing power.
At the operator level, the open-sourced HPC Stem+BSA operators convert sparsity gains into tangible hardware acceleration, reducing first-token latency by 3.7 times under a 128K context.
View Original Text
This article was automatically translated by AI, the original language version should be considered the authoritative version. AASTOCKS.com Limited does not guarantee its accuracy or completeness and accepts no liability for any damages or losses arising from the use of this translation.
Comments
Sounds like a game changer for long-context reasoning! The Stem sparse attention algorithm looks really promising, especially with those innovations like Token Position Decay. Reducing compute needs while maintaining accuracy is huge, and cutting first-token latency so much is impressive. Can't wait to see how this impacts performance in real-world applications!
This Stem sparse attention algorithm sounds like a game changer for long-context reasoning! It's impressive that it can achieve nearly the same accuracy as dense attention while using only a quarter of the computing power. Reducing latency by such a significant margin seems like a huge advantage for applications requiring quick responses. I'm curious to see how this will be implemented in real-world scenarios!