static sparse attention llm - SERP Analysis

https://medium.com/%40ikim1994914/advanced-modern-llm-part-3-sparse-attention-and-application-to-long-document-decoding-153a7ceb794d

https://arxiv.org/html/2406.14909v1

https://openreview.net/forum?id=HmwneoGoy9

http://arxiv.org/abs/2502.14866

https://mlsys.org/virtual/2025/session/3153

https://hanlab.mit.edu/projects/lserve

https://openaccess.thecvf.com/content/CVPR2024/papers/Song_Low-Rank_Approximation_for_Sparse_Attention_in_Multi-Modal_LLMs_CVPR_2024_paper.pdf

https://huggingface.co/papers/2410.13276

https://hanlab.mit.edu/blog/block-sparse-attention

https://neurips.cc/virtual/2024/poster/94208

https://openreview.net/pdf?id=HmwneoGoy9

https://x.com/rohanpaul_ai/status/1868273972149895416

https://llmmodels.org/blog/sparse-attention-in-transformers-step-by-step-implementation/

https://www.microsoft.com/en-us/research/publication/minference-1-0-accelerating-pre-filling-for-long-context-llms-via-dynamic-sparse-attention/

https://www.youtube.com/watch?v=Q2RKLwDzCpc

https://research.nvidia.com/publication/2023-07_efficient-transformer-inference-statically-structured-sparse-attention

https://huggingface.co/papers/2502.14866

https://www.linkedin.com/posts/pascalbiese_seerattention-learning-intrinsic-sparse-activity-7254188859612119043-2nJZ

https://www.reddit.com/r/MachineLearning/comments/1h0gl2j/d_a_blog_post_explaining_sparse_transformers_the/

https://aclanthology.org/2024.findings-acl.837.pdf

https://www.researchgate.net/publication/389547983_Progressive_Sparse_Attention_Algorithm_and_System_Co-design_for_Efficient_Attention_in_LLM_Serving

https://mlsys.org/virtual/2025/poster/3258

https://scispace.com/pdf/post-training-sparse-attention-with-double-sparsity-6ujbie0yrdst.pdf

https://medium.com/%40vishal09vns/sparse-attention-dad17691478c

https://openai.com/index/sparse-transformer/

https://www.youtube.com/watch?v=DUkWMoi5nG4

https://proceedings.neurips.cc/paper_files/paper/2024/file/5dfbe6f5671e82c76841ba687a8a9ecb-Paper-Conference.pdf

https://blog.gopenai.com/understanding-sparse-and-efficient-attention-mechanisms-in-large-language-models-llms-3c01570633ac

https://www.reddit.com/r/MachineLearning/comments/167cmys/p_efficient_way_to_implement_sparse_crossattention/

https://dev.to/nareshnishad/day-29-sparse-transformers-efficient-scaling-for-large-language-models-59j5

https://ajithp.com/2025/02/21/natively-sparse-attention-nsa-the-future-of-efficient-long-context-modeling-in-large-language-models/

https://dl.acm.org/doi/abs/10.1145/3676641.3715991

https://ui.adsabs.harvard.edu/abs/2024arXiv240615486Z/abstract

https://github.com/thu-nics/MoA

https://www.marktechpost.com/2024/07/06/minference-milliontokens-inference-a-training-free-efficient-method-for-the-pre-filling-stage-of-long-context-llms-based-on-dynamic-sparse-attention/

https://www.datacamp.com/blog/attention-mechanism-in-llms-intuition

https://www.hc2023.hotchips.org/assets/program/tutorials/ml/HC2023.Moffett.ZhibinXiao.Sparse_ML_Inference_Tutorial_v1.0.pdf

https://hqjiang.com/minference.html

https://www.deepchecks.com/glossary/sliding-window-attention/

https://zenodo.org/doi/10.5281/zenodo.14989916

https://www.aussieai.com/research/attention

https://paperswithcode.com/paper/lserve-efficient-long-sequence-llm-serving

https://www.themoonlight.io/review/lserve-efficient-long-sequence-llm-serving-with-unified-sparse-attention

https://web-cdn.bsky.app/profile/alphaxiv.org/post/3lisa2s5uuc2u

https://www.cerebras.ai/blog/accelerating-llm-training-with-variable-sparse-pre-training-and-dense-fine-tuning

https://bohrium.dp.tech/paper/arxiv/2410.13276

https://github.com/microsoft/MInference

https://www.linkedin.com/posts/techsachinkumar_star-attention-block-sparse-attention-mechanism-activity-7268949266935472128-s7za

https://www.microsoft.com/en-us/research/project/minference-million-tokens-prompt-inference-for-long-context-llms/

https://www.researchgate.net/publication/381665984_Near-Lossless_Acceleration_of_Long_Context_LLM_Inference_with_Adaptive_Structured_Sparse_Attention

https://m.youtube.com/watch?v=mF7OM_XU2S4&pp=ygUPI2ZsYXNoYXR0ZW50aW9u

https://www.hyperstack.cloud/technical-resources/tutorials/llm-inference-techniques-to-reduce-latency-and-boost-performance

https://www.adaline.ai/blog/understanding-attention-mechanisms-in-llms

https://ui.adsabs.harvard.edu/abs/2025arXiv250214866Y/abstract

https://kennethzhao24.github.io/images/isca_presentation_final.pdf

https://datamagiclab.com/dynamic-sparse-attention-revolutionizing-efficiency-in-transformer-models/

https://paperswithcode.com/paper/tactic-adaptive-sparse-attention-with

https://www.facebook.com/groups/DeepNetGroup/posts/2403009003425301/

https://www.ijcai.org/proceedings/2024/0904.pdf

https://news.ycombinator.com/item?id=43931366

https://blogs.novita.ai/activation-sparsity-revolution-rnhancing-model-performance-and-speedup-inference-without-retraining/

https://developer.nvidia.com/blog/mastering-llm-techniques-inference-optimization/

https://www.youtube.com/watch?v=0ZErAOkdCRI

https://vgel.me/posts/faster-inference/

https://www.youtube.com/watch?v=Fg8urTOImpY

https://developers.google.com/machine-learning/glossary

https://en.wikipedia.org/wiki/Attention_(machine_learning)

https://lilianweng.github.io/posts/2023-01-10-inference-optimization/

https://www.clausiuspress.com/assets/default/article/2024/11/12/article_1731408067.pdf

https://www.youtube.com/watch?v=z9zjILgfsyg

https://www.ctol.digital/news/lserve-faster-efficient-long-sequence-llm-serving/

https://hfday.ru/

https://www.digitalocean.com/community/conceptual-articles/hands-on-guide-to-object-detection-with-vision-language-models

https://icml.cc/virtual/2025/papers.html

https://ai.gopubby.com/sparse-transformers-d46463e0a9a6

https://www.youtube.com/watch?v=VMSeQKZDYH0

https://www.youtube.com/watch?v=mx4lHNfwbM0

https://semiengineering.com/hw-aligned-sparse-attention-architecture-for-efficient-long-context-modeling-deepseek-et-al/

https://radicaldatascience.wordpress.com/tag/genai/

https://www.youtube.com/watch?v=64kCDOC5weY

https://ollama.hf-mirror.com/papers?q=sparse%20attention

https://radicaldatascience.wordpress.com/category/big-data/

https://www.youtube.com/watch?v=sOPDGQjFcuM&pp=0gcJCdgAo7VqN5tD

https://www.youtube.com/watch?v=jz_GS2kDaFo

https://www.youtube.com/watch?v=0g351WQTaf8