softmas is off by 1 - SERP Analysis

https://www.evanmiller.org/attention-is-off-by-one.html

https://github.com/kyegomez/AttentionIsOFFByOne

https://www.reddit.com/r/MachineLearning/comments/1594zb4/d_attention_is_off_by_one/

https://medium.com/%40diagnosta/is-attention-really-off-by-one-d244df57558e

https://news.ycombinator.com/item?id=36851494

https://discuss.pytorch.org/t/softmax-always-returns-1/80713

https://datascience.stackexchange.com/questions/117689/error-useing-soft-max-gives-outputs-greater-than-1

https://www.linkedin.com/posts/evanmmiller_attention-is-off-by-one-activity-7089274787872546819-uJWT

https://github.com/pytorch/pytorch/issues/120589

https://news.ycombinator.com/item?id=36853329

https://www.singlestore.com/blog/a-guide-to-softmax-activation-function/

https://medium.com/%40harrietfiagbor/softmax-uncovered-balancing-precision-with-numerical-stability-in-deep-learning-b8876490d411

https://community.deeplearning.ai/t/why-use-softmax-instead-of-a-linear-transform-that-sums-to-1/9107

https://discuss.pytorch.org/t/softmax-calculation-is-slightly-off/82083

https://stats.stackexchange.com/questions/309642/why-is-softmax-output-not-a-good-uncertainty-measure-for-deep-learning-models

https://stackoverflow.com/questions/58544097/turning-off-softmax-in-tensorflow-models

https://www.pinecone.io/learn/softmax-activation/

https://publish.obsidian.md/sherlock/blog/ML%26DL/From%2BSoftmax%2Bto%2BFlashAttention

https://arxiv.org/html/2309.17453v3

https://openreview.net/forum?id=oScaeIibRx

https://kyunghyuncho.me/softmax-forever-or-why-i-like-softmax/

https://x.com/EvMill/status/1683508861762695168

https://www.jmlr.org/papers/volume23/21-1306/21-1306.pdf

https://www.moleculardevices.com/sites/default/files/en/assets/user-guide/br/softmax-pro-user-guide-v4-8.pdf

https://proceedings.neurips.cc/paper/2021/file/b1d10e7bafa4421218a51b1e1f1b0ba2-Paper.pdf

https://iiswc.org/iiswc2022/IISWC2022_63.pdf

https://forum.mabtech.com/topic/868-how-to-define-softmax-pro-range-result-as-0/

https://syncedreview.com/2021/10/28/deepmind-podracer-tpu-based-rl-frameworks-deliver-exceptional-performance-at-low-cost-133/

https://uk.mathworks.com/help/deeplearning/ref/softmax.html

https://openreview.net/forum?id=RSiGFzQapl&referrer=%5Bthe%20profile%20of%20Yizeng%20Han%5D(%2Fprofile%3Fid%3D~Yizeng_Han1)

https://www.gaohongnan.com/playbook/why_softmax_preserves_order_translation_invariant_not_invariant_scaling.html

https://arxiv.org/html/2501.19399v1

https://www.moleculardevices.com/sites/default/files/en/assets/user-guide/br/softmax-pro-data-acquisition-and-analysis-software-v7.pdf

https://scale.snu.ac.kr/papers/2022-11-Conference-IISWC-Softmax-recomposition.pdf

https://www.youtube.com/watch?v=ZK52zfvHVh8

https://blester125.com/blog/softmax.html

https://escholarship.org/content/qt3jd1t2hr/qt3jd1t2hr_noSplash_44f241fc884bceb088556d351ec20ab3.pdf

https://www.nature.com/articles/s41598-021-94691-7

https://en.wikipedia.org/wiki/Attention_(machine_learning)

https://www.dremio.com/wiki/softmax-function/

https://lca.ece.utexas.edu/pubs/SoftMax___ASAP2020_June27.pdf

https://wandb.ai/krishamehta/softmax/reports/How-to-Implement-the-Softmax-Function-in-Python--VmlldzoxOTUwNTc

https://d2l.djl.ai/chapter_linear-networks/softmax-regression.html

https://www.linkedin.com/pulse/types-activation-functions-sigmoid-tanh-relu-softmax-part-dave-uipuc

https://www.datacamp.com/tutorial/softmax-activation-function-in-python

https://towardsdatascience.com/deep-dive-into-softmax-regression-62deea103cb8/

https://ieeexplore.ieee.org/document/9975410/

https://neptune.ai/blog/gumbel-softmax-loss-function-guide-how-to-implement-it-in-pytorch

https://alpslab.stanford.edu/papers/FrankeDegen_submitted.pdf

https://mriquestions.com/softmax.html

https://www.instagram.com/softmaxtheory/?hl=en

https://neurips.cc/media/neurips-2023/Slides/73926.pdf

https://www.sciencedirect.com/science/article/abs/pii/S1568494618302758

https://www.mathworks.com/help/deeplearning/ref/dlarray.softmax.html

https://community.deeplearning.ai/t/doubt-regarding-the-notation-in-softmax-lab/422917

https://d2l.ai/chapter_linear-classification/softmax-regression.html

https://sassafras13.github.io/GumbelSoftmax/

https://www.quora.com/What-is-softmax-regression-and-how-is-it-related-to-logistic-regression

https://levelup.gitconnected.com/killer-combo-softmax-and-cross-entropy-5907442f60ba

https://towardsdatascience.com/cross-entropy-demystified-f0886a64883f/

https://forums.fast.ai/t/understanding-softmax-probabilities-output-on-a-multi-class-classification-problem/8194

https://www.youtube.com/watch?v=RQ0zVyicqxg

https://www.sciencedirect.com/topics/computer-science/softmax-function

https://orbit.dtu.dk/files/266444756/s41598_021_94691_7.pdf

https://www.kaggle.com/code/residentmario/linear-and-non-linear-activation-and-softmax

https://docs.amd.com/r/2024.1-English/Vitis-Tutorials-AI-Engine-Development/Softmax-Function

https://nolanbconaway.github.io/blog/2017/softmax-numpy.html

https://zhuanlan.zhihu.com/p/645844743

https://www.youtube.com/watch?v=D-vNLrZRvo0

https://www.quora.com/How-does-the-Sigmoid-function-differs-from-Softmax-function

https://dage110.com/article/softmax_bug

https://cs231n.stanford.edu/reports/2017/pdfs/130.pdf

https://www.datacamp.com/tutorial/how-transformers-work

https://www.youtube.com/watch?v=M59JElEPgIg&pp=0gcJCdgAo7VqN5tD

https://www.instagram.com/reel/DFxblknxjQo/?hl=en

https://pubs.aip.org/asa/jasa/article/157/6/4217/3348984/Classification-of-sounds-from-Pacific-white-sided

https://www.youtube.com/watch?v=rf4WF-5y8uY

https://thesalt.substack.com/

https://www.nature.com/articles/s41597-025-05200-8

https://sausheong.com/how-to-train-a-llm-from-scratch-1c3490e8b2ce

https://cloud.tencent.com/developer/article/2309077

https://link.springer.com/article/10.1007/s41060-025-00797-w

https://www.mdpi.com/2227-9717/13/6/1825

https://keras.io/api/applications/

https://developer.nvidia.com/cudnn

https://www.louisbouchard.ai/how-llms-learn/

https://www.youtube.com/watch?v=RaVu5dFzYHw

https://link.springer.com/article/10.1007/s41884-025-00167-2

https://en.wikipedia.org/wiki/Sigmoid_function

https://www.instagram.com/reel/DF2NjrExdDt/?api=%E9%A9%AC%E6%9D%A5%E8%A5%BF%E4%BA%9A%E4%B8%87%E6%B4%A5%E7%BA%A6%E7%82%AEWhatsApp%EF%BC%9A601167898268%E4%B8%8A%E9%97%A8%E6%9C%8D%E5%8A%A1.kgdx&hl=zh-cn

https://gitee.com/ascend/ModelZoo-PyTorch/issues/ICDOA1

https://www.geeksforgeeks.org/introduction-convolution-neural-network/

https://www.artforum.com/lists/sidony-oneal-top-ten/

http://14.139.161.31/OddSem-0822-1122/Hands-On_Machine_Learning_with_Scikit-Learn-Keras-and-TensorFlow-2nd-Edition-Aurelien-Geron.pdf

https://www.mdpi.com/2079-9292/14/12/2328

https://cloud.tencent.com/developer/article/2310253

https://www.aimodels.fyi/papers

https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/

https://www.youtube.com/watch?v=dmngpQv5D9Y&pp=0gcJCdgAo7VqN5tD

https://www.zhihu.com/question/613980181/answer/3135797292