Scalable MatMul-free Language Modeling (Paper Explained)
Yannic Kilcher · 49:45
The paper "Scalable MatMul-free Language Modeling" removes every matrix multiplication from a large language model. Dense layers become ternary (−1/0/+1) accumulations, and attention becomes a linearized, parallelizab...