Transformers Need Glasses! [Federico Barbero]
Machine Learning Street Talk · 60:55
Federico Barbero (DeepMind/Oxford, now founding the Zurich lab Tabs) explains his paper "Why do LLMs attend to the first token?" / representational collapse: because causal attention creates far more information pathw...