Positional Encodings and Group Theory | 3Blue1Brown and Alok Puranik
Jane Street · 76:29
This interview derives, from first principles, that essentially *all* practical positional encodings in transformers are matrix exponentials — i.e. representations of the additive group of time — and that the only pos...