Byte Latent Transformer: Patches Scale Better Than Tokens (Paper Explained)
Yannic Kilcher · 36:14
The Byte Latent Transformer (BLT) replaces fixed-vocabulary tokenization with dynamic, entropy-based "patches" of raw bytes. At the same training compute (FLOPs), it scales better than BPE-tokenized models like Llama....