What does a larger vocabulary enable in Llama 3? #llama3
AI at Meta · 0:44
Llama 3 quadrupled its tokenizer vocabulary from 32k to 128k tokens. The speaker says this lets the model handle text more efficiently and improves tasks such as coding and reasoning. This is also the stated reason th...