January 9, 2024Understanding Large Language Models - Words vs TokensA closer look at tokens and why they matter