https://baampark.github.io/tags/llm/Recent content in LLM on Baam's TechlogHugo -- 0.128.0en-usTue, 08 Jul 2025 21:40:50 -0400https://baampark.github.io/posts/2025-07-08_special_token/Tue, 08 Jul 2025 21:40:50 -0400https://baampark.github.io/posts/2025-07-08_special_token/A tokenizer converts natural language into a sequence of tokens. Among these tokens are special tokens, which are not regular words but serve specific functions for the model (e.g., <BOS> and <EOS>). While reviewing academic literature on LLMs and VLMs, I came across several studies that introduce new special tokens to enhance model capabilities. In this blog, we’ll explore what special tokens are in LLM tokenization and, more importantly, examine when and why researchers choose to add new special tokens.https://baampark.github.io/posts/2025-06-03_llm_decoding/Tue, 03 Jun 2025 15:28:55 -0400https://baampark.github.io/posts/2025-06-03_llm_decoding/Most large language models (LLMs) today are autoregressive models. Before LLMs, NLP was fragmented — different problems like text classification, translation, summarization, and question answering all needed their own models, datasets, and training tricks. But then came GPT-2, and everything changed. GPT-2 is an autoregressive model trained purely on text generation — predicting the next word in a sequence — that’s called decoding.Surprisingly, this simple setup made it capable of handling a wide range of NLP tasks, often without fine-tuning.