A large language model (LLM) is an AI model trained on enormous amounts of text to understand and generate human language. It works by predicting the next piece of text, called a token, again and again. At large scale, this simple objective produces models that can answer questions, write, summarise, translate, reason through problems and write code.
Key takeaways
- “Large” refers to billions of parameters and training data measured in trillions of tokens.
- Almost all modern LLMs are built on the transformer architecture.
- Training has stages: pretraining, then instruction tuning and preference tuning such as RLHF.
- LLMs generate plausible text, not verified facts, so they can hallucinate.
- Examples include GPT, Claude, Gemini, Llama, Mistral, Qwen and DeepSeek model families.
An LLM explained in simple terms
Imagine someone who has read a large share of the public internet, many books and a lot of code, and has become extremely good at one game: guessing the next word. To guess well, they must pick up grammar, facts, writing styles and patterns of reasoning. An LLM is a machine that has played this game at huge scale. When you ask it a question, it continues the conversation with the text most likely to be a helpful answer.
How an LLM works
- Your prompt is split into tokens by tokenization.
- Each token becomes a vector, an embedding.
- Transformer layers use attention to relate every token to the others, building a rich understanding of context.
- The model outputs a probability for every possible next token.
- One token is chosen, influenced by settings such as temperature, and added to the text.
- Steps 3–5 repeat until the answer is complete. This process is LLM inference.
How LLMs are trained
| Stage | What happens | Result |
|---|---|---|
| Pretraining | Predict the next token across trillions of tokens of text and code | A base model with broad knowledge |
| Supervised fine-tuning | Learn from example prompts and ideal answers | Follows instructions |
| Preference tuning (RLHF, DPO) | Learn which answers people prefer | More helpful, honest and safe |
| Optional reasoning training | Reinforcement learning on problems with checkable answers | A reasoning model |
LLM vs language model
Every LLM is a language model, but not every language model is large. Older language models, such as n-gram models, used simple word statistics. LLMs are neural networks with billions of parameters, trained on vastly more data, which gives them general abilities that small models lack.
Types of LLMs
- Closed models: accessed through an app or API; weights are not published.
- Open-weight models: weights can be downloaded and run or fine-tuned.
- Small language models: compact models for phones, laptops and low-cost tasks.
- Reasoning models: think step by step before answering.
- Multimodal LLMs: also understand images, audio or video.
What LLMs are used for
Chat assistants, customer support, drafting and editing, summarising documents, coding help, translation, data extraction with structured output, search and question answering over company knowledge, and AI agents that use tools.
Limitations
- Hallucinations: confident but false statements.
- Knowledge cut-off: no knowledge of events after training unless given current information.
- Limited context window: they can only consider a certain amount of text at once.
- Bias: they reflect patterns in training data.
- Cost: large models are expensive to train and run.
- Security: vulnerable to prompt injection when processing untrusted content.