Reinforcement learning (RL) is a type of machine learning in which an agent learns to make decisions by taking actions in an environment and receiving rewards or penalties. Over many attempts, it learns a strategy, called a policy, that earns the most reward over time. There are no labelled answers; the agent learns from the consequences of its own actions.
Key takeaways
- RL learns by trial and error, guided by rewards.
- Its core parts are the agent, environment, state, action, reward and policy.
- It balances exploration (trying new actions) with exploitation (using what works).
- RL powered game-playing breakthroughs such as AlphaGo and helps fine-tune large language models through RLHF.
Reinforcement learning explained in simple terms
Think about training a dog. You do not explain the rules of “sit”. The dog tries different things, and when it sits, it gets a treat. Over time it learns that sitting on command leads to rewards. Reinforcement learning works the same way. The agent tries actions, sees what happens and adjusts its behaviour to earn more rewards.
Key concepts
| Term | Meaning | Game example |
|---|---|---|
| Agent | The learner and decision-maker | The player |
| Environment | The world the agent acts in | The game |
| State | The current situation | The board position |
| Action | A choice the agent makes | A move |
| Reward | Feedback signal, positive or negative | Points, winning or losing |
| Policy | The agent’s strategy for choosing actions | How the player decides moves |
| Value | Expected long-term reward from a state | How promising a position is |
How does reinforcement learning work?
- The agent observes the current state.
- It chooses an action using its current policy.
- The environment responds with a new state and a reward.
- The agent updates its policy to favour actions that led to higher total reward.
- This loop repeats, often millions of times, usually in simulation.
A key challenge is delayed reward. In chess, a move early in the game may only prove good many moves later. RL methods estimate the long-term value of actions, not just the immediate reward. Problems are usually described mathematically as Markov decision processes.
Exploration vs exploitation
If the agent only repeats what has worked, it may miss better options. If it only experiments, it never benefits from what it knows. Good RL balances both. A common simple method, epsilon-greedy, takes the best known action most of the time and a random one occasionally.
Main types of RL methods
- Value-based (such as Q-learning): learn how good each action is in each state.
- Policy-based (such as policy gradient methods): learn the strategy directly.
- Actor-critic: combine both approaches.
- Model-based: learn a model of the environment and plan with it.
- Deep RL: use neural networks to handle complex states such as images.
Famous examples
DeepMind’s AlphaGo defeated Go champion Lee Sedol in 2016, using deep reinforcement learning combined with search. RL has also been applied to robotics, data centre cooling and chip design. Today, reinforcement learning from human feedback (RLHF) is widely used to make language models more helpful and safer: people rate responses, and the model is trained to produce responses people prefer.
RL vs supervised and unsupervised learning
Supervised learning learns from correct answers. Unsupervised learning finds structure without answers. Reinforcement learning learns from rewards for actions in a changing environment. It suits sequential decisions, where each choice affects what happens next.
Limitations
RL often needs huge amounts of trial and error, which is why most training happens in simulation. Designing the reward is hard: agents can find loopholes that earn reward without achieving the real goal, a problem called reward hacking. RL can be unstable to train, and moving from simulation to the real world often reduces performance.