Regression in machine learning is a supervised learning task in which a model predicts a continuous numeric value, such as a price, a quantity or a time, from input features. The model learns the relationship between inputs and a known numeric outcome in past data, then uses it to estimate the outcome for new cases.
Key takeaways
- Regression predicts numbers; classification predicts categories.
- Linear regression is the simplest and most explainable form.
- Common metrics are MAE, RMSE and R².
- “Logistic regression” is actually a classification method, despite its name.
Regression explained in simple terms
An estate agent sees hundreds of house sales. Over time they learn that more bedrooms, a better location and a garden push prices up, while age and noise push them down. When a new house comes on the market, they estimate a price. A regression model does the same, but it measures the effect of each feature precisely and applies it consistently to thousands of cases.
How does regression work?
- Collect examples with known numeric outcomes, such as past sales with prices.
- Select and engineer features that may influence the outcome (feature engineering).
- Train: the model predicts values, a loss function such as mean squared error measures the gap, and gradient descent or a direct formula reduces it.
- Evaluate on held-out data.
- Predict values for new cases, ideally with a range of uncertainty.
Linear regression
Linear regression fits a straight line (or flat surface) through the data. With one feature: price = a + b × size. The model learns a (the starting value) and b (how much price rises per unit of size). With many features, each gets its own weight. Its big advantage is transparency: you can read exactly how much each feature contributes.
Types of regression
| Type | What it does | When to use |
|---|---|---|
| Linear regression | Straight-line relationship | Simple, explainable baselines |
| Polynomial regression | Curved relationships | When effects grow or shrink non-linearly |
| Ridge and lasso regression | Linear regression with regularisation to prevent overfitting; lasso can drop weak features | Many features, some redundant |
| Decision tree and random forest regression | Splits data into ranges | Non-linear tabular data |
| Gradient boosting regression | Builds many small trees that correct each other | High accuracy on business data |
| Neural network regression | Learns complex patterns | Images, audio, large datasets |
| Time-series regression | Uses past values over time | Forecasting demand or prices |
How to measure a regression model
- MAE (mean absolute error): the average size of errors, in the original units. Easy to explain: “off by ₹2,000 on average”.
- RMSE (root mean squared error): like MAE but punishes large errors more.
- MAPE: average error as a percentage; misleading when true values are near zero.
- R² (coefficient of determination): the share of variation the model explains, where 1 is perfect and 0 is no better than predicting the average.
Always compare with a baseline, such as predicting the average or last period’s value.
Regression vs classification
Both are supervised learning. The difference is the output. “How much will this house sell for?” is regression. “Will this house sell within 30 days?” is classification. The same data can often be framed either way, and the right choice depends on the decision you need to make.
Limitations
Regression models extrapolate poorly: predictions outside the range of training data are unreliable. Outliers can pull linear models off course. Correlated features make individual weights hard to interpret. And a strong relationship in the data does not prove cause and effect.