Feature engineering is the process of transforming raw data into input variables, called features, that help a machine learning model learn patterns more easily and accurately. It includes cleaning data, creating new variables, encoding categories and scaling numbers. On structured business data, good features often matter more than the choice of algorithm.
Key takeaways
- A feature is a measurable input the model uses, such as age, price or “days since last purchase”.
- Domain knowledge is the biggest source of strong features.
- Common techniques include encoding, scaling, binning, date extraction and aggregation.
- Avoid data leakage: never use information that would not be available at prediction time.
Feature engineering explained in simple terms
A doctor does not diagnose from raw data like “born 14 March 1980” and a list of every meal eaten. They use useful summaries: age, average weekly alcohol intake, body mass index. Feature engineering does the same for models. It turns raw records into meaningful signals the model can learn from.
Why feature engineering matters
A model only sees what you give it. If churn depends on “fewer logins this month than last month”, but you only provide total logins, the model may miss the signal. Creating that change as a feature makes the pattern obvious. Simple models with well-designed features often beat complex models with raw inputs, and they are easier to explain.
Common feature engineering techniques
| Technique | What it does | Example |
|---|---|---|
| Handling missing values | Fill, flag or drop gaps | Fill missing income with the median and add an “income_missing” flag |
| One-hot encoding | Turns categories into 0/1 columns | City = Pune becomes city_Pune = 1 |
| Target / frequency encoding | Replaces categories with statistics | Replace product ID with its purchase frequency |
| Scaling and normalisation | Puts numbers on similar ranges | Scale income and age to comparable values |
| Log transforms | Reduce the effect of very large values | Log of transaction amount |
| Binning | Groups numbers into ranges | Age groups: 18–25, 26–40 |
| Date and time features | Extract useful parts of timestamps | Day of week, month, is_holiday, hour |
| Ratios and interactions | Combine features | Debt-to-income ratio, price per square foot |
| Aggregations | Summarise history | Orders in last 30 days, average basket value |
| Text features | Convert text to numbers | Word counts, TF-IDF, embeddings |
The feature engineering process
- Understand the problem and talk to domain experts about what drives the outcome.
- Explore the data for gaps, outliers and distributions.
- Create candidate features based on hypotheses.
- Test them on the validation set, keeping those that improve results.
- Remove redundant or noisy features, a step called feature selection.
- Automate the transformations in an ML pipeline so training and production use identical logic.
Data leakage: the biggest trap
Data leakage happens when a feature contains information that would not be available at the moment of prediction. For example, using “account closed date” to predict churn. The model looks brilliant in testing and fails in production. Also compute scaling and encoding statistics from the training set only, never from validation or test data.
Feature engineering in deep learning
Deep learning models learn many features automatically from raw images, audio or text, which reduces manual work. That is called representation learning. But feature engineering still matters for tabular data, for combining different data sources and for adding domain knowledge models cannot infer.
Limitations
Feature engineering takes time and expertise. Too many features increase the risk of overfitting. Features that work today can break when upstream data changes. And complex engineered features can make models harder to explain if they are not well documented.