Feature engineering data prep

Adaptive Synthetic Sampling, an extension of SMOTE that generates more synthetic samples for minority examples that are harder to learn, meaning those with many majority-class neighbours. It shifts the decision boundary toward difficult regions.

Example. Minority points surrounded by majority points receive many more synthetic neighbours than points deep inside a minority cluster.

Related: SMOTE, Oversampling, Class imbalance, K-nearest neighbours

A feature computed by summarising many rows that belong to one entity, such as a customer's total spend, average order value or number of transactions in 30 days. Aggregations turn event-level logs into per-entity features.

Example. From a UPI transaction log, you compute each user's transaction count and average amount over the last 30 days.

Related: Group by, Window feature, Feature engineering, Rolling statistics

A statistical test that compares group means to decide whether a numeric feature differs across the classes of a categorical target. As a filter method it ranks numeric features by F-value, where higher values mean stronger separation.

Example. Comparing average income across 'default' and 'no default' groups gives an F-score used to rank income against other numeric features.

Related: Filter method, Analysis of variance, Feature selection, Chi-square test

Using software to generate many candidate features automatically, typically by stacking aggregations and transformations over related tables, and then selecting the useful ones. It speeds up exploration but can produce many redundant or leaky features.

Example. A tool such as Featuretools builds features like 'mean order value per customer' and 'count of orders in last month' from linked customer and order tables.

Related: Feature engineering, Aggregation feature, Feature selection, Data leakage

A greedy wrapper method that starts with all features and removes, one at a time, the one whose removal hurts the model least, stopping when removal begins to hurt. It considers all features together at the start, but is costly with many features.

Example. Fit a regression on all 30 predictors, drop the one with the highest p-value, refit and repeat until all remaining are significant.

Related: Forward selection, Recursive feature elimination, Wrapper method, P-value

A text representation that describes a document by how many times each vocabulary word appears, ignoring word order and grammar. It is simple and effective for classification but creates large, sparse vectors and loses context.

Example. With vocabulary [good, bad, movie], the review 'good good movie' becomes [2, 0, 1].

Related: Count vectorizer, TF-IDF, N-gram, Sparse matrix

Encoding categories by first assigning each an integer, then writing that integer in binary and splitting its digits across columns. It needs only about log2(k) columns for k categories, a compromise between label and one-hot encoding for high-cardinality features.

Example. With 100 categories, binary encoding needs 7 columns, since 2^7 = 128, instead of 100 one-hot columns.

Related: One-hot encoding, High-cardinality feature, Hashing trick, Label encoding

A feature selection algorithm built around random forests. It adds shuffled 'shadow' copies of features and keeps only real features that repeatedly beat the best shadow feature in importance, giving a statistically grounded all-relevant selection.

Example. A real column that never beats the best shuffled shadow column across many rounds is rejected as no better than noise.

Related: Wrapper method, Random forest, Feature importance, Permutation importance

A power transform that finds the exponent lambda, by maximum likelihood, that makes strictly positive data look most normal. With lambda = 0 it becomes the natural log; otherwise y = (x^lambda - 1) / lambda. It cannot handle zero or negative values.

Example. Fitting Box-Cox on positive delivery times might choose lambda near 0.2, close to a log transform, and reduce right skew.

Related: Yeo-Johnson transform, Power transform, Log transform, Skewness

A variable that takes values from a limited set of labels or groups instead of measuring a quantity. Models need numbers, so categories must be encoded first. They may be nominal (no order) or ordinal (ordered).

Example. City with values Mumbai, Delhi, Bengaluru and Chennai is categorical, as is payment type with UPI, card and cash.

Related: Nominal variable, Ordinal variable, One-hot encoding, Cardinality

A situation where one class in a classification dataset is far more common than another, as in fraud or disease detection. Models tend to favour the majority class, and accuracy becomes misleading, so special metrics and resampling are needed.

Example. In a dataset of 1,00,000 transactions with 200 frauds (0.2 percent), a model that always says 'not fraud' is 99.8 percent accurate yet useless.

Related: Oversampling, Undersampling, SMOTE, Class weights

Multipliers applied in the loss function that make mistakes on rare classes cost more, so the model pays them more attention without changing the data. A common choice is weight proportional to 1 / class frequency.

Example. With 1 percent fraud, giving fraud a weight of 99 makes one missed fraud cost as much as 99 missed non-frauds.

Related: Class imbalance, Cost-sensitive learning, Loss function, Logistic regression

A scikit-learn tool that applies different preprocessing steps to different columns of a table, for instance scaling numeric columns and one-hot encoding categorical ones, then joins the results into one feature matrix.

Example. Scale 'age' and 'income' with StandardScaler while one-hot encoding 'city' in one ColumnTransformer.

Related: Pipeline, One-hot encoding, Standardization, Feature engineering

A column that holds exactly the same value in every row of the dataset. It carries no information for prediction, so it is dropped during cleaning or by a variance threshold.

Example. A 'country' column that reads 'India' in all rows adds nothing to a model.

Related: Variance threshold, Feature selection, Data cleaning, Correlation filter

Filling missing values with a fixed chosen value, such as 0, -1 or the label 'Missing'. It makes missingness explicit for tree models and categorical features, though a poorly chosen constant can mislead linear models.

Example. Missing 'previous_loans' is set to 0, and a missing 'occupation' is set to the category 'Unknown'.

Related: Missing value imputation, Missing indicator, Mode imputation, Categorical variable

A measure of how much a regression's fitted values change when a single observation is removed. Large values flag points with strong influence on the model, combining how extreme the point is in features and how far it lies from the fit.

Example. A point with Cook's distance far larger than the rest could shift the fitted line noticeably, so it is investigated before being kept.

Related: Outlier treatment, Linear regression, Leverage, Residual

A feature selection step that removes one of each pair of features whose correlation exceeds a threshold, such as 0.9, because they carry nearly the same information. It reduces redundancy and multicollinearity.

Example. Height in cm and height in inches have correlation 1.0, so only one is kept.

Related: Multicollinearity, Correlation coefficient, Filter method, Feature selection

A tool that converts a collection of text documents into a matrix of word counts, with rows for documents and columns for vocabulary terms. It is the standard way to build bag-of-words features.

Example. Three short SMS messages are turned into a 3 x 20 matrix of counts over a 20-word vocabulary using CountVectorizer.

Related: Bag of words, TF-IDF, Stop words, Sparse matrix

Representing a cyclic feature, such as hour or month, with sine and cosine so the end of the cycle sits next to the start: sin(2*pi*x / period) and cos(2*pi*x / period). Otherwise a model treats hour 23 and hour 0 as far apart.

Example. Hour 23 and hour 0 are 23 apart as plain numbers but almost identical after sine and cosine encoding with a period of 24.

Related: Datetime features, Feature transformation, Ordinal encoding, Periodicity

Useful numbers and flags extracted from a timestamp, such as year, month, day of week, hour, is_weekend, is_holiday and days since an event. Raw timestamps are rarely useful to models, but these components expose patterns like seasonality.

Example. From an order timestamp you extract day_of_week and is_festival_week, capturing the Diwali demand spike.

Related: Cyclical encoding, Lag feature, Feature engineering, Seasonality

Converting a continuous numeric feature into a set of ordered intervals, or bins, and labelling each value by its bin. It can capture non-linear effects in linear models and reduce noise, but loses detail within a bin.

Example. Age is split into 18-25, 26-35, 36-50 and 50+, and each customer is tagged with one of four age bands.

Related: Equal-width binning, Equal-frequency binning, Categorical variable, Feature transformation

Features designed using expertise about the problem area, capturing concepts that raw data do not show directly. They often beat automated features because they encode how the real process works.

Example. For a credit model, 'debt-to-income ratio' encodes a risk concept every lender knows and is more useful than raw debt or income.

Related: Feature engineering, Ratio feature, Aggregation feature, Interaction feature

A binary 0/1 column that indicates whether an observation belongs to a category. In regression, one category is often dropped as the baseline so the remaining dummies are not perfectly redundant, which avoids the dummy variable trap.

Example. For a 3-category plan (basic, plus, pro), two dummies is_plus and is_pro are enough; basic is when both are 0.

Related: One-hot encoding, Dummy variable trap, Multicollinearity, Categorical variable

The perfect multicollinearity created when all one-hot columns of a feature are kept in a model with an intercept, because they always sum to 1. It makes coefficients unidentifiable in ordinary linear regression. The fix is to drop one category's column.

Example. Keeping is_male and is_female both plus an intercept lets either column be computed from the other, so regression coefficients cannot be uniquely estimated.

Related: Dummy variable, One-hot encoding, Multicollinearity, Linear regression

Finding and dropping repeated records in a dataset, whether exact copies or near-duplicates that differ only slightly. Duplicates overweight some cases, inflate accuracy when copies land in both training and test sets and skew statistics.

Example. A customer table with two rows for the same email and phone is reduced to one row, using drop_duplicates in pandas.

Related: Data cleaning, Data leakage, Deduplication, Train-test split

A feature selection approach in which selection happens during model training, as with L1 regularisation shrinking coefficients to zero or a tree ensemble ranking feature importance. It balances the speed of filters and the accuracy of wrappers.

Example. Training a lasso model zeroes out the coefficients of 60 of 100 features, leaving the 40 that matter.

Related: Lasso regression, Filter method, Wrapper method, Feature importance

Learning a dense, low-dimensional vector for each category value as part of training a neural network, so similar categories end up with nearby vectors. It is an alternative to one-hot encoding for high-cardinality categories and can be reused by other models.

Example. A retail model learns a 10-number vector for each of 5,000 store IDs, and stores with similar demand patterns end up with similar vectors.

Related: High-cardinality feature, One-hot encoding, Embedding, Feature learning

Discretising a numeric feature so each bin holds roughly the same number of observations, using quantiles as bin edges. It handles skewed data better than equal-width bins, but bin widths can vary widely.

Example. Splitting 1,000 customers into quartile bins by spending puts about 250 customers in each band, even if the top band spans a huge range.

Related: Equal-width binning, Discretization, Percentile, Quantile transform

Discretising a numeric feature into bins that all span the same range of values, computed as (max - min) / number of bins. It is simple, but skewed data leave some bins crowded and others nearly empty.

Example. Salaries from 0 to 10,00,000 in 5 bins give bands of Rs 2,00,000 each, though most people fall into the first band.

Related: Equal-frequency binning, Discretization, Skewness, Quantile transform

Creating a new categorical feature from the combination of two or more categorical values, so each combination gets its own weight. It is common in linear models for ads and recommendations, where specific pairings carry signal.

Example. Crossing city with day_of_week gives values like Mumbai_Friday and Pune_Sunday, each with its own learned effect.

Related: Interaction feature, One-hot encoding, Hashing trick, Categorical variable

The work of turning raw data into input columns that help a model learn, by cleaning, combining, transforming and creating variables. Good features expose the signal a model could not easily find on its own. For tabular data it is often the biggest driver of accuracy, more than the choice of algorithm.

Example. From a raw timestamp of a Swiggy order, you create hour_of_day, is_weekend and is_lunch_rush columns, which predict delivery delay far better than the raw timestamp.

Related: Feature extraction, Feature selection, Feature transformation, Domain knowledge

Building a smaller or more useful set of new features from raw, often unstructured, data such as text, images or signals. Unlike selecting existing columns, it computes new representations, for example word counts from text or edge statistics from pictures. It reduces raw complexity into numbers a model can use.

Example. From a customer-support email you extract word counts, email length and whether it contains a refund keyword, giving three numeric columns for a classifier.

Related: Feature engineering, Feature selection, Bag of words, Dimensionality reduction

Rescaling numeric features so they sit on comparable ranges. Models that use distances or gradients, such as k-nearest neighbours, SVMs and neural networks, are distorted when one feature is measured in lakhs and another in single digits. Scaling puts them on equal footing and speeds up gradient-based training.

Example. Age (18 to 60) and annual salary (2,00,000 to 40,00,000) are both rescaled, so salary does not swamp age when distances between customers are computed.

Related: Min-max scaling, Standardization, Robust scaling, Normalization

Choosing a subset of the available features to use in a model, dropping irrelevant or redundant ones. It reduces overfitting, speeds up training, cuts cost and makes models easier to interpret.

Example. From 200 candidate columns, a model keeps the 25 most predictive ones and performs as well as with all 200.

Related: Filter method, Wrapper method, Embedded method, Dimensionality reduction

A central system that stores, documents and serves computed features for both training and real-time prediction, keeping definitions consistent and reusable across teams. It helps avoid training-serving skew and duplicated work.

Example. A fintech team computes 'transactions_last_30d' once in the feature store, and both the credit model and the fraud model reuse it.

Related: Training-serving skew, Feature engineering, Pipeline, Data pipeline

Applying a mathematical function to an existing feature to change its scale or shape, such as a logarithm, square root or power. The aim is to make relationships closer to linear, reduce the influence of extreme values or satisfy a model's assumptions.

Example. Monthly income in rupees ranges from 15,000 to 5,00,00,000; applying log10 compresses it to a range of about 4.2 to 7.7, so a linear model is not dominated by a few very high earners.

Related: Log transform, Power transform, Feature scaling, Skewness

A feature selection approach that scores each feature with a statistic, such as correlation, chi-square or mutual information, independent of any model, and keeps the top scorers. It is fast and scalable, but ignores feature interactions.

Example. You compute the correlation of each of 300 columns with the target and keep the 40 with the highest absolute correlation.

Related: Wrapper method, Embedded method, Mutual information, Chi-square test

The rule that data preprocessors learn their parameters, such as mean or category list, from training data only using fit, then apply them to training, validation and test data using transform. This avoids information from test data leaking in.

Example. A scaler is fitted on training rows to learn mean 50, then the same mean 50 is used to transform the test rows.

Related: Data leakage, Pipeline, Feature scaling, Train-test split

A greedy wrapper method that begins with no features and adds, one at a time, the feature that most improves the model, stopping when improvement ends. It is simple but can miss combinations that only work together.

Example. Start with an empty model, add 'distance' because it helps most, then 'hour', and stop when adding more gives no gain.

Related: Backward elimination, Wrapper method, Stepwise selection, Feature selection

Replacing each category with how often it appears in the training data, either as a count or a proportion. It needs no target, gives one numeric column and can help when frequency itself is informative, but different categories with equal counts become indistinguishable.

Example. If Bengaluru appears in 2,000 of 10,000 orders, it is encoded as 2,000 (or 0.2).

Related: Count encoding, Target encoding, High-cardinality feature, Categorical variable

Features derived from location data, such as distance between two points, distance to a landmark, grid or geohash cell and neighbourhood statistics. They convert raw latitude and longitude into information that models can use.

Example. For a cab fare model, you compute the haversine distance between pickup and drop coordinates and the pickup's distance to the airport.

Related: Feature engineering, Haversine distance, Aggregation feature, High-cardinality feature

Splitting data so that all rows belonging to one group, such as a patient or customer, fall entirely in either train or test. It prevents leakage caused by the same entity appearing on both sides.

Example. With several scans per patient, all scans of one patient go to the same fold, so the model is tested on unseen patients.

Related: Data leakage, Train-test split, Cross-validation, Stratified k-fold

Mapping features, such as categories or words, to a fixed number of columns by applying a hash function to their names and taking the result modulo the column count. It needs no vocabulary and bounded memory, at the cost of occasional hash collisions where different values share a column.

Example. A user-ID feature with millions of values is hashed into 1,024 columns; two different users may collide but the model still trains at fixed size.

Related: Hash function, High-cardinality feature, Hash collision, One-hot encoding

A text featuriser that applies the hashing trick to word tokens, mapping them straight to a fixed number of columns without storing a vocabulary. It is memory-efficient and stateless, but the features cannot be traced back to words.

Example. HashingVectorizer(n_features=2**18) turns any text into a 262,144-column sparse vector without fitting a vocabulary first.

Related: Hashing trick, Count vectorizer, Bag of words, Sparse matrix

The great-circle distance between two points on a sphere given their latitude and longitude, calculated with a formula using sines of half-angle differences. It approximates real-world distance between places on Earth.

Example. The haversine distance from Bengaluru to Chennai is roughly 290 km, while the road distance is longer.

Related: Geospatial features, Euclidean distance, Feature engineering, Coordinates

A categorical feature with a very large number of distinct values, such as user IDs, product SKUs or pincodes. One-hot encoding it creates enormous, sparse data, so alternatives such as target, frequency or hashed encoding, or embeddings, are used.

Example. India has over 19,000 pincodes; one-hot encoding a pincode column would create thousands of mostly-zero columns.

Related: Cardinality, Target encoding, Hashing trick, Entity embedding

Filling a missing value by copying an observed value from a similar record, called the donor, in the same dataset. Because it borrows real values, it preserves the distribution better than mean imputation.

Example. A survey respondent with a blank income gets the income of a randomly chosen respondent with the same age band and city.

Related: Missing value imputation, KNN imputation, Mean imputation, Missing at random

A credit-scoring statistic that summarises how well a feature separates good from bad outcomes, computed by summing (share good - share bad) x WoE across bins. Higher values indicate more predictive power, and it is used to rank and filter variables.

Example. A feature with information value of 0.02 is usually treated as nearly useless, while one around 0.3 is considered fairly strong.

Related: Weight of evidence, Feature selection, Binning, Filter method

A feature made by combining two or more features, typically by multiplication or ratio, so a model can capture effects that only appear when they act together. It expresses that the effect of one variable depends on another.

Example. For a delivery model, distance multiplied by is_raining captures that rain slows long trips far more than short ones.

Related: Polynomial features, Feature crossing, Ratio feature, Feature engineering

Estimating missing values between known points, most simply by drawing a straight line between the neighbours. It suits smooth, ordered data such as time series, but uses future values, which can leak information if done before splitting.

Example. With sales of 100 on Monday and 140 on Wednesday, linear interpolation fills Tuesday as 120.

Related: Last observation carried forward, Missing value imputation, Time series, Temporal leakage

A rule for flagging outliers as values below Q1 - 1.5 x IQR or above Q3 + 1.5 x IQR, where IQR is the interquartile range Q3 - Q1. It relies on quartiles, so it is not distorted by the outliers it is trying to find.

Example. With Q1 = 20 and Q3 = 40, IQR = 20, so values below -10 or above 70 are flagged.

Related: Interquartile range, Outlier treatment, Z-score, Box plot

An approach that models each feature with missing values as a function of the other features, and repeatedly predicts and refills the blanks in rounds until estimates settle. It captures relationships among columns, unlike single-value fills.

Example. Missing income is predicted from age and education, then missing age from income and education, and the cycle repeats until stable.

Related: Multiple imputation, KNN imputation, Missing value imputation, Regression

Computing target encoding values for each fold using only the other folds' data, so a row's encoding never uses its own label. This out-of-fold approach makes the encoded feature safer and gives a more honest estimate on validation data.

Example. With 5 folds, rows in fold 1 get city means calculated from folds 2 to 5 only, and the same is done for every other fold.

Related: Target encoding, Cross-validation, Data leakage, Leave-one-out encoding

Filling a missing value using the values of the k most similar rows, where similarity is measured on the other features, typically by averaging their values. It respects local patterns but is slower and needs scaled features.

Example. A missing salary is filled with the average salary of the 5 customers closest in age, city and job level.

Related: Missing value imputation, K-nearest neighbours, Feature scaling, Iterative imputation

A measure of how heavy the tails of a distribution are compared with a normal distribution. High kurtosis means extreme values occur more often than a bell curve would predict. It helps judge whether outliers are a systematic feature of the data.

Example. Daily stock returns typically have high kurtosis: rare big crashes or jumps occur more often than a normal distribution suggests.

Related: Skewness, Outlier, Variance, Distribution

Replacing each category with an arbitrary integer, such as Mumbai = 0, Delhi = 1. It is compact but creates a false ordering, which harms linear and distance-based models, though tree-based models can cope with it. It is best for targets or ordered categories.

Example. Encoding class labels cat, dog, bird as 0, 1, 2 for a classifier is fine because the target is not treated as a numeric quantity.

Related: Ordinal encoding, One-hot encoding, Nominal variable, Categorical variable

A feature that carries a previous value of a time series into the current row, such as yesterday's sales in today's row. It lets standard models use recent history to forecast the next value.

Example. To predict tomorrow's electricity demand, you add demand_lag_1 (today), demand_lag_7 (a week ago) as columns.

Related: Rolling statistics, Time series, Window feature, Temporal leakage

An imputation method for time-ordered data that fills a gap with the most recent earlier observed value. It is simple and never uses future data, but can hide real change and bias trends.

Example. If a sensor reports 24 degrees at 10:00 and nothing at 11:00, the 11:00 reading is filled as 24.

Related: Missing value imputation, Time series, Lag feature, Interpolation

A form of target encoding where each row's category value is the target mean of all other rows in that category, excluding the row itself. This reduces direct target leakage, though it can still leak subtly and can be exploited by trees.

Example. For a city with 4 rows having targets 1, 0, 1, 1, the first row is encoded as (0 + 1 + 1) / 3 = 0.67.

Related: Target encoding, Data leakage, K-fold target encoding, High-cardinality feature

Reducing a word to its dictionary base form, called the lemma, using vocabulary and grammar knowledge, so 'better' becomes 'good' and 'was' becomes 'be'. It is slower than stemming but yields real words.

Example. 'Feet' is lemmatised to 'foot' and 'running' to 'run'.

Related: Stemming, Text preprocessing, Tokenization, Part-of-speech tagging

Removing every row that has any missing value before analysis. It is simple and safe when data are missing completely at random and few rows are lost, but can discard much data and introduce bias otherwise.

Example. Dropping the 40 of 10,000 customers with a blank age is fine; dropping 4,000 rows with some blank would waste much information.

Related: Missing value imputation, Missing completely at random, Missing data, Sample size

Replacing a value x with log(x), often log(1 + x) when zeros are possible. It compresses large values, turns multiplicative effects into additive ones and reduces right skew, which suits prices, incomes, counts and other quantities spanning several orders of magnitude.

Example. Sales of Rs 1,000 and Rs 10,00,000 differ by 1000x, but their log10 values are 3 and 6, a gap of just 3.

Related: Skewness, Power transform, Feature transformation, Box-Cox transform

A distance from a point to the centre of a distribution that accounts for the spread and correlation of the features, using the inverse covariance matrix: sqrt((x - mean)^T S^-1 (x - mean)). It is used to detect multivariate outliers.

Example. A person who is 150 cm tall and 90 kg may be unremarkable on each feature alone but far from the centre in Mahalanobis terms because the pair is unusual.

Related: Outlier treatment, Covariance matrix, Euclidean distance, Anomaly detection

A scaling method that divides each feature by its largest absolute value so results fall between -1 and 1. It does not shift the data, so zeros stay zeros, which makes it suitable for sparse data where preserving sparsity matters.

Example. A sparse column with values 0, 0, 4, -8 becomes 0, 0, 0.5, -1 after dividing by 8, and the zeros stay exactly zero.

Related: Min-max scaling, Sparse matrix, Feature scaling, Standardization

Replacing every missing value in a numeric column with that column's average from the training data. It is simple and keeps the mean unchanged, but shrinks variance and weakens correlations.

Example. If the average monthly bill is Rs 800, every blank bill is filled with Rs 800.

Related: Median imputation, Mode imputation, Missing value imputation, Variance

Replacing missing values in a numeric column with the median, the middle value. It is less affected by outliers than the mean, so it is better for skewed features such as income.

Example. With incomes of 20k, 25k, 30k and 5,00k, the median is 27.5k, a more typical fill than the mean of 143.8k.

Related: Mean imputation, Mode imputation, Missing value imputation, Outlier

A scaling method that linearly maps a feature to a fixed range, usually 0 to 1, using the smallest and largest values seen in training data: x_scaled = (x - min) / (max - min). It preserves the shape of the distribution but is sensitive to extreme values, which squash the rest into a narrow band.

Example. Marks out of 100 in a class range from 40 to 90. A mark of 65 becomes (65 - 40) / (90 - 40) = 0.5.

Related: Standardization, Feature scaling, Robust scaling, Outlier

A missingness pattern where the chance of a value being missing depends on other observed variables but not on the missing value itself. Imputation using the observed columns can then give unbiased results.

Example. Younger respondents skip an income question more often; since age is recorded, income is missing at random given age.

Related: Missing completely at random, Missing not at random, Multiple imputation, Iterative imputation

A missingness pattern where the chance a value is missing is unrelated to any data, observed or unobserved. Dropping such rows only reduces sample size without adding bias, but this assumption is often unrealistic.

Example. A sensor logs nothing on random days because of a power glitch unrelated to the readings, so those gaps are missing completely at random.

Related: Missing at random, Missing not at random, Missing value imputation, Listwise deletion

An extra binary column marking whether a value was missing in the original row. It lets a model learn from the fact of missingness itself, which is often informative, while the original column is imputed.

Example. If applicants who leave 'income' blank default more often, an income_missing flag lets the model exploit this.

Related: Missing value imputation, Missing not at random, Constant imputation, Feature engineering

A missingness pattern where whether a value is missing depends on the missing value itself. It is the hardest case, as bias cannot be fixed with the data alone and needs domain reasoning or a model of the missingness.

Example. High earners leave the income field blank because it is high, so the blanks hide exactly the values that matter.

Related: Missing at random, Missing completely at random, Missing indicator, Selection bias

Filling in missing entries with estimated values so that models needing complete data can be used. Simple rules use the mean, median or most frequent value; advanced ones predict the value from other columns. The choice can bias results.

Example. Missing ages in a customer table are replaced with the median age of 34.

Related: Mean imputation, Median imputation, KNN imputation, Missing indicator

Replacing missing values in a categorical column with its most frequent category. It is quick and keeps the data type, but can exaggerate the dominant category and hide missingness patterns.

Example. If 70 percent of customers list 'Mumbai' and some rows are blank, those blanks become 'Mumbai'.

Related: Median imputation, Missing value imputation, Categorical variable, Missing indicator

A statistical method that creates several completed copies of a dataset with different plausible fills for missing values, analyses each and combines the results. It reflects the uncertainty due to missing data instead of pretending the fill is known.

Example. Five imputed datasets are created, a regression is fitted on each, and the coefficients are pooled using Rubin's rules.

Related: Iterative imputation, Missing value imputation, Missing at random, Confidence interval

A measure of how much knowing one variable reduces uncertainty about another, based on entropy. Unlike correlation it captures non-linear relationships, so it is used to rank features against the target.

Example. Mutual information between hour_of_day and traffic congestion is high even though their linear correlation is near zero because the pattern rises and falls.

Related: Filter method, Entropy, Correlation coefficient, Feature selection

A categorical variable whose values are names with no natural order or ranking. Encoding must not imply any order, so methods like one-hot encoding are preferred over plain integer codes for linear and distance-based models.

Example. Blood group (A, B, AB, O) or Indian state names have no inherent ranking, so they are nominal.

Related: Ordinal variable, Categorical variable, One-hot encoding, Label encoding

Encoding a categorical feature by creating one new binary column per category, with 1 in the column matching the row's category and 0 elsewhere. It avoids implying any order among categories, but adds many columns when a feature has many categories.

Example. Payment type with UPI, card and cash becomes three columns; a UPI payment is [1, 0, 0] and a cash payment is [0, 0, 1].

Related: Dummy variable, Label encoding, High-cardinality feature, Sparse matrix

Mapping ordered categories to integers that respect their ranking, such as Low = 1, Medium = 2, High = 3. It keeps the order information in a single column, but assumes the steps between levels are roughly meaningful.

Example. Education level mapped to 1 for 10th, 2 for 12th, 3 for graduate, 4 for postgraduate keeps the natural ranking in one column.

Related: Ordinal variable, Label encoding, One-hot encoding, Categorical variable

A categorical variable whose values have a meaningful order but no guaranteed equal spacing between levels. Preserving that order in the encoding, with integers such as 1, 2, 3, lets a model use the ranking.

Example. Customer satisfaction rated Poor, Average, Good, Excellent is ordinal; Excellent is above Good but the gap is not a fixed amount.

Related: Nominal variable, Ordinal encoding, Categorical variable, Label encoding

The set of choices for handling extreme values: keep them, cap them, transform them or remove them. The right choice depends on whether they are errors or real rare events, so it needs domain judgement.

Example. A recorded age of 250 is a data-entry error to fix, while a Rs 5 crore purchase in an e-commerce dataset may be real and worth keeping.

Related: Winsorization, IQR method, Z-score, Robust scaling

Balancing classes by increasing the number of minority-class examples, either by copying existing ones or generating new ones. It keeps all data but can cause overfitting when copies are exact, and must be applied only to training data.

Example. Duplicating the 200 fraud cases 10 times gives 2,000 fraud examples in the training set.

Related: Undersampling, SMOTE, Class imbalance, Data leakage

New features made from powers and products of existing ones, such as x^2, x^3 and x1*x2. They let a linear model fit curved relationships and interactions, but the number of columns grows rapidly with degree.

Example. From features a and b with degree 2, you get a, b, a^2, ab and b^2, letting a linear model fit a curved surface.

Related: Interaction feature, Polynomial regression, Overfitting, Feature transformation

A family of monotonic functions, such as square root, cube root, Box-Cox and Yeo-Johnson, that raise values to a power to make a distribution more symmetric and closer to normal. The best power is usually estimated from the data.

Example. Applying a square root to a right-skewed count feature such as website visits per user pulls in the long tail before fitting a linear model.

Related: Box-Cox transform, Yeo-Johnson transform, Log transform, Skewness

A transformation that maps a feature's values to their rank-based quantiles, so the result follows a chosen distribution, typically uniform or normal. It sharply reduces the effect of outliers and can straighten odd distributions, but it distorts distances between values.

Example. Applying a uniform quantile transform to salaries makes the median salary 0.5 and the top 1 percent all lie between 0.99 and 1, regardless of how huge the maximum is.

Related: Robust scaling, Outlier, Percentile, Power transform

Merging infrequent category values into a single 'Other' bucket before encoding. It reduces column count, stops the model overfitting to tiny groups and prepares for unseen categories at prediction time.

Example. Of 300 car brands in a dataset, all brands with under 50 rows are relabelled 'Other', leaving about 20 distinct values.

Related: High-cardinality feature, One-hot encoding, Overfitting, Unknown category handling

A feature formed by dividing one quantity by another to give a normalised, comparable measure. Ratios often carry more signal than raw amounts because they adjust for size or scale.

Example. For loans, EMI divided by monthly income (EMI-to-income ratio) predicts default better than either number alone.

Related: Interaction feature, Domain knowledge, Feature engineering, Aggregation feature

A wrapper method that trains a model, removes the least important feature or features, and repeats on the remaining set until the desired number is reached. Importance comes from coefficients or tree importances.

Example. Starting with 50 features, remove the weakest 5 each round until 10 remain, retraining every time.

Related: Wrapper method, Feature importance, Feature selection, Forward selection

A scaling method that subtracts the median and divides by the interquartile range (Q3 - Q1) instead of using mean and standard deviation. Because medians and quartiles barely move when extreme values appear, it keeps typical data well scaled in the presence of outliers.

Example. For house prices with a few Rs 50 crore mansions, robust scaling centres on the median price and keeps most homes in a sensible range, where min-max would squash them near 0.

Related: Min-max scaling, Standardization, Interquartile range, Outlier

Summary values such as mean, median or standard deviation computed over a sliding window of consecutive observations. They smooth noise, describe local trend and volatility, and are common time-series features.

Example. A 7-day rolling mean of daily orders smooths weekday spikes and reveals the underlying trend.

Related: Lag feature, Window feature, Time series, Moving average

Per-row numbers that tell a model how much each training example should count in the loss. They are used to reflect importance, recency or sampling design, and to compensate for imbalance.

Example. Recent transactions get weight 2 and older ones weight 1, so the model attends more to current behaviour.

Related: Class weights, Class imbalance, Loss function, Cost-sensitive learning

Synthetic Minority Over-sampling Technique: it creates new minority-class examples by picking a minority point and one of its nearest minority neighbours and generating a point at a random position on the line between them. It reduces the overfitting of plain duplication.

Example. Between two fraud cases with amounts 900 and 1,100, SMOTE might create a synthetic fraud with amount 1,020, with other features interpolated similarly.

Related: Oversampling, ADASYN, Class imbalance, K-nearest neighbours

A scaling method that shifts a feature to mean 0 and stretches it to standard deviation 1: z = (x - mean) / std. Each value then says how many standard deviations it lies from the average. It suits algorithms that assume centred data, such as logistic regression, PCA and neural networks.

Example. If the average delivery time is 30 minutes with a standard deviation of 8, a 46-minute delivery has z = (46 - 30) / 8 = 2.

Related: Z-score, Min-max scaling, Feature scaling, StandardScaler

Cutting words down to a crude root by stripping endings using fixed rules, so 'running', 'runs' and 'runner' map to similar stems. It reduces vocabulary size quickly but may produce non-words.

Example. The Porter stemmer reduces 'studies' and 'studying' to 'studi'.

Related: Lemmatization, Text preprocessing, Stop words, Bag of words

A family of greedy feature selection procedures that add or remove one variable at a time based on a criterion such as p-value, AIC or cross-validated score, sometimes allowing both moves in the same run. It is quick but can overfit and is unstable.

Example. Starting from an empty regression, add the best predictor, then check whether any earlier variable should now be dropped, and repeat until AIC stops improving.

Related: Forward selection, Backward elimination, Akaike information criterion, Feature selection

Very common words, such as 'the', 'is' and 'of', that carry little meaning for many text tasks. Removing them shrinks the vocabulary and can reduce noise, though for some tasks, such as sentiment, they matter.

Example. Filtering 'the', 'a' and 'is' from 'the food is great' leaves 'food great'.

Related: Bag of words, Tokenization, Stemming, Text preprocessing

Replacing each category with a statistic of the target for that category, usually its mean, so one numeric column captures how predictive the category is. It handles many categories compactly but leaks target information unless computed out-of-fold or smoothed.

Example. If loans from Pincode 560001 default 4 percent of the time in training data, every row with that pincode gets the value 0.04.

Related: Mean encoding, Data leakage, High-cardinality feature, Smoothing

The cleaning steps applied to raw text before modelling, such as lowercasing, removing punctuation and stop words, tokenising and stemming or lemmatising. It standardises text so equal words are treated equally.

Example. 'Great Food!!' becomes ['great', 'food'] after lowercasing, stripping punctuation and tokenising.

Related: Tokenization, Stop words, Stemming, Lemmatization

Choosing the probability cutoff at which a model's score is turned into a class label, instead of defaulting to 0.5. On imbalanced data, moving the threshold is a simple way to trade precision against recall.

Example. Lowering the fraud threshold from 0.5 to 0.2 catches more frauds (higher recall) but flags more genuine transactions (lower precision).

Related: Class imbalance, Precision, Recall, Cost-sensitive learning

A pair of points from opposite classes that are each other's nearest neighbour. Removing the majority-class member of each pair cleans the class boundary and reduces noise, and is used as an undersampling technique.

Example. If a non-fraud point and a fraud point are each other's nearest neighbour, the non-fraud one is dropped.

Related: Undersampling, SMOTE, Class imbalance, Decision boundary

A mismatch between the features a model saw in training and those it receives in production, caused by different code paths, data sources or timing. It silently degrades performance and is prevented by sharing the same feature logic in both places.

Example. Training computed 'average spend' with Python, while production uses an SQL query that treats nulls differently, so values differ for the same customer.

Related: Feature store, Pipeline, Data drift, Data validation

Balancing classes by removing examples from the majority class so the classes end up closer in size. It speeds training but throws away information, so it suits very large datasets.

Example. Randomly keeping 2,000 of 99,800 non-fraud cases beside 200 frauds gives a much more balanced training set.

Related: Oversampling, SMOTE, Class imbalance, Tomek links

Rescaling each row (sample) so that its vector has length 1, usually using the L2 norm. It keeps the direction of the row but discards its magnitude, which is useful when only the relative pattern matters, as in cosine-similarity comparisons of text vectors.

Example. A row [3, 4] has L2 length 5, so it becomes [0.6, 0.8] after row-wise normalisation.

Related: L2 norm, Cosine similarity, TF-IDF, Feature scaling

The policy for categories seen at prediction time but absent from training data. Common options are ignoring them (all-zero one-hot row), mapping them to an 'unknown' bucket or raising an error. Without a plan, production pipelines crash or misbehave.

Example. A model trained before a new city launched sees 'Kochi' in production; with handle_unknown set to ignore, the one-hot row is simply all zeros.

Related: One-hot encoding, Rare category grouping, Training-serving skew, Pipeline

A simple filter method that drops features whose variance is below a chosen cutoff, since a column that barely changes across rows carries little information for prediction. A constant column has zero variance.

Example. A column where 99.9 percent of rows are 'India' has almost no variance and gets removed.

Related: Filter method, Variance, Feature selection, Constant feature

An encoding from credit scoring that replaces a category or bin by the natural log of the ratio of the share of good outcomes to the share of bad outcomes: WoE = ln(% good / % bad). It creates a monotonic, interpretable signal for logistic regression scorecards.

Example. If an income band holds 30 percent of repayers and 10 percent of defaulters, its WoE is ln(0.30 / 0.10) = 1.10.

Related: Information value, Target encoding, Binning, Logistic regression

A feature calculated over a fixed span of recent history, such as the sum of purchases in the last 7 days. Time windows capture recent behaviour and trends and must only use data available before the prediction time.

Example. For a churn model, sessions_last_7d and sessions_last_30d are window features; a sharp drop between them signals waning interest.

Related: Aggregation feature, Rolling statistics, Lag feature, Temporal leakage

Limiting extreme values by replacing those beyond chosen percentiles with the value at that percentile, for example capping at the 1st and 99th percentiles. It reduces outlier influence without discarding rows.

Example. Order values above the 99th percentile of Rs 45,000 are all set to Rs 45,000.

Related: Outlier treatment, IQR method, Percentile, Robust scaling

A feature selection approach that evaluates subsets of features by actually training a model on each and measuring performance. It accounts for interactions and suits the chosen model, but is computationally expensive.

Example. You train a logistic regression on many candidate subsets and keep the one with the best cross-validated accuracy.

Related: Filter method, Embedded method, Recursive feature elimination, Forward selection

An extension of the Box-Cox power transform that also works with zero and negative values. Like Box-Cox, it estimates a parameter from the data to make the feature more symmetric and normal-looking, making it a general-purpose choice.

Example. A profit column containing losses and gains, such as -5,000 to 80,000, can be Yeo-Johnson transformed, though Box-Cox would fail.

Related: Box-Cox transform, Power transform, Skewness, Feature transformation