Feature engineering is the process of transforming raw data into the input variables, or features, that a machine learning model uses, including designing, selecting, and transforming them. Typical examples include extracting the day of the week from a date, combining columns into a new variable, encoding categorical values as numbers, and normalizing value ranges. In traditional machine learning it is a key step that strongly affects model performance and requires knowledge of the data and the domain.
Tabular models such as XGBoost require feature engineering to be repeated for each task, whereas deep learning reduces this burden by learning representations directly from raw data. Foundation models for tabular data are an attempt to reduce it further.