Cafe Sales Data Cleaning Pipeline

A Python data cleaning pipeline built with Pandas to process, impute, and standardize messy cafe sales data.

Project Overview

This project processes raw cafe sales data containing invalid string markers, missing timestamps, incomplete categorical metadata, and missing financial metrics. Instead of dropping rows indiscriminately, the pipeline applies deterministic mathematical reconstruction, item-based statistical imputation, and type normalization to maximize data retention.

Data Cleaning & Imputation Rules

Feature / Category	Problem / Issue	Applied Rule & Strategy
Error Standardization	String placeholders ("ERROR", "UNKNOWN", empty strings)	Replaced with standard NumPy NaN across all columns.
Numeric Casting	String representations of numbers	Cast to float/int using pd.to_numeric(..., errors='coerce').
Deterministic Imputation	Missing Total Spent, Price Per Unit, or Quantity	Reconstructed via inverse mathematical formulas: • $\text{Total Spent} = \text{Quantity} \times \text{Price Per Unit}$ • $\text{Price Per Unit} = \text{Total Spent} / \text{Quantity}$ • $\text{Quantity} = \text{Total Spent} / \text{Price Per Unit}$
Statistical Fallback	Completely missing prices	Imputed using the median price per Item across the historical dataset.
Categorical Imputation	Missing Payment Method or Location	Filled with "Unknown" to retain valid financial transaction records.
Date Normalization	Invalid, empty, or whitespace-only dates	Stripped and parsed via pd.to_datetime(..., errors='coerce'). Missing dates are set to NaT.
Row Retention Policy	Corrupted records	Rows are dropped only when both Item and financial columns cannot be recovered.

Repository Structure

.
├── clean_data.py          # Core data cleaning script
├── cafe_sales_cleaned.csv # Standardized output dataset
├── .gitignore             # Excludes venv, cache, and OS metadata
└── README.md              # Project documentation

How to Run

Clone the repository:

• Bash git clone https://github.com/Jacopos311/Cafe-sales-data-pipeline cd cafe-sales-data-pipeline Set up virtual environment & install dependencies:

• Bash python3 -m venv venv source venv/bin/activate pip install pandas datasets Execute the cleaning pipeline:

• Bash python clean_data.py