Project Title: GraphGE - Uncertainty-Aware Fraud Detection
Date: December 15, 2025
Dataset: Elliptic Bitcoin Dataset
Model: GraphSAGE with Monte Carlo Dropout
Focus: Bayesian Uncertainty Quantification and Calibration
This report details the implementation of a Graph Neural Network (GNN) for fraud detection on the Elliptic Bitcoin dataset. The primary objective was to establish principled uncertainty quantification using Monte Carlo Dropout, ensuring reliable confidence estimates for high-stakes financial applications. The system prioritizes well-calibrated predictions (ECE < 0.05) and provides deep insights through epistemic and aleatoric uncertainty decomposition.
The implementation features a 2-layer GraphSAGE model with MC Dropout (T=30), class imbalance handling, and advanced feature engineering. Temporal drift analysis was conducted to demonstrate the model's awareness of distribution shifts over time.
- Dataset: Elliptic Bitcoin Dataset (203,769 nodes)
- Preprocessing: RobustScaler for feature normalization and addition of normalized node degree features.
- Handling Imbalance: Inverse frequency weighting (Fraud weight: 7.63x) to address class skewness.
- Framework: GraphSAGE with 2 convolutional layers and 64 hidden dimensions.
- Uncertainty Mechanism: Monte Carlo Dropout for Bayesian approximation of the posterior distribution.
- Loss Function: Negative Log-Likelihood (NLL) with class-specific penalties.
- Performance: F1-Score, PR-AUC.
- Uncertainty: Expected Calibration Error (ECE), Entropy-AUC.
- Analysis: Decomposition of total uncertainty into Epistemic (model-related) and Aleatoric (data-related) components.
- Baseline F1: 0.3320
- Tuned F1: 0.4209 (+8.9% via validation-based threshold optimization)
- Calibration (ECE): 0.0450 (highly calibrated)
- Separation: Uncertainty effectively distinguishes correct from incorrect predictions.
| Dropout | F1 Score | ECE | Notes |
|---|---|---|---|
| 0.0 | 0.3044 | 0.1658 | Poor calibration without regularization |
| 0.2 | 0.3237 | 0.1052 | Optimal balance of performance and uncertainty |
| 0.5 | 0.2898 | 0.0747 | Strong calibration, conservative performance |
Integrating degree features improved the F1-score by 3% and significantly enhanced the model's ability to separate correct predictions from errors based on entropy levels.
Statistical analysis (p < 0.05) revealed a significant increase in predictive uncertainty over time, confirming the model's sensitivity to temporal shifts in the Bitcoin transaction graph.
- Reliable Confidence: MC Dropout provides actionable uncertainty estimates suitable for automated triage systems.
- Superior Calibration: ECE levels under 0.05 ensure that predicted probabilities accurately reflect true success rates.
- Drift Awareness: The model's increasing uncertainty on newer data provides a natural trigger for retraining or manual oversight.
GNN_Fraud_Detection_Pipeline.ipynb: Main model training and evaluation pipeline.Uncertainty_Quantification_Study.ipynb: In-depth analysis of Bayesian uncertainty metrics.Extended_Experimental_Ablations.ipynb: Comprehensive sensitivity analysis and ablation studies.
graphge/src/: Modularized source code for data loading, models, and uncertainty functions.graphge/results/: Performance metrics and visualization artifacts (reliability diagrams, risk-coverage curves).
The GraphGE system demonstrates a robust approach to fraud detection by integrating GNNs with Bayesian uncertainty quantification. The results validate that focusing on calibration and uncertainty quality provides a more reliable foundation for deployment than optimizing for raw accuracy alone. This work addresses critical deployment concerns, including model drift and risk assessment, making it suitable for production-level financial monitoring.