This repository contains our solution for the NIA Alzheimer's Disease and Related Dementias (ADRD) Challenge (DrivenData Competition). The goal of this challenge is to classify speech recordings into three categories:
- ADRD (Alzheimer's Disease and Related Dementias)
- MCI (Mild Cognitive Impairment)
- Healthy Controls
Our approach is based on a fusion of multimodal deep learning models, incorporating the following components:
- Acoustic Transformers: Features extracted using Whisper or mHuBERT.
- Linguistic Transformer: Text-based embeddings obtained from MultilingualGTE.
- Demographic Features: Age-based embeddings to enhance classification accuracy.
By combining these modalities, our model aims to improve the robustness and accuracy of ADRD detection from speech data.
To run this project successfully, you need to install Python 3.10.0.
If you're using Linux or macOS, run the following commands:
sudo apt update
sudo apt install -y software-properties-common
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update
sudo apt install -y python3.10 python3.10-venv python3.10-devThen make sure to use Python 3.10:
python3.10 --versionIf you're using Windows, download Python 3.10.0 from the official site:
📎 https://www.python.org/downloads/release/python-3100/
├── bias # Scripts for bias mitigation
├── data # Contains final preprocessed CSV/Excel files
├── dataset # Dataset-related utility functions
├── fairness_analysis # Scripts to evaluate fairness metrics
├── model # Model architecture definitions
├── performance_analysis/ # Tools for AUC curve plotting and performance comparison
├── preprocessing/ # Full preprocessing pipeline: filtering, transcription, demographic prep
├── SpeechCARE_architecture_visualization # Visual diagrams of the framework architecture
├── utils # Utility scripts shared across components
│
├── config.yml # Main configuration file with paths, hyperparameters, etc.
├── inference.py # Script for running single-audio inference
├── main.py # Core training and batch inference script
├── requirements.txt # Python dependencies
└── README.md # Project documentation
- Make sure to update
config.ymlbefore running the training script. - Ensure that the dataset paths and model checkpoints are correctly set.
