TikTok Virality Prediction

A clean PyTorch multimodal deep learning system for predicting TikTok video virality. Combines text (Whisper + CLIP), visual (CLIP), and audio features (CNN) to predict engagement.

What is this?

This project predicts how "viral" a TikTok video will be by analyzing:

• Visual content: Frames extracted from the video
• Audio: Spectral features and MFCCs
• Text: Transcriptions, captions, and hashtags

Uses a unified multimodal model (CLIP + Audio CNN) trained on 1,000 trending TikTok videos.

Setup

Prerequisites

• Python 3.13+
• uv package manager
• Kaggle API credentials
• FFmpeg
• OpenAI API key (for Whisper transcription)
• Apify API token (for batch inference)

Installation

# Install dependencies
uv sync

# Configure environment variables
cp .env.example .env
# Edit .env and add your API keys:
#   OPENAI_API_KEY=your-openai-api-key
#   APIFY_API_TOKEN=your-apify-api-token
#   KAGGLE_USERNAME=your-kaggle-username
#   KAGGLE_KEY=your-kaggle-api-key

Note: You can also use ~/.kaggle/kaggle.json for Kaggle credentials instead of .env

How to Run

All configuration is in config.yaml. Edit it to customize paths, model architecture, and hyperparameters.

Training

# 1. Download dataset from Kaggle
python src/data/download.py

# 2. Preprocess videos (extract features)
python src/data/preprocess.py

# 3. Train the model
python src/training/train.py

Batch Inference & Evaluation

Create data/inference/input.txt with TikTok URLs (one per line):

https://www.tiktok.com/@user/video/123456789
https://www.tiktok.com/@user/video/987654321

Make sure APIFY_API_TOKEN is set in your .env file, then run:

python src/batch_inference.py

This will:

1. Scrape videos from Apify
2. Download and process each video
3. Generate predictions using the trained model
4. Calculate accuracy metrics (MAE, RMSE, correlation, R²)
5. Export results to data/inference/output.xlsx

The Excel output contains:

• Results sheet: URL, predictions, actual virality, TikTok stats, transcriptions
• Accuracy Metrics sheet: MAE, RMSE, MAPE, correlation, R²

Configuration Tips

• Speed up preprocessing: Set skip_transcription: true in config to skip Whisper API calls
• Reduce memory: Set freeze_clip: true and lower batch_size
• Device: Change device: cuda to cpu or mps depending on your hardware

Type Checking

The project uses pyright for static type checking:

uv run pyright src/

Dataset

Uses TikTok Trending December 2020 from Kaggle (1,000 videos with engagement metrics).

License

Educational purposes only.