CTFdown

A CLI tool for downloading and managing CTF challenges from CTFd platforms. CTFdown provides efficient downloading, filtering, validation, and archiving capabilities for CTF competitions.

Features

• Resume/Update Mode - Only download new or changed challenges using hash-based change detection
• Parallel Downloads - Optional parallel file downloads for improved performance
• Progress Indicators - Visual progress bars using tqdm (optional)
• Retry Logic - Automatic retries with exponential backoff for failed requests
• Challenge Filtering - Filter by category, tags, points, solved status, and challenge IDs
• Selective Updates - Smart update detection using file hashes to avoid redundant downloads
• Command-Line Interface - Clean argparse-based CLI with organized help and examples
• File Integrity Checks - SHA256 hash verification for all downloaded files
• Challenge Validation - Validate challenge structure and completeness
• Archive Creation - Create ZIP or TAR.GZ archives of downloaded challenges
• State Management - Persistent state tracking for efficient updates

Installation

Prerequisites

• Python 3.7 or higher
• pip package manager

Install Dependencies

pip install -r requirements.txt

Make Executable (Optional)

chmod +x ctfdown.py

Quick Start

Basic Usage

Download all challenges from a CTFd platform:

ctfdown https://ctf.example.com SESSION_COOKIE

Getting Your Session Cookie

1. Log in to the CTFd platform in your browser
2. Open Developer Tools (F12)
3. Navigate to Application/Storage → Cookies
4. Find the session cookie value
5. Copy the value and use it as the session_cookie argument

Usage Examples

Download All Challenges

ctfdown https://lakectf.epfl.ch 'your_session_cookie'

Update Existing Challenges

Only download new or changed challenges:

ctfdown https://ctf.example.com SESSION_COOKIE --update

Filter by Category

Download only specific categories:

ctfdown https://ctf.example.com SESSION_COOKIE --category pwn,web

Filter by Points Range

Download challenges within a points range:

ctfdown https://ctf.example.com SESSION_COOKIE --min-points 100 --max-points 500

Download Unsolved Challenges

Download only challenges you haven't solved:

ctfdown https://ctf.example.com SESSION_COOKIE --unsolved-only

Parallel Downloads

Enable parallel file downloads for faster performance:

ctfdown https://ctf.example.com SESSION_COOKIE --parallel

Validate Challenges

Validate all downloaded challenges for completeness:

ctfdown https://ctf.example.com SESSION_COOKIE --validate

Create Archive

Create a ZIP archive after downloading:

ctfdown https://ctf.example.com SESSION_COOKIE --zip --archive-name MyCTF

Combined Options

Combine multiple features:

ctfdown https://ctf.example.com SESSION_COOKIE \
    --update \
    --parallel \
    --validate \
    --category pwn,web \
    --zip \
    --archive-name PwnWebChallenges

Command-Line Reference

Positional Arguments

Argument	Description
url	CTFd platform base URL (e.g., https://ctf.example.com)
session_cookie	Session cookie value for authentication

Filtering Options

Option	Description
--category CATEGORY	Filter by category (comma-separated list)
--tags TAGS	Filter by tags (comma-separated list)
--solved-only	Only download challenges you have solved
--unsolved-only	Only download challenges you haven't solved
--min-points MIN_POINTS	Minimum points filter (integer)
--max-points MAX_POINTS	Maximum points filter (integer)
--challenge-ids CHALLENGE_IDS	Specific challenge IDs (comma-separated list)

Mode Options

Option	Description
--update	Update mode: only download new/changed challenges
--parallel	Download files in parallel (faster but more resource-intensive)
--validate	Validate downloaded challenges for completeness
--inspect	Inspect API structure without downloading
--no-progress	Disable progress bars

Archive Options

Option	Description
--zip	Create ZIP archive after downloading
--tar, --tar.gz, --tgz	Create TAR.GZ archive after downloading
--archive-name ARCHIVE_NAME	Custom archive name (default: CTF_Challenges)

General Options

Option	Description
-o, --output-dir OUTPUT_DIR	Output directory for challenges (default: challenges)
-h, --help	Show help message and exit

Advanced Options

Option	Description	Default
--max-workers MAX_WORKERS	Maximum number of parallel workers	4
--max-retries MAX_RETRIES	Maximum retry attempts for failed requests	3
--retry-backoff RETRY_BACKOFF	Retry backoff factor (seconds)	1.0

Output Structure

CTFdown organizes downloaded challenges in the following structure:

challenges/
├── .ctfdown_state.json          # State file for update tracking
├── category1/
│   ├── challenge1/
│   │   ├── Title_Description.txt
│   │   └── challenge_files/
│   │       ├── file1.zip
│   │       └── file2.txt
│   └── challenge2/
│       └── Title_Description.txt
└── category2/
    └── challenge3/
        ├── Title_Description.txt
        └── challenge_files/
            └── file3.tar.gz

Title_Description.txt Format

Each challenge includes a Title_Description.txt file with the following information:

Title: Challenge Name
Category: category_name
Type: challenge_type
Points: 100
Solves: 42
Tags: tag1, tag2
Attribution: Author Name

Description:
Challenge description text with preserved links.

Connection Info:
nc chall.example.com 1337

Hints:
  1. Hint text here
  2. Another hint

Solution:
Solution text (if visible)

State Management

CTFdown maintains a .ctfdown_state.json file in the output directory to track:

• Challenge metadata hashes for change detection
• Downloaded file hashes for integrity verification
• Update state for efficient resume/update operations

This enables:

• Update Mode: Only download challenges that have changed
• Integrity Verification: Verify file integrity on subsequent runs
• Resume Capability: Resume interrupted downloads

How It Works

1. Authentication: Uses session cookie for API authentication
2. Challenge Fetching: Retrieves challenge list from CTFd API
3. Filtering: Applies user-specified filters (category, tags, points, etc.)
4. Change Detection: Compares challenge hashes with stored state
5. Download: Downloads challenge metadata and files
6. Validation: Validates challenge structure and file integrity
7. State Update: Updates state file with new hashes

Error Handling

CTFdown includes robust error handling:

• Automatic Retries: Failed requests are automatically retried with exponential backoff
• Graceful Degradation: Continues processing even if individual challenges fail
• Error Reporting: Clear error messages for troubleshooting
• State Preservation: State file is saved even if download is interrupted

Performance Considerations

• Parallel Downloads: Use --parallel for faster downloads, but be mindful of server load
• Update Mode: Use --update to avoid redundant downloads
• Progress Bars: Disable with --no-progress for cleaner output in scripts
• Worker Count: Adjust --max-workers based on your system and network capacity

Troubleshooting

Connection Issues

If you encounter connection errors:

1. Verify the CTFd URL is correct
2. Check that your session cookie is valid and not expired
3. Ensure you have network connectivity
4. Try increasing --max-retries and --retry-backoff

Missing Files

If challenge files are missing:

1. Check that you have access to the challenge (not locked or hidden)
2. Verify your session cookie has proper permissions
3. Run with --validate to check for issues

Update Mode Not Working

If update mode isn't detecting changes:

1. Check that .ctfdown_state.json exists in the output directory
2. Delete the state file to force a full re-download
3. Verify file permissions on the output directory

Contributing

Contributions are welcome! Please feel free to submit issues, feature requests, or pull requests.

License

MIT License

Acknowledgments

• Built for the CTF community
• Compatible with CTFd platform
• Inspired by the need for efficient challenge management