Fuzz Harness Expiry

This repository contains artifacts for the "An Empirical Study of Fuzzing Harness Degradation" paper. Including scraping code, analysis notebook, and case studies.

• The Rust code is used for scraping the data. Note that Monorail was replaced the Google Issue Tracker and this code will need to be updated.
• The analysis is available as a Julia Pluto notebook in harnesses_pluto.jl.
• The notes on the case studies are available in case_studies/notes.md.
• The notes on case study resolutions between the judges is available in case_studies/judge_mismatches.pdf.

Setup

To improve reproducibility, we provide a Nix flake environment, for reference see:

• https://nixos.org/learn/, to install nix
• https://wiki.nixos.org/wiki/Flakes, for information on nix flakes

Versions

If you want to run the analysis without nix, we used the following versions of the tools:

$ julia -v
julia version 1.11.5

$ cargo --version
cargo 1.88.0 (873a06493 2025-05-10)

Running the Analysis Notebook

The analysis is written as a Pluto notebook. Which requires the database and Julia.

Note

If you are using nix, Julia will be installed as part of the nix develop commands.

To download the database, run the following:

./download_db.sh

Run this once to install the Pluto package, other packages are installed as part of the notebook:

nix --experimental-features 'nix-command flakes' develop -c julia -e 'import Pkg; Pkg.add("Pluto")'

Then run the notebook:

nix --experimental-features 'nix-command flakes' develop -c julia --project=analysis -e "import Pluto; Pluto.run(require_secret_for_open_links=false, require_secret_for_access=false)"

To open the notebook access the URL printed in the terminal and under Open a notebook select harnesses_pluto.jl. The notebook will require around 15GB of memory and can take over 10 minutes to run.

Collecting Fuzz Introspector alerts

To gather the current alerts shown in the per project overview of Fuzz Introspector run the following command. Note that this will issue around 500 http requests, parsing the results.

cargo run --bin introspector_scraper -- --bulk alert_data/projects.json --output introspector_alerts.json --concurrent 10

# For data collected with the command above.
python3 analyze_coverage_alerts.py introspector_alerts.json
# For the data shown in the paper
python3 analyze_coverage_alerts.py alert_data/introspector_alerts.json

Running the Scraping Code

Warning

Note that this scraping code currently will not work as Monorail was replaced by the Google Issue Tracker. The code will need to be updated.

Also the number of threads are optimized for the machine we used for our study.

The scraping process used more than 1TB of network bandwidth, as well as 250GB of disk space.

The scraping code can be run using:

nix --experimental-features 'nix-command flakes' develop -c cargo run --release"