Skip to content

Commit 1e49f17

Browse files
jonn-smithclaude
andcommitted
Add fetch_refs.sh to download BFVD + ICTV backing data
Idempotent wget-based shell script that pulls the three BFVD metadata/taxid/lineage TSVs from steineggerlab and the current ICTV VMR MSL41 xlsx into refs/downloads/. README updated with a new step 0 that runs before build_bfvd_refs / build_host_table. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1 parent d3dce06 commit 1e49f17

2 files changed

Lines changed: 72 additions & 0 deletions

File tree

analysis/hvp_viral_viz/README.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,9 @@ External fill (host species, see [`HOST_FILL.md`](HOST_FILL.md)):
7070
## Pipeline
7171

7272
```bash
73+
# 0. Fetch backing data sources (BFVD TSVs + ICTV VMR xlsx) — one-time
74+
bash fetch_refs.sh --out-dir refs/downloads
75+
7376
# 1. Build reference joins (one-time)
7477
PYTHONPATH=.. python -m hvp_viral_viz.build_bfvd_refs
7578
PYTHONPATH=.. python -m hvp_viral_viz.uniprot_host --source bfvd
Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
#!/usr/bin/env bash
2+
# Fetch the backing data sources required by build_bfvd_refs.py + build_host_table.py.
3+
#
4+
# Downloads (idempotent — skips files already present and non-empty):
5+
# 1. BFVD metadata + taxid + lineage TSVs from steineggerlab.
6+
# 2. ICTV VMR MSL41 xlsx from ictv.global.
7+
#
8+
# Usage:
9+
# bash fetch_refs.sh [--out-dir DIR]
10+
#
11+
# Then point the build scripts at the downloaded files:
12+
# python -m hvp_viral_viz.build_bfvd_refs --bfvd-dir <out>/bfvd --out-dir refs
13+
# python -m hvp_viral_viz.build_host_table --refs-dir refs --ictv-vmr <out>/ictv_vmr_msl41.xlsx
14+
15+
set -euo pipefail
16+
17+
OUT_DIR="./refs/downloads"
18+
while [[ $# -gt 0 ]]; do
19+
case "$1" in
20+
--out-dir) OUT_DIR="$2"; shift 2 ;;
21+
-h|--help) sed -n '2,15p' "$0"; exit 0 ;;
22+
*) echo "unknown arg: $1" >&2; exit 2 ;;
23+
esac
24+
done
25+
26+
mkdir -p "$OUT_DIR/bfvd"
27+
28+
# ---- BFVD ----------------------------------------------------------------
29+
BFVD_BASE="https://bfvd.steineggerlab.workers.dev/latest"
30+
BFVD_FILES=(
31+
"bfvd_metadata.tsv"
32+
"bfvd_taxid.tsv"
33+
"bfvd_taxid_rank_scientificname_lineage.tsv"
34+
)
35+
36+
for f in "${BFVD_FILES[@]}"; do
37+
dest="$OUT_DIR/bfvd/$f"
38+
if [[ -s "$dest" ]]; then
39+
echo "[fetch_refs] skip BFVD $f (already present)"
40+
continue
41+
fi
42+
echo "[fetch_refs] downloading BFVD $f"
43+
wget -c -O "$dest.partial" "$BFVD_BASE/$f"
44+
mv "$dest.partial" "$dest"
45+
done
46+
47+
# ---- ICTV VMR ------------------------------------------------------------
48+
# Canonical filename build_host_table.py looks for: ictv_vmr_msl41.xlsx.
49+
# Upstream filename embeds release date and may change between MSLs; we
50+
# rename to the stable local name.
51+
ICTV_URL="https://ictv.global/sites/default/files/VMR/VMR_MSL41.v1.20260320.xlsx"
52+
ICTV_DEST="$OUT_DIR/ictv_vmr_msl41.xlsx"
53+
54+
if [[ -s "$ICTV_DEST" ]]; then
55+
echo "[fetch_refs] skip ICTV VMR (already present)"
56+
else
57+
echo "[fetch_refs] downloading ICTV VMR MSL41"
58+
wget -c -O "$ICTV_DEST.partial" "$ICTV_URL"
59+
mv "$ICTV_DEST.partial" "$ICTV_DEST"
60+
fi
61+
62+
# ---- Summary -------------------------------------------------------------
63+
echo
64+
echo "[fetch_refs] done. files in $OUT_DIR/:"
65+
ls -lh "$OUT_DIR/bfvd/" "$OUT_DIR/"*.xlsx 2>/dev/null || true
66+
echo
67+
echo "Next steps:"
68+
echo " python -m hvp_viral_viz.build_bfvd_refs --bfvd-dir $OUT_DIR/bfvd --out-dir refs"
69+
echo " python -m hvp_viral_viz.build_host_table --refs-dir refs --ictv-vmr $ICTV_DEST"

0 commit comments

Comments
 (0)