Bioinformatics-first AI shell for Nextflow and nf-core.
helixsh is a zero-dependency Python CLI that wraps Nextflow and nf-core pipelines with intent parsing, schema validation, resource estimation, audit trails, cloud cost estimates, HPC environment module generation, Seqera Platform integration, and more — all without any PyPI dependencies beyond Python 3.10+.
- Features
- Installation
- Quick Start
- Commands Reference
- Core Execution
- Intent Parsing
- nf-core Pipelines
- Samplesheet Tools
- Reference Genomes
- Bioconda Integration
- Trace & Cost Analysis
- Seqera Platform (Tower)
- HPC Environment Modules
- Snakemake Bridge
- Schema & Validation
- Resource Estimation
- AI Planning (MCP)
- Diagnostics
- Audit & Provenance
- Security & Compliance
- RBAC: Role-Based Access
- Environment Variables
- Configuration
- Architecture
- Local Development
- Packaging
| Category | Capabilities |
|---|---|
| Execution | Nextflow run with Docker, Podman, Singularity, Apptainer, Conda |
| Intent | Natural-language → Nextflow plan (RNA-seq, WGS, WES, ChIP-seq, ATAC-seq, …) |
| Pipelines | 20-pipeline bundled registry with version checks |
| Samplesheets | Validate and auto-generate nf-core CSV samplesheets |
| References | Download and cache GRCh38, GRCh37, GRCm39, TAIR10, and more |
| Bioconda | Search, install packages, create environments (dry-run safe) |
| Tracing | Parse Nextflow trace.txt into per-process summaries |
| Cost | AWS / GCP / Azure on-demand and spot cost estimates |
| Tower | Submit, monitor, and list compute envs on Seqera Platform |
| HPC | Generate modules.config for Lmod / Environment Modules clusters |
| Snakemake | Import rule-level resource declarations into helixsh calibration |
| Audit | SQLite provenance DB, HMAC-signed JSONL audit log |
| RBAC | Auditor / Analyst / Admin roles with per-command enforcement |
| MCP | Claude Code proposal → approve → execute pipeline |
git clone https://github.com/musicofthings/helixsh
cd helixsh
python -m venv .venv
source .venv/bin/activate
pip install -e .
helixsh doctorBuild a self-contained zipapp that runs on any Python 3.10+ without a venv:
./scripts/package_local.sh
./dist/helixsh.pyz doctorCopy dist/helixsh.pyz anywhere on your $PATH and rename it helixsh.
python -m helixsh doctor| Requirement | Notes |
|---|---|
| Python 3.10+ | No third-party packages needed |
| Nextflow 25.x | java 17+ must be on $PATH |
| Docker / Podman / Singularity / Apptainer | At least one container runtime |
| conda / mamba (optional) | For -with-conda workflows |
# Check your environment
helixsh doctor
# Parse a natural-language intent into a Nextflow plan
helixsh intent "run nf-core rnaseq on tumor-normal samples using docker"
# Suggest a pipeline and profile for an assay
helixsh profile-suggest --assay wgs --reference GRCh38
# Validate a samplesheet
helixsh samplesheet-validate --file samplesheet.csv --pipeline rnaseq
# Auto-generate a samplesheet from a FASTQ directory
helixsh samplesheet-generate --fastq-dir /data/fastq --pipeline rnaseq --out samplesheet.csv
# Run an nf-core pipeline (dry-run by default, add --execute to run for real)
helixsh run nf-core/rnaseq --runtime docker --input samplesheet.csv --resume
# Estimate cloud cost before submitting
helixsh cost-estimate --cpu 32 --memory-gb 128 --hours 6 --provider aws
# View Nextflow trace summary
helixsh trace-summary --file results/pipeline_info/trace.txt
# Download a reference genome
helixsh ref-download --genome GRCh38 --cache-root /ref --execute
# Submit to Seqera Platform
helixsh tower-submit --pipeline nf-core/rnaseq --profile docker \
--work-dir s3://my-bucket/work --executeGlobal flags available on every command:
helixsh [--strict] [--role auditor|analyst|admin] <command> [options]
--strict— require explicit--executeand--yesfor all side-effecting commands--role— enforce RBAC policy (default:analyst)
Run an nf-core pipeline via Nextflow.
helixsh run nf-core/rnaseq \
--runtime docker \
--input samplesheet.csv \
--resume \
--outdir results/
# With additional nextflow args
helixsh run nf-core/sarek \
--runtime singularity \
--input samplesheet.csv \
-- --genome GRCh38 --tools haplotypecallerOptions:
| Flag | Description |
|---|---|
--runtime |
docker, podman, singularity, apptainer, conda |
--input |
Path to samplesheet CSV |
--resume |
Add -resume to resume from cache |
--outdir |
Output directory |
--offline |
Add -offline flag (use cached containers) |
--execute |
Actually run (default is dry-run preview) |
Check that all required tools are installed and show their versions.
helixsh doctorChecks: nextflow, java, docker, podman, singularity, apptainer, conda, mamba, micromamba, git.
Display planning guidance for building a Nextflow command.
helixsh planRender an explicit POSIX exec sh -c boundary wrapper.
helixsh posix-wrap nextflow run nf-core/rnaseq -profile docker --input s.csv
# Add --execute to actually run it
helixsh posix-wrap --execute nextflow run nf-core/rnaseq -profile docker --input s.csvRun all pre-flight checks in one shot before execution.
helixsh preflight \
--schema nextflow_schema.json \
--params params.json \
--workflow main.nf \
--cache-root .helixsh_cache \
--samplesheet samplesheet.csv \
--config nextflow.config \
--image ghcr.io/nf-core/rnaseq@sha256:abc123Map a natural-language request to a concrete Nextflow plan.
helixsh intent "run nf-core rnaseq on tumor-normal samples using docker"
helixsh intent "analyse chip-seq data with singularity resume if possible"
helixsh intent "run scrna-seq offline with apptainer"
helixsh intent "process amplicon 16S data with conda"Supported pipelines detected by intent: rnaseq, sarek, chipseq, atacseq, methylseq, scrnaseq, ampliseq.
Suggest a pipeline and Nextflow profile arguments based on assay and reference.
helixsh profile-suggest --assay rnaseq --reference GRCh38
helixsh profile-suggest --assay wgs --reference GRCh37 --offline
helixsh profile-suggest --assay chip-seqList all curated nf-core pipelines with descriptions.
helixsh nf-listLaunch a pipeline using nextflow launch or Seqera Platform.
helixsh nf-launch nf-core/rnaseq \
--revision 3.14.0 \
--profile docker \
--outdir results/ \
--params '{"genome":"GRCh38","aligner":"star_salmon"}' \
--executeOptions:
| Flag | Default | Description |
|---|---|---|
--revision |
main |
Pipeline revision / tag |
--profile |
docker |
Nextflow profile |
--outdir |
results/ |
Output directory |
--params |
— | JSON string of pipeline params |
--work-dir |
— | Nextflow work directory |
--execute |
— | Actually launch (default: dry-run) |
Show Seqera Platform (TOWER_ACCESS_TOKEN) authentication status.
helixsh nf-authList known nf-core pipelines with their latest version from the bundled registry.
helixsh pipeline-list
# Force refresh from nf-core API
helixsh pipeline-list --refreshCheck if a pinned pipeline version matches the latest release.
helixsh pipeline-update nf-core/rnaseq --pinned 3.12.0
helixsh pipeline-update nf-core/sarek --pinned 3.4.0 --refreshValidate an nf-core samplesheet CSV for required columns and data integrity.
helixsh samplesheet-validate --file samplesheet.csv --pipeline rnaseq
helixsh samplesheet-validate --file samplesheet.csv --pipeline sarekChecks: required columns present, fastq_1 files exist, paired-end consistency, strandedness values.
Auto-generate an nf-core samplesheet by scanning a FASTQ directory for R1/R2 pairs.
helixsh samplesheet-generate \
--fastq-dir /data/fastq/ \
--pipeline rnaseq \
--strandedness reverse \
--out samplesheet.csvOptions:
| Flag | Default | Description |
|---|---|---|
--fastq-dir |
— | Directory containing FASTQ files |
--pipeline |
rnaseq |
Target pipeline (affects column names) |
--strandedness |
auto |
forward, reverse, unstranded, auto |
--out |
samplesheet.csv |
Output file path |
List all reference genomes in the built-in catalogue.
helixsh ref-listAvailable genomes: GRCh38, GRCh37, GRCm39, GRCm38, TAIR10, R64-1-1, WBcel235.
Download and cache a reference genome (FASTA + GTF) from Ensembl.
# Dry-run (show what would be downloaded)
helixsh ref-download --genome GRCh38 --cache-root /ref/cache
# Actually download
helixsh ref-download --genome GRCh38 --cache-root /ref/cache --execute
helixsh ref-download --genome GRCm39 --cache-root ~/helixsh_refs --executeFiles are stored at <cache-root>/<genome>/ with SHA-256 checksums written alongside each file.
Search the Bioconda channel for a tool.
helixsh conda-search samtools
helixsh conda-search star
helixsh conda-search gatk4Install one or more Bioconda packages (dry-run by default).
# Preview install command
helixsh conda-install samtools bwa-mem2
# Install into a specific environment
helixsh conda-install samtools --env-name myenv
# Actually execute the install
helixsh conda-install samtools bwa-mem2 --executeCreate a complete conda environment with Bioconda tools (dry-run by default).
# Preview environment creation
helixsh conda-env myenv --tools samtools,bwa-mem2,gatk4
# Specify Python version
helixsh conda-env myenv --tools star,salmon --python-version 3.11
# Actually create it
helixsh conda-env myenv --tools samtools,bwa-mem2 --executeAll conda commands use the recommended channel stack:
conda-forge → bioconda with --strict-channel-priority and defaults channel removed.
Parse a Nextflow trace.txt file and show per-process resource usage.
helixsh trace-summary --file results/pipeline_info/trace.txtOutput includes: total tasks, failed count, per-process CPU time, memory, wall time, and I/O.
Estimate cloud cost for a pipeline run across AWS, GCP, and Azure.
# Basic estimate
helixsh cost-estimate --cpu 32 --memory-gb 128 --hours 6
# Provider-specific with instance family
helixsh cost-estimate --cpu 64 --memory-gb 256 --hours 12 \
--provider aws --instance-family compute
# Compare all providers
helixsh cost-estimate --cpu 32 --memory-gb 128 --hours 6 --compare
# Override pricing
helixsh cost-estimate --cpu 32 --memory-gb 128 --hours 6 \
--price-per-cpu-hour 0.04 --price-per-gb-hour 0.005Options:
| Flag | Default | Description |
|---|---|---|
--cpu |
— | Total vCPU count |
--memory-gb |
— | Total memory in GB |
--hours |
— | Estimated wall-clock hours |
--provider |
aws |
aws, gcp, azure |
--instance-family |
general |
general, compute, memory, spot |
--compare |
— | Show all three providers |
Set TOWER_ACCESS_TOKEN (and optionally TOWER_WORKSPACE_ID) before using these commands.
export TOWER_ACCESS_TOKEN=your_token_here
export TOWER_WORKSPACE_ID=12345 # optionalVerify your Seqera Platform token and connectivity.
helixsh tower-authSubmit a pipeline run to Seqera Platform REST API.
# Dry-run (default)
helixsh tower-submit \
--pipeline nf-core/rnaseq \
--revision 3.14.0 \
--profile docker \
--work-dir s3://my-bucket/work
# Actually submit
helixsh tower-submit \
--pipeline nf-core/rnaseq \
--profile docker \
--work-dir s3://my-bucket/work \
--params '{"genome":"GRCh38","input":"s3://my-bucket/samplesheet.csv"}' \
--executeOptions:
| Flag | Default | Description |
|---|---|---|
--pipeline |
— | Pipeline name (e.g. nf-core/rnaseq) |
--revision |
main |
Git revision / tag |
--profile |
docker |
Nextflow profile |
--work-dir |
s3://your-bucket/work |
Work directory |
--params |
— | JSON string of pipeline params |
--compute-env-id |
— | Seqera compute environment ID |
--workspace-id |
— | Override TOWER_WORKSPACE_ID |
--execute |
— | Actually submit |
Get the status of a submitted workflow run.
helixsh tower-status --workflow-id 1abc23def456List available compute environments in your workspace.
helixsh tower-envs
helixsh tower-envs --workspace-id 12345For clusters running Lmod or Environment Modules (no container runtime).
List all tools with known HPC module names.
helixsh envmodules-listIncludes 25+ tools: STAR, HISAT2, BWA, samtools, GATK, Salmon, fastp, MultiQC, and more.
Generate a Nextflow modules.config that loads HPC environment modules per process.
# Print to stdout
helixsh envmodules-wrap star samtools gatk4 salmon multiqc
# Write to file
helixsh envmodules-wrap star samtools gatk4 \
--out modules.config \
--process-prefix nf-core
# Include in a Nextflow run
nextflow run nf-core/rnaseq -profile hpc -c modules.configImport resource declarations (threads, mem_mb, runtime) from a Snakefile into helixsh calibration format.
# Show extracted resources as JSON
helixsh snakemake-import --file Snakefile
# Export as helixsh calibration file
helixsh snakemake-import --file Snakefile \
--export-calibration calibration.jsonValidate a parameters JSON file against an nf-core-style schema.
helixsh validate-schema \
--schema nextflow_schema.json \
--params params.jsonSummarize a samplesheet and nextflow.config defaults.
helixsh context-check \
--samplesheet samplesheet.csv \
--config nextflow.configCheck offline cache readiness (schemas, containers, assets).
helixsh offline-check --cache-root .helixsh_cacheParse Nextflow process blocks from a .nf file and check container policy.
helixsh parse-workflow --file main.nfEstimate CPU and memory requirements for a tool, assay, and sample count.
helixsh resource-estimate --tool star --assay rnaseq --samples 8
helixsh resource-estimate --tool gatk4 --assay wgs --samples 4
# With empirical calibration multipliers
helixsh resource-estimate --tool salmon --assay rnaseq \
--samples 12 --calibration calibration.jsonFit calibration multipliers from an observations JSON file.
helixsh fit-calibration \
--observations observations.json \
--out calibration.jsonThe observations file is a JSON array of {"tool": "star", "cpu_actual": 12, "mem_actual_gb": 38, ...} records collected from real runs.
helixsh uses a proposal-review-execute pattern so AI suggestions never run without human approval.
Generate a Claude AI plan proposal and store it for review.
helixsh claude-plan --prompt "fix schema validation for tumor-normal sarek run"Check whether an MCP gateway capability is permitted.
helixsh mcp-check execute_commands
helixsh mcp-check read_filesStore a proposal for review.
helixsh mcp-propose \
--kind file_patch \
--summary "update nextflow.config memory settings" \
--payload '{"file":"nextflow.config","patch":"..."}'List all pending proposals.
helixsh mcp-proposalsApprove a proposal by ID.
helixsh mcp-approve --id 1Execute an approved proposal.
helixsh mcp-execute --id 1Diagnose a failed Nextflow process by exit code.
helixsh diagnose --process QUANTIFY --exit-code 137
helixsh diagnose --process ALIGN_READS --exit-code 137 --memory-gb 4
helixsh diagnose --process TRIM_READS --exit-code 1Common exit codes interpreted: 137 (OOM), 139 (segfault), 143 (SIGTERM), 127 (command not found), 1 (general failure).
Summarize pipeline cache and resume efficiency.
helixsh cache-report --total 120 --cached 98 --invalidated ALIGN_READS,INDEX_GENOMEExplain the latest command plan.
helixsh explain lastShow current helixsh development roadmap completion status.
helixsh roadmap-statusRecord the start of a pipeline execution in the SQLite provenance database.
helixsh execution-start \
--command "nextflow run nf-core/rnaseq" \
--params '{"genome":"GRCh38","input":"samplesheet.csv"}' \
--pipeline nf-core/rnaseq \
--profile dockerReturns an execution ID used for subsequent execution-finish and audit-show calls.
Record the completion of a pipeline execution.
helixsh execution-finish \
--id exec_20250101_abc123 \
--status success \
--exit-code 0Show the full execution bundle from the provenance DB.
helixsh audit-show --id exec_20250101_abc123Export the JSONL audit log with a reproducible SHA-256 digest.
helixsh audit-export --out audit_export.jsonVerify the integrity and shape of the audit log.
helixsh audit-verifySign the audit log with an HMAC key.
helixsh audit-sign --key-file audit.key --out audit.sigVerify an HMAC-signed audit log.
helixsh audit-verify-signature \
--key-file audit.key \
--signature-file audit.sigGenerate a reproducible execution hash record.
helixsh provenance \
--command "nextflow run nf-core/rnaseq" \
--params '{"genome":"GRCh38","aligner":"star_salmon"}'Check whether a container image has a pinned digest (required for reproducibility).
helixsh image-check --image ghcr.io/nf-core/rnaseq@sha256:abc123...
helixsh image-check --image biocontainers/samtools:1.17Check whether a role is permitted to perform an action.
helixsh rbac-check --role auditor --action run
helixsh rbac-check --role analyst --action conda-install
helixsh rbac-check --role admin --action conda-installEvaluate clinical compliance policy for container images and model confidence.
helixsh compliance-check \
--images ghcr.io/nf-core/rnaseq@sha256:abc123 \
--agreement-score 0.9 \
--confidences 0.85 0.92Run an agent task via the HAPS v1 (Helix Agent Protocol Spec).
helixsh agent-run \
--agent variant_classification \
--task "classify variants from sarek VCF" \
--payload '{"vcf":"variants.vcf"}'Arbitrate between multiple agent responses using a strategy.
helixsh arbitrate \
--responses responses.json \
--strategy majority_voteEvery command is gated by one of three built-in roles. Pass --role <role> globally or set the default via your organisation's deployment.
| Role | Description | Additional permissions vs. previous role |
|---|---|---|
auditor |
Read-only inspection | doctor, explain, plan, validate-schema, parse-workflow, diagnose, cache-report, roadmap-status, rbac-check, report, context-check, offline-check, audit-export, audit-verify, audit-sign, audit-verify-signature, resource-estimate, fit-calibration, image-check, agent-run, arbitrate, compliance-check, mcp-check, mcp-proposals, nf-auth, ref-list, pipeline-list, envmodules-list, tower-auth, tower-status, tower-envs, trace-summary, cost-estimate |
analyst |
+ pipeline operations | All auditor commands + run, intent, profile-suggest, provenance, posix-wrap, preflight, execution-start, execution-finish, audit-show, mcp-propose, mcp-approve, mcp-execute, claude-plan, nf-launch, samplesheet-validate, samplesheet-generate, ref-download, pipeline-update, envmodules-wrap, tower-submit, snakemake-import |
admin |
+ environment management | All analyst commands + conda-install, conda-env, conda-search |
Example:
# Auditor: read-only checks
helixsh --role auditor doctor
helixsh --role auditor trace-summary --file trace.txt
# Analyst: run pipelines
helixsh --role analyst run nf-core/rnaseq --runtime docker --input s.csv
# Admin: install tools
helixsh --role admin conda-install star salmon| Variable | Required | Description |
|---|---|---|
TOWER_ACCESS_TOKEN |
For Tower commands | Seqera Platform personal access token |
TOWER_API_ENDPOINT |
No | Default: https://api.cloud.seqera.io |
TOWER_WORKSPACE_ID |
No | Seqera workspace numeric ID |
HELIXSH_AUDIT_FILE |
No | Path to audit JSONL (default: .helixsh_audit.jsonl) |
HELIXSH_PROVENANCE_DB |
No | Path to SQLite provenance DB (default: .helixsh_provenance.db) |
Strict mode prevents any side-effecting command from running without --execute and --yes:
helixsh --strict run nf-core/rnaseq --runtime docker --input s.csv
# → blocked: requires --execute --yes
helixsh --strict run nf-core/rnaseq --runtime docker --input s.csv --execute --yes
# → runsUse strict mode in clinical genomics, regulated HPC, or production environments.
All commands append a structured JSONL entry to .helixsh_audit.jsonl including:
- Timestamp, role, command, parameters
- Reproducible execution hash (SHA-256)
- Container images used
All pipeline executions are recorded in an SQLite database (.helixsh_provenance.db) with full parameter sets, execution IDs, status, and timing.
┌─────────────────────────────────────┐
│ helixsh CLI │
│ (argparse, zero external deps) │
└──────────────┬──────────────────────┘
│
┌──────────┴──────────┐
▼ ▼
┌─────────────┐ ┌───────────────┐
│ Intent & │ │ Validation & │
│ Context │ │ Schema Layer │
│ intent.py │ │ schema.py │
│ profiles.py │ │ context.py │
│ resources.py│ │ workflow.py │
└──────┬──────┘ └───────┬───────┘
│ │
└────────┬─────────┘
▼
┌───────────────────────┐
│ Execution Layer │
│ nextflow.py │
│ nf_launch.py │
│ bioconda.py │
│ tower.py │
└───────────┬───────────┘
│
┌───────────┴───────────┐
│ Audit & Provenance │
│ provenance_db.py │
│ signing.py │
│ compliance.py │
└───────────────────────┘
- Zero external dependencies — everything uses Python stdlib only (
urllib,csv,json,re,subprocess,sqlite3,hashlib). - Dry-run by default — all destructive operations (downloads, conda installs, Tower submissions, Nextflow runs) require explicit
--execute. - Nextflow is the authority — helixsh plans, validates, and diagnoses; it never replaces Nextflow.
- Brace-aware DSL2 parsing — process block extraction uses a depth-tracking brace parser, not fragile regex.
# Clone and set up
git clone https://github.com/musicofthings/helixsh
cd helixsh
python -m venv .venv
source .venv/bin/activate
pip install -e .
# Run the full test suite
pytest
# Run a specific test file
pytest tests/test_features.py -v
# Run with coverage
pytest --tb=short -qThe test suite covers 188+ tests across all modules with no network calls (all external I/O is mocked or bypassed by dry-run defaults).
- Test suite status: 188 passed (
python -m pytest -q). - Bytecode compile check: pass (
python -m compileall -q src). - UI dependency check via
npm outdatedwas blocked by registry policy (HTTP 403) in this environment. - Rust dependency check via
cargo outdatedrequires installing the externalcargo-outdatedsubcommand first.
Based on upstream package registries and docs looked up on 2026-05-21:
- Nextflow stable docs currently show v25.10.3 (project requires
25.x). @tauri-apps/apilatest visible npm version is 2.8.0 (project range:^2).@tauri-apps/clilatest visible npm version is 2.8.4 (project range:^2).vitelatest visible npm version is 7.1.5 (project range:^6; consider validating upgrade path to v7).
Recommendation: keep
^2Tauri ranges as-is unless you need a specific bugfix, and schedule a controlled Vite 6 → 7 UI upgrade with a quick smoke test.
Build a single-file executable for distribution:
./scripts/package_local.sh
ls -lh dist/helixsh.pyzThe .pyz is a Python zipapp — copy it anywhere, make it executable, and run it directly. No installation required on any machine with Python 3.10+.
chmod +x dist/helixsh.pyz
./dist/helixsh.pyz doctor