ctl-25996_cellpose_github_upload_error.txt
ctl-25996_cellpose_github_upload_output.txt
submission_output_github_upload.txt
Description
I am running a Xenium segmentation pipeline using sopa on an HPC cluster equipped with NVIDIA H100 GPUs. My pipeline explicitly requests the standard nuclei model (model_type='nuclei') with gpu=True.
However, sopa detects cellpose==4.0.8 and, because it fails to pass the specific pretrained_model argument required by the new Cellpose API, it defaults to the 'cpsam' (Cellpose-SAM) model.
Critical Issue: When this fallback occurs, the model execution silently reverts to CPU, completely ignoring the gpu=True flag and the available CUDA devices. This results in execution times that are orders of magnitude slower than expected. I manually terminated the job after 20 minutes because it had processed only ~4% of the patches on the CPU, while the H100 GPU sat idle.
Environment
- sopa: 2.1.11
- cellpose: 4.0.8
- segment-anything: 1.0
- torch: 2.5.1 (CUDA 12.3)
- Hardware: NVIDIA H100 80GB (LSF Scheduler)
Full Conda Environment (`conda list`)
# packages in environment at /project/hipaa_ycheng11lab/software/virtual_environments/sopa_rushil_v3:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main
_openmp_mutex 5.1 1_gnu
aiobotocore 2.26.0 pypi_0 pypi
aiohappyeyeballs 2.6.1 pypi_0 pypi
aiohttp 3.13.3 pypi_0 pypi
aioitertools 0.13.0 pypi_0 pypi
aiosignal 1.4.0 pypi_0 pypi
anndata 0.11.4 pypi_0 pypi
annotated-types 0.7.0 pypi_0 pypi
anyio 4.10.0 py310h06a4308_0
aom 3.9.1 hac33072_0 conda-forge
argon2-cffi 25.1.0 py310h06a4308_0
argon2-cffi-bindings 21.2.0 py310h5eee18b_1
arpack 3.9.1 nompi_hf03ea27_102 conda-forge
array-api-compat 1.13.0 pypi_0 pypi
asciitree 0.3.3 pypi_0 pypi
asttokens 3.0.0 pypi_0 pypi
async-lru 2.0.4 pypi_0 pypi
async-timeout 5.0.1 pypi_0 pypi
attrs 25.1.0 pypi_0 pypi
babel 2.11.0 py310h06a4308_0
beautifulsoup4 4.12.3 py310h06a4308_0
bleach 4.1.0 pyhd3eb1b0_0
bokeh 3.6.2 pypi_0 pypi
botocore 1.36.3 pypi_0 pypi
bottleneck 1.4.2 py310ha9d4c09_0
branca 0.8.1 pypi_0 pypi
brotli 1.0.9 h5eee18b_8
brotli-bin 1.0.9 h5eee18b_8
brotlicffi 1.0.9.2 pypi_0 pypi
bzip2 1.0.8 h5eee18b_6
c-ares 1.19.1 h5eee18b_0
ca-certificates 2024.12.31 h06a4308_0
cached-property 1.5.2 pypi_0 pypi
cachetools 5.5.1 pypi_0 pypi
cellpose 4.0.8 pypi_0 pypi
certifi 2024.12.14 py310h06a4308_0
cffi 1.17.1 py310h1fdaa30_1
charset-normalizer 3.4.1 pypi_0 pypi
click 8.1.8 pypi_0 pypi
cloudpickle 3.1.1 pypi_0 pypi
colorcet 3.1.0 pypi_0 pypi
comm 0.2.1 py310h06a4308_0
contourpy 1.3.1 pypi_0 pypi
cryptography 44.0.0 pypi_0 pypi
cycler 0.12.1 pypi_0 pypi
cytoolz 1.0.1 py310h5eee18b_0
dask 2025.1.0 pypi_0 pypi
dask-core 2025.1.0 pypi_0 pypi
dask-expr 1.1.21 pypi_0 pypi
dask-image 2024.10.0 pypi_0 pypi
datashader 0.17.0 pypi_0 pypi
dav1d 1.2.1 h5eee18b_0
debugpy 1.6.7 py310h6a678d5_0
decorator 5.1.1 pyhd3eb1b0_0
defusedxml 0.7.1 pyhd3eb1b0_0
distributed 2025.1.0 pypi_0 pypi
docstring_parser 0.16 pypi_0 pypi
entrypoints 0.4 py310h06a4308_0
exceptiongroup 1.2.0 py310h06a4308_0
executing 0.8.3 pyhd3eb1b0_0
fasteners 0.19 pypi_0 pypi
fastjsonschema 2.20.0 py310h06a4308_0
fastremap 1.15.0 pypi_0 pypi
filelock 3.13.1 py310h06a4308_0
fill-voids 2.1.1 pypi_0 pypi
fire 0.7.0 pypi_0 pypi
flexcache 0.3 pypi_0 pypi
flexparser 0.4 pypi_0 pypi
flowio 1.4.0 pypi_0 pypi
folium 0.19.4 pypi_0 pypi
fonttools 4.55.3 pypi_0 pypi
freetype 2.12.1 h4a9f257_0
frozenlist 1.5.0 pypi_0 pypi
fsspec 2024.12.0 pypi_0 pypi
gdal 3.6.2 py310h2422eb7_4
geopandas 1.0.1 pypi_0 pypi
geos 3.8.0 he6710b0_0
giflib 5.2.2 h5eee18b_0
gmpy2 2.1.2 py310heeb90bb_0
google-api-core 2.24.1 pypi_0 pypi
google-auth 2.38.0 pypi_0 pypi
googleapis-common-protos 1.66.0 pypi_0 pypi
h11 0.14.0 pypi_0 pypi
h5py 3.12.1 py310h2b02380_0
hdf5 1.12.1 h2b7332f_3
heapdict 1.0.1 pypi_0 pypi
html5lib 1.1 pyhd3eb1b0_0
httpcore 1.0.7 pypi_0 pypi
httpx 0.28.1 pypi_0 pypi
icu 73.1 h6a678d5_0
idna 3.7 py310h06a4308_0
igraph 0.11.8 pypi_0 pypi
imagecodecs 2024.12.30 pypi_0 pypi
imageio 2.36.1 pypi_0 pypi
importlib-metadata 7.0.1 py310h06a4308_0
ipykernel 6.29.5 py310h06a4308_0
ipython 8.30.0 py310h06a4308_0
jbig 2.1 h7f8727e_0
jedi 0.19.2 py310h06a4308_0
jinja2 3.1.4 py310h06a4308_0
jmespath 1.0.1 pypi_0 pypi
joblib 1.4.2 py310h06a4308_0
jpeg 9e h5eee18b_3
json5 0.9.25 py310h06a4308_0
jsonschema 4.23.0 py310h06a4308_0
jsonschema-specifications 2023.7.1 py310h06a4308_0
jupyter-events 0.10.0 py310h06a4308_0
jupyter-lsp 2.2.0 py310h06a4308_0
jupyter_client 8.6.0 py310h06a4308_0
jupyter_core 5.7.2 py310h06a4308_0
jupyter_server 2.14.1 py310h06a4308_0
jupyter_server_terminals 0.4.4 py310h06a4308_1
jupyterlab 4.2.5 py310h06a4308_0
jupyterlab_pygments 0.1.2 py_0
jupyterlab_server 2.27.3 py310h06a4308_0
kiwisolver 1.4.4 py310h6a678d5_0
krb5 1.20.1 h143b758_1
lcms2 2.12 h3be6417_0
ld_impl_linux-64 2.40 h12ee557_0
lerc 3.0 h295c915_0
libabseil 20240116.2 cxx17_h6a678d5_0
libaec 1.1.3 h6a678d5_0
libbrotlicommon 1.0.9 h5eee18b_8
libbrotlidec 1.0.9 h5eee18b_8
libbrotlienc 1.0.9 h5eee18b_8
libcurl 8.9.1 h251f7ec_0
libdeflate 1.17 h5eee18b_1
libedit 3.1.20230828 h5eee18b_0
libev 4.33 h7f8727e_1
libffi 3.4.4 h6a678d5_1
libgcc-ng 11.2.0 h1234567_1
libgdal 3.6.2 h84d2f6d_4
libgfortran-ng 11.2.0 h00389a5_1
libgfortran5 11.2.0 h1234567_1
libgomp 11.2.0 h1234567_1
libjpeg-turbo 2.0.0 h9bf148f_0
libkml 1.3.0 h7ecb851_5
libllvm14 14.0.6 hdb19cb5_4
libnghttp2 1.57.0 h2d74bed_0
libpng 1.6.39 h5eee18b_0
libpoppler 22.12.0 h4f585f7_3
libpq 17.2 hdbd6064_0
libprotobuf 4.25.3 he621ea3_0
libsodium 1.0.18 h7b6447c_0
libspatialindex 1.9.3 h2531618_0
libssh2 1.11.0 h251f7ec_0
libstdcxx-ng 11.2.0 h1234567_1
libtiff 4.5.1 h6a678d5_0
libuuid 1.41.5 h5eee18b_0
libwebp 1.3.2 h11a3e52_0
libwebp-base 1.3.2 h5eee18b_1
libxcb 1.15 h7f8727e_0
libxml2 2.13.5 hfdd30dd_0
libzopfli 1.0.3 he6710b0_0
llvmlite 0.43.0 py310he621ea3_0
lmdb 1.6.2 pypi_0 pypi
locket 1.0.0 pypi_0 pypi
loompy 3.0.6 pypi_0 pypi
lxml 5.3.0 pypi_0 pypi
lz4 4.3.3 pypi_0 pypi
lz4-c 1.9.4 h6a678d5_1
mapclassify 2.8.1 pypi_0 pypi
markdown-it-py 3.0.0 py310h06a4308_0
markupsafe 2.1.3 py310h5eee18b_0
matplotlib 3.10.0 pypi_0 pypi
matplotlib-inline 0.1.6 py310h06a4308_0
matplotlib-scalebar 0.8.1 pypi_0 pypi
mdurl 0.1.0 py310h06a4308_0
mistune 2.0.4 py310h06a4308_0
mkl 2023.1.0 h213fc3f_46344
mkl_fft 1.3.11 py310h5eee18b_0
mkl_random 1.2.8 py310h1128e8f_0
mkl-service 2.4.0 py310h5eee18b_1
mpc 1.1.0 h10f8cd9_1
mpfr 4.0.2 hb69a4c5_1
mpi 1.0 mpich
mpi4py 4.0.2 pypi_0 pypi
mpich 4.1.1 h004f64a_0
mpmath 1.3.0 py310h06a4308_0
msgpack 1.1.0 pypi_0 pypi
multidict 6.1.0 pypi_0 pypi
multipledispatch 1.0.0 pypi_0 pypi
multiscale_spatial_image 1.0.0 pypi_0 pypi
munkres 1.1.4 py_0
narwhals 1.21.0 pypi_0 pypi
natsort 8.4.0 pypi_0 pypi
nbclient 0.8.0 py310h06a4308_0
nbconvert 7.10.0 py310h06a4308_0
nbformat 5.9.2 py310h06a4308_0
ncurses 6.4 h6a678d5_0
nest-asyncio 1.6.0 py310h06a4308_0
networkx 3.4.2 pypi_0 pypi
notebook 7.2.2 py310h06a4308_1
notebook_shim 0.2.3 py310h06a4308_0
numba 0.60.0 py310h2c66911_0
numcodecs 0.13.1 pypi_0 pypi
numexpr 2.10.1 py310h3c60e43_0
numpy 2.0.2 pypi_0 pypi
numpy-base 1.26.4 py310h8a23956_0
numpy-groupies 0.11.2 pypi_0 pypi
ome-types 0.5.2 pypi_0 pypi
ome-zarr 0.9.0 pypi_0 pypi
openjpeg 2.5.2 he7f1fd0_0
openssl 3.0.15 h5eee18b_0
overrides 7.4.0 py310h06a4308_0
packaging 24.2 pypi_0 pypi
pandas 2.2.3 pypi_0 pypi
pandocfilters 1.5.0 pyhd3eb1b0_0
param 2.2.0 pypi_0 pypi
parso 0.8.3 pyhd3eb1b0_0
partd 1.4.2 pypi_0 pypi
patsy 1.0.1 pypi_0 pypi
pcre2 10.42 hebb0a14_1
pexpect 4.8.0 pyhd3eb1b0_3
pillow 11.0.0 pypi_0 pypi
pims 0.7 pypi_0 pypi
pint 0.24.4 pypi_0 pypi
pip 24.2 py310h06a4308_0
pixman 0.40.0 h7f8727e_1
platformdirs 3.10.0 py310h06a4308_0
pooch 1.8.2 pypi_0 pypi
postgresql 17.2 hdbd6064_0
prometheus_client 0.14.1 py310h06a4308_0
prompt-toolkit 3.0.43 py310h06a4308_0
prompt_toolkit 3.0.43 hd3eb1b0_0
propcache 0.2.1 pypi_0 pypi
proto-plus 1.24.0 pypi_0 pypi
protobuf 4.25.3 py310h12ddb61_0
psutil 5.9.0 py310h5eee18b_0
ptyprocess 0.7.0 pyhd3eb1b0_2
pure_eval 0.2.2 pyhd3eb1b0_0
pyarrow 17.0.0 pypi_0 pypi
pyasn1 0.6.1 pypi_0 pypi
pyasn1-modules 0.4.1 pypi_0 pypi
pycparser 2.21 pyhd3eb1b0_0
pyct 0.5.0 pypi_0 pypi
pydantic 2.10.5 pypi_0 pypi
pydantic-core 2.27.2 pypi_0 pypi
pydantic-extra-types 2.10.1 pypi_0 pypi
pygments 2.15.1 py310h06a4308_1
pynndescent 0.5.13 pypi_0 pypi
pyogrio 0.10.0 pypi_0 pypi
pyparsing 3.2.0 pypi_0 pypi
pyproj 3.7.0 pypi_0 pypi
pysocks 1.7.1 py310h06a4308_0
python 3.10.16 he870216_1
python-dateutil 2.9.0.post0 pypi_0 pypi
python-fastjsonschema 2.16.2 py310h06a4308_0
python-json-logger 2.0.7 py310h06a4308_0
python-tzdata 2023.3 pyhd3eb1b0_0
pytorch 2.5.1 py3.10_cuda12.4_cudnn9.1.0_0 pytorch
pytorch-cuda 12.4 hc786d27_7 pytorch
pytorch-mutex 1.0 cuda pytorch
pytz 2024.1 py310h06a4308_0
pyyaml 6.0.2 py310h5eee18b_0
pyzmq 25.1.2 py310h6a678d5_0
rasterio 1.3.10 py310h43501c5_0
readfcs 2.0.0 pypi_0 pypi
readline 8.2 h5eee18b_0
referencing 0.30.2 py310h06a4308_0
requests 2.32.3 py310h06a4308_1
rfc3339-validator 0.1.4 py310h06a4308_0
rfc3986-validator 0.1.1 py310h06a4308_0
rich 13.9.4 pypi_0 pypi
roifile 2024.9.15 pypi_0 pypi
rpds-py 0.10.6 py310h5eee18b_0
rsa 4.9 pypi_0 pypi
ruff 0.9.1 pypi_0 pypi
s3fs 2025.1.0 pypi_0 pypi
scanpy 1.10.4 pypi_0 pypi
scikit-image 0.25.0 pypi_0 pypi
scikit-learn 1.6.1 pypi_0 pypi
scipy 1.15.1 pypi_0 pypi
seaborn 0.13.2 pypi_0 pypi
segment-anything 1.0 pypi_0 pypi
send2trash 1.8.2 py310h06a4308_0
session-info 1.0.0 py310h06a4308_0
setuptools 75.1.0 py310h06a4308_0
shapely 2.0.6 pypi_0 pypi
shellingham 1.5.4 pypi_0 pypi
six 1.16.0 pyhd3eb1b0_1
slicerator 1.1.0 pypi_0 pypi
sniffio 1.3.0 py310h06a4308_0
snappy 1.2.1 h6a678d5_0
sopa 2.1.11 pypi_0 pypi
sortedcontainers 2.4.0 pypi_0 pypi
soupsieve 2.5 py310h06a4308_0
spatial-image 1.1.0 pypi_0 pypi
spatialdata 0.5.0 pypi_0 pypi
spatialdata-io 0.3.0 pypi_0 pypi
spatialdata-plot 0.2.9 pypi_0 pypi
sqlite 3.45.3 h5eee18b_0
stack_data 0.2.0 pyhd3eb1b0_0
statsmodels 0.14.4 pypi_0 pypi
sympy 1.13.3 py310h06a4308_0
tbb 2021.8.0 hdb19cb5_0
tblib 3.0.0 pypi_0 pypi
terminado 0.17.1 py310h06a4308_0
texttable 1.7.0 pypi_0 pypi
threadpoolctl 3.5.0 py310h2f386ee_0
tifffile 2024.12.12 pypi_0 pypi
tiledb 2.16.3 hc287e07_4
tinycss2 1.2.1 py310h06a4308_0
tk 8.6.14 h39e8969_0
toml 0.10.2 pyhd3eb1b0_0
tomli 2.0.1 py310h06a4308_0
toolz 1.0.0 pypi_0 pypi
toposort 1.10 pypi_0 pypi
torch 2.5.1 pypi_0 pypi
torchaudio 2.5.1 py310_cu124 pytorch
torchvision 0.20.1 py310_cu124 pytorch
tornado 6.4.2 py310h5eee18b_0
tqdm 4.67.1 pypi_0 pypi
traitlets 5.14.3 py310h06a4308_0
triton 3.1.0 py310_cuda12.4_0 pytorch
typer 0.15.1 pypi_0 pypi
types-python-dateutil 2.9.0.20241206 pypi_0 pypi
typing-extensions 4.12.2 py310h06a4308_0
typing_extensions 4.12.2 py310h06a4308_0
tzdata 2024b h04d1e81_0
umap-learn 0.5.7 pypi_0 pypi
unicodedata2 15.1.0 pypi_0 pypi
urllib3 2.3.0 pypi_0 pypi
uvloop 0.21.0 pypi_0 pypi
wcwidth 0.2.5 pyhd3eb1b0_0
webencodings 0.5.1 py310h06a4308_1
websocket-client 1.8.0 py310h06a4308_0
wheel 0.44.0 py310h06a4308_0
wrapt 1.17.0 pypi_0 pypi
xarray 2025.1.0 pypi_0 pypi
xarray-dataclass 1.7.0 pypi_0 pypi
xarray-schema 0.0.3 pypi_0 pypi
xarray-spatial 0.4.0 pypi_0 pypi
xerces-c 3.2.4 h6a678d5_1
xorg-libx11 1.8.10 h4f94536_0
xorg-libxau 1.0.11 h7f8727e_0
xorg-libxdmcp 1.1.5 h7f8727e_0
xorg-xorgproto 2024.1 hb9d3cd8_0
xsdata 24.12 pypi_0 pypi
xz 5.4.6 h5eee18b_1
yaml 0.2.5 h7b6447c_0
yarl 1.18.3 pypi_0 pypi
zarr 2.18.2 pypi_0 pypi
zeromq 4.3.5 h6a678d5_0
zfp 1.0.0 h6a678d5_0
zict 3.0.0 pypi_0 pypi
zipp 3.21.0 py310h06a4308_0
zlib 1.2.13 h5eee18b_1
zstd 1.5.6 hc292b87_0
Reproduction Steps
1. Bash Submission Script (Relevant configuration & Debug info):
#!/bin/bash
# ... (Configuration variables omitted) ...
# Environment Setup
source /appl/anaconda3-2024-6/etc/profile.d/conda.sh
conda activate "${VENV_PATH}"
module load cuda/12.3
# FIX: PREVENT THREAD EXPLOSION
export OMP_NUM_THREADS=1
export MKL_NUM_THREADS=1
export OPENBLAS_NUM_THREADS=1
export VECLIB_MAXIMUM_THREADS=1
export NUMEXPR_NUM_THREADS=1
# Debug Info to prove hardware availability
echo "--- Job Setup ---"
echo "Host: $(hostname)"
echo "GPU: $(nvidia-smi -L)"
echo "Python: $(which python)"
echo "-----------------"
# Run Script
python -u "${SCRIPT_PATH}" \
--input "${input_path}" \
--id "${sample_id}" \
--outdir "${OUT_DIR}" \
--tempdir "${TEMP_DIR}" \
--method "cellpose"
2. Python Script (03_segmentation.py):
Note: Includes a Dask config fix we added to rule out memory contention issues.
#!/usr/bin/env python
import dask.config
# Enforce synchronous execution to manage GPU memory usage
dask.config.set(scheduler='synchronous')
import spatialdata_io as sio
import sopa
import logging
# ... (Logging setup) ...
def main():
# ... (Arg parsing) ...
# Intending to use standard N
[ctl-25996_cellpose_github_upload_error.txt](https://github.com/user-attachments/files/25002264/ctl-25996_cellpose_github_upload_error.txt)
[ctl-25996_cellpose_github_upload_output.txt](https://github.com/user-attachments/files/25002266/ctl-25996_cellpose_github_upload_output.txt)
[submission_output_github_upload.txt](https://github.com/user-attachments/files/25002265/submission_output_github_upload.txt)
uclei model on the GPU
sopa.segmentation.cellpose(
sdata,
image_key="morphology_focus",
model_type="nuclei", # <--- This argument is IGNORED by Sopa/Cellpose 4.x
channels=["DAPI", "DAPI"],
diameter=35,
gpu=True
)
if __name__ == "__main__":
main()Logs & Detailed Debugging Output
1. Proof of Hardware Availability:
The bash script output confirms the job is running on a node with 4x NVIDIA H100 GPUs available.
--- Job Setup ---
Host: gpunode06.hpc.local
GPU: GPU 0: NVIDIA H100 80GB HBM3 (UUID: GPU-7a1fc0d0-1ffa-246e-eded-a1b216f273a5)
GPU 1: NVIDIA H100 80GB HBM3 (UUID: GPU-18857b2f-3015-3db8-cc21-65ff89d00a30)
...
Python: /project/.../sopa_rushil_v3/bin/python
2. The Critical Warning (Root Cause):
Sopa explicitly detects the newer Cellpose version and forces a default to cpsam, ignoring our model_type='nuclei' request.
2026-02-02 01:34:25,124 - [INFO] Creating image patches...
[INFO] (sopa.patches.patches) Added 2070 patch(es) to sdata['image_patches']
2026-02-02 01:34:25,244 - [INFO] Running Cellpose (Nuclei)...
[INFO] (sopa.segmentation.methods._cellpose) You use cellpose=4.0.8, which requires a pretrained_model. Defaulting to 'cpsam'.
3. Execution Behavior (CPU Fallback):
Following the cpsam default, the job execution slows to a crawl, indicating a fallback to CPU despite the valid GPU context. The job ran for ~20 minutes before I manually terminated it via KeyboardInterrupt.
[# ] | 3% Completed | 17min 26.2s
[# ] | 3% Completed | 17min 26.3s
...
[# ] | 4% Completed | 17min 28.4s
...
Terminated at Mon Feb 2 01:53:47 2026
Analysis
It appears sopa.segmentation.methods._cellpose has logic to detect cellpose >= 4.0. However, when it detects this version, if a pretrained_model argument is not explicitly provided, it defaults immediately to cpsam. It does not map the existing model_type argument (e.g., "nuclei", "cyto") to the new pretrained_model argument expected by Cellpose 4.x.
Critically, when this default fallback to cpsam occurs, the model fails to utilize the GPU. Even though gpu=True is passed to the function, the fallback execution path runs entirely on the CPU, rendering the pipeline unusable for large-scale datasets on HPC hardware.
ctl-25996_cellpose_github_upload_error.txt
ctl-25996_cellpose_github_upload_output.txt
submission_output_github_upload.txt
Description
I am running a Xenium segmentation pipeline using
sopaon an HPC cluster equipped with NVIDIA H100 GPUs. My pipeline explicitly requests the standard nuclei model (model_type='nuclei') withgpu=True.However,
sopadetectscellpose==4.0.8and, because it fails to pass the specificpretrained_modelargument required by the new Cellpose API, it defaults to the 'cpsam' (Cellpose-SAM) model.Critical Issue: When this fallback occurs, the model execution silently reverts to CPU, completely ignoring the
gpu=Trueflag and the available CUDA devices. This results in execution times that are orders of magnitude slower than expected. I manually terminated the job after 20 minutes because it had processed only ~4% of the patches on the CPU, while the H100 GPU sat idle.Environment
Full Conda Environment (`conda list`)
Reproduction Steps
1. Bash Submission Script (Relevant configuration & Debug info):
2. Python Script (
03_segmentation.py):Note: Includes a Dask config fix we added to rule out memory contention issues.
Logs & Detailed Debugging Output
1. Proof of Hardware Availability:
The bash script output confirms the job is running on a node with 4x NVIDIA H100 GPUs available.
2. The Critical Warning (Root Cause):
Sopa explicitly detects the newer Cellpose version and forces a default to
cpsam, ignoring ourmodel_type='nuclei'request.3. Execution Behavior (CPU Fallback):
Following the
cpsamdefault, the job execution slows to a crawl, indicating a fallback to CPU despite the valid GPU context. The job ran for ~20 minutes before I manually terminated it viaKeyboardInterrupt.Analysis
It appears
sopa.segmentation.methods._cellposehas logic to detectcellpose >= 4.0. However, when it detects this version, if apretrained_modelargument is not explicitly provided, it defaults immediately tocpsam. It does not map the existingmodel_typeargument (e.g., "nuclei", "cyto") to the newpretrained_modelargument expected by Cellpose 4.x.Critically, when this default fallback to
cpsamoccurs, the model fails to utilize the GPU. Even thoughgpu=Trueis passed to the function, the fallback execution path runs entirely on the CPU, rendering the pipeline unusable for large-scale datasets on HPC hardware.