scpComponentAnalysis gives PCs with all NaN for some datasets without warning

[edemmott]

I'm looking at some combined single cell datasets, the nature of the datasets is they have high missing data following integration. Performing variance analysis and differential expression via scplainer works fine, but component analysis is currently failing. The nature of the issue is that the scpCompponentAnalysis completes, and does return the caRes object, but that the PC's within the object contain all NaN.

I believe it is down to the high missing data as scpComponentAnalysis works for the individual datasets that make up the integrated dataset, and when some but not all combinations of the experiments are attempted. Currently it seems to start failing at ~81% missing data and higher.

It would be good in introduce a check into scpComponentAnalysis which inspects the PCs prior to returning caRes, and shows an error if they are all NA/NaN.

sessionInfo()
R version 4.4.2 (2024-10-31)
Platform: aarch64-apple-darwin20
Running under: macOS Sonoma 14.5

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/lib/libRlapack.dylib; LAPACK version 3.12.0

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: Europe/London
tzcode source: internal

attached base packages:
[1] stats4 stats graphics grDevices utils datasets methods base

other attached packages:
[1] SCP.replication_0.2.1 SingleCellExperiment_1.28.1 scpdata_1.13.0
[4] ExperimentHub_2.14.0 AnnotationHub_3.14.0 BiocFileCache_2.14.0
[7] dbplyr_2.5.0 EnsDb.Hsapiens.v86_2.99.0 ensembldb_2.30.0
[10] AnnotationFilter_1.30.0 GenomicFeatures_1.58.0 AnnotationDbi_1.68.0
[13] patchwork_1.3.0 dplyr_1.1.4 ggplot2_3.5.2
[16] scp_1.16.0 QFeatures_1.16.0 MultiAssayExperiment_1.32.0
[19] SummarizedExperiment_1.36.0 Biobase_2.66.0 GenomicRanges_1.58.0
[22] GenomeInfoDb_1.42.3 IRanges_2.40.1 S4Vectors_0.44.0
[25] BiocGenerics_0.52.0 MatrixGenerics_1.18.1 matrixStats_1.5.0

loaded via a namespace (and not attached):
[1] RColorBrewer_1.1-3 rstudioapi_0.17.1 jsonlite_2.0.0
[4] magrittr_2.0.3 farver_2.1.2 BiocIO_1.16.0
[7] zlibbioc_1.52.0 vctrs_0.6.5 memoise_2.0.1
[10] Rsamtools_2.22.0 RCurl_1.98-1.17 S4Arrays_1.6.0
[13] BiocBaseUtils_1.8.0 curl_6.2.2 SparseArray_1.6.2
[16] plyr_1.8.9 cachem_1.1.0 GenomicAlignments_1.42.0
[19] igraph_2.1.4 lifecycle_1.0.4 pkgconfig_2.0.3
[22] Matrix_1.7-3 R6_2.6.1 fastmap_1.2.0
[25] GenomeInfoDbData_1.2.13 clue_0.3-66 fdrtool_1.2.18
[28] pkgload_1.4.0 lpsymphony_1.34.0 RSQLite_2.3.11
[31] filelock_1.0.3 labeling_0.4.3 httr_1.4.7
[34] abind_1.4-8 mgcv_1.9-3 compiler_4.4.2
[37] bit64_4.6.0-1 withr_3.0.2 BiocParallel_1.40.2
[40] DBI_1.2.3 MASS_7.3-65 rappdirs_0.3.3
[43] DelayedArray_0.32.0 rjson_0.2.23 OrgMassSpecR_0.5-3
[46] tools_4.4.2 glue_1.8.0 restfulr_0.0.15
[49] nlme_3.1-168 grid_4.4.2 cluster_2.1.8.1
[52] reshape2_1.4.4 generics_0.1.4 sva_3.54.0
[55] gtable_0.3.6 tidyr_1.3.1 metapod_1.14.0
[58] XVector_0.46.0 ggrepel_0.9.6 BiocVersion_3.20.0
[61] pillar_1.10.2 stringr_1.5.1 limma_3.62.2
[64] genefilter_1.88.0 splines_4.4.2 lattice_0.22-7
[67] survival_3.8-3 rtracklayer_1.66.0 bit_4.6.0
[70] annotate_1.84.0 tidyselect_1.2.1 locfit_1.5-9.12
[73] Biostrings_2.74.1 ProtGenerics_1.38.0 edgeR_4.4.2
[76] IHW_1.34.0 statmod_1.5.0 stringi_1.8.7
[79] UCSC.utils_1.2.0 lazyeval_0.2.2 yaml_2.3.10
[82] codetools_0.2-20 nipals_1.0 MsCoreUtils_1.18.0
[85] tibble_3.2.1 BiocManager_1.30.25 cli_3.6.5
[88] xtable_1.8-4 reticulate_1.42.0 dichromat_2.0-0.1
[91] Rcpp_1.0.14 png_0.1-8 XML_3.99-0.18
[94] parallel_4.4.2 blob_1.2.4 bitops_1.0-9
[97] viridisLite_0.4.2 slam_0.1-55 scales_1.4.0
[100] purrr_1.0.4 crayon_1.5.3 rlang_1.1.6
[103] KEGGREST_1.46.0

[cvanderaa]

Hi @edemmott,

Thanks for the report. This is probably associated with nipals which we use to compute the PCA in presence of missing values.

We could indeed add a check and throw an error if the PCs are all NA. But I feel we should first dig deeper into why this happens and eventually provide a more informative error (or better, a fix 😉).

I already had this kind of issue with nipals in the past, which lead me to adapt the default initialization parameters of the function. Would you mind sending me the modelled data and the function call that leads to the issue?

[lgatto]

It would be good to understand what leads to this issue, that is bound to happen again to somebody.

[edemmott]

Ive just sent it to Laurent – can fill you in on the dataset. Best Ed From: cvanderaa ***@***.***> Date: Wednesday, May 21, 2025 at 10:25 AM To: UCLouvain-CBIO/scp ***@***.***> Cc: Emmott, Edward ***@***.***>, Mention ***@***.***> Subject: Re: [UCLouvain-CBIO/scp] scpComponentAnalysis gives PCs with all NaN for some datasets without warning (Issue #91) Caution: This email originated from outside of the University. Do not click links or open attachments unless you recognise the source of this email and know the content is safe. Check sender address, hover over URLs and don't open suspicious email attachments. [Image removed by sender.]cvanderaa left a comment (UCLouvain-CBIO/scp#91)<#91 (comment)> Hi @edemmott<https://github.com/edemmott>, Thanks for the report. This is probably associated with nipals which we use to compute the PCA in presence of missing values. We could indeed add a check and throw an error if the PCs are all NA. But I feel we should dig deeper into why this happens before and eventually provide a more informative error (or better, a fix 😉). I already had this kind of issue with nipals in the past, which lead me to adapt the default initialization parameters of the function. Would you mind sending me the modelled data and the function call that leads to the issue? — Reply to this email directly, view it on GitHub<#91 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AEDTVAEEVNSP3Q2SGI4U7N327RA6JAVCNFSM6AAAAAB5LOUUUWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQOJXGI2DMMRVGQ>. You are receiving this because you were mentioned.Message ID: ***@***.***>