How to integrate a new dataset?

[epaaso]

Search for datasets:

Considered:

• zuani & cvejic 9 I-II
• Deng & Liu 2024 I-II 43
• Hu & zhang 2023 III 3 https://genomemedicine.biomedcentral.com/articles/10.1186/s13073-023-01164-9 (bajado)) Ademas tienen 12 post treatment immunotherapy
• trinks & bishoff 2021 9 I-II 3 III 2 (bajado) https://codeocean.com/capsule/8321305/tree/v1
• Zhu & wang 9 I NO NECESITAMOS EARLY STAGE https://doi.org/10.1038%2Fs12276-022-00896-9 (bajado)
• song & zhang 4 I-II
• yang & zhou I-II 7
• Liwen & Feng 2023 I 2 https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE198099

Discarded:

• hanley & thomas I-II 9 III ENRICHED FOR FIBROBLASTS https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE153935 (bajado)
• Lavin & Merad 2017 I 1 MARS-Seq old and maybe very different
• Huang & Yin 2024 I-IV 4 But the article provides the counts but no tumor stage. Sent a mail. https://www.biorxiv.org/content/10.1101/2024.02.18.580893v1.full
• Chan & Rudin 2024 III-IV 2 Porque muchos tienen tratamiento. Pero no tienen los UMI counts. Hay que pedir acceso a traves de un PI. https://www.cell.com/cancer-cell/fulltext/S1535-6108(21)00497-9?_returnURL=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS1535610821004979%3Fshowall%3Dtrue#app-1
• Wu & Tsai 2023 III 4 IV 3 But they are emulsion extractions so no cancer https://doi.org/10.7150%2Fthno.85084
• Long & Shin 2024. I-II 16, III 3 No se puedo por FACS para tener mas representación. https://doi.org/10.1038/s41467-024-52356-9
• Xing & Wang 2021 I-IV 25 Solo por request de un PI https://ngdc.cncb.ac.cn/gsa-human/browse/HRA000154 (nunca se pudo)
• Zhang & Zhong II 3 III 13 https://doi.org/10.1016/j.xcrm.2024.101615 (solo despues de immunoterapia y quimio)
• Chen & Hacohen I-IV 70 DOES NOT ACUTALLY HAVE scRNA and all with therapy... https://doi.org/10.1038/s41590-024-01792-2

[epaaso]

The file nb_annot/deng.ipynb has steps to do it with the scLUCA reference atlas. We checked for correctness with normal adjacent samples and it predicts a negligible amount of tumor cells. Though we have only 10 reduced dims while HCLA has 30.

[epaaso]

We have annotated Zuani dataset by building our own reference atlas from the luca dataset.
We had to integrate the filte to do that that is in gdrive, because we are using google cloud to run the analysis with a gpu.

[epaaso]

We achieved 98 accuracy with the the help of hyper-parameter tuning.
We are missing 7 samples to have 100 for III-IV (late) stage. For I-II we already have ~130