Hi, I'm in the process of evaluating different perturbation models for use with our in-house dataset, and STATE is one of the contenders. For some reason, I find the paper somewhat convoluted and difficult to interpret, probably mostly due to my lack of familiarity with the methodology, and the immense complexisty associated with in-silcio perturbation predictions at the single-cell level. I'll appreciate it if you could help me understand if STATE is capable of carrying out truly zero shot predictions, i.e., for unseen perturbations as well as cellular contexts.
The abstract in the preprint mentions that "STATE identified strong perturbations in novel cellular contexts where no perturbations were observed during training", so yes, it appears that STATE has performed well for such zero shot predictions. I got interested in STATE as most other models are not capable of this (but are upfront about the limitation, e.g., PerturbNet clearly mentions "PerturbNet cannot predict the effects of unseen perturbations on unseen cell types", and I appreciate the authors for mentioning this) Could you please tell me which section or figure in the preprint I should consult to understand this claim better?
Sections 2.2 and 4.2.1 discuss generalizability to new cellular contexts, but they 30% of the perturbations from the "held-out" dataset are used for training.
Sections 2.3 and 4.2.2 discuss "zero-shot perturbation prediction across contexts," but Section 4.2.2 also notes that "the contexts used for fine-tuning contain a superset of the perturbations found in the held-out context", i.e., the perturbations themselves have been seen during fine-tuning.
So, I couldn't find evaluations in the model for cases with "novel cellular contexts where no perturbations were observed during training". In practical applications, researchers are often interested in such in-silico perturbation methods for getting predictions for hypothetical scenarios for which matched perturbations & cellular contexts are typically unavailable. Could you please tell me if STATE is capable of such true zero-shot predictions, so that I can understand what the requirements are when I'm generating experimental data?
Hi, I'm in the process of evaluating different perturbation models for use with our in-house dataset, and STATE is one of the contenders. For some reason, I find the paper somewhat convoluted and difficult to interpret, probably mostly due to my lack of familiarity with the methodology, and the immense complexisty associated with in-silcio perturbation predictions at the single-cell level. I'll appreciate it if you could help me understand if STATE is capable of carrying out truly zero shot predictions, i.e., for unseen perturbations as well as cellular contexts.
The abstract in the preprint mentions that "STATE identified strong perturbations in novel cellular contexts where no perturbations were observed during training", so yes, it appears that STATE has performed well for such zero shot predictions. I got interested in STATE as most other models are not capable of this (but are upfront about the limitation, e.g., PerturbNet clearly mentions "PerturbNet cannot predict the effects of unseen perturbations on unseen cell types", and I appreciate the authors for mentioning this) Could you please tell me which section or figure in the preprint I should consult to understand this claim better?
Sections 2.2 and 4.2.1 discuss generalizability to new cellular contexts, but they 30% of the perturbations from the "held-out" dataset are used for training.
Sections 2.3 and 4.2.2 discuss "zero-shot perturbation prediction across contexts," but Section 4.2.2 also notes that "the contexts used for fine-tuning contain a superset of the perturbations found in the held-out context", i.e., the perturbations themselves have been seen during fine-tuning.
So, I couldn't find evaluations in the model for cases with "novel cellular contexts where no perturbations were observed during training". In practical applications, researchers are often interested in such in-silico perturbation methods for getting predictions for hypothetical scenarios for which matched perturbations & cellular contexts are typically unavailable. Could you please tell me if STATE is capable of such true zero-shot predictions, so that I can understand what the requirements are when I'm generating experimental data?