Inquiry regarding significant score variance in zero-shot mutation prediction

[bingjielu]

Dear SaProt Team,

I am a researcher using SaProt for mutational scanning. I’ve encountered an unexpected issue where the model produces significantly different scores and mutation rankings across multiple runs, despite using the exact same input file, environment, and parameters.

Since the top recommended mutation sites change entirely between runs, I am concerned about the reproducibility of my results. I would appreciate your guidance on:

Inference Determinism: Is there any inherent randomness in the structure encoding or the FoldMSA processing that could cause such large score shifts?

Implementation Details: Do you recommend any specific settings (e.g., torch.use_deterministic_algorithms) or environment variables to ensure bit-wise identical results for site recommendation?

Hardware Sensitivity: Could this be a known issue related to specific CUDA kernels or GPU memory handling during inference?

Thank you for developing this great tool. I look forward to your insights on how to stabilize the mutation rankings.

Best regards,
Caroline

[LTEnjoy]

Hi,

Could you give some details about how you used SaProt for mutational scanning such as the running code? In general, SaProt will generate identical result given the same input file.