Major Refactoring of Current Koina Models

[pcruzparri]

SUMMARY:

• Current Issues
  • Current implementation of these models may experience timing out exceptions for large numbers of inputs (>200k), which can cause problems when predicting, for example, all tryptic peptides from human protein db (even when filtered to unique peptides only).
  • The current use feels a little funky using the entire model as disposable. It was an ok approach but doesn't represent how models are typically used ( take some inputs -> model.predict(inputs) ).
  • General Solutions
    • Throttling. Take all every batch and send the batches in chunks separated by some delay. This ensures that the responses we get back are never being waited on for too long to cause timeout errors.
    • Lots of refactoring. One key change is the methods that call model.Predict() does not need to change to be asynchronous. The asynchronous implementation is fully awaited only by the model, and the final results are returned.
  • Note
    • The spectral library generation method of FragmentIntensityModel-derived classes may be removed later. It might be just an util method, but Shortreed is currently handling a better approach (see Intenenal ions squared #1022).

Copilot Summary:

🔧 Key Changes

1. Architecture Refactoring
   • Introduced Generic Type Parameters: All abstract model classes now use generic type parameters <TModelInput, TModelOutput> for type-safe prediction workflows
   • New Interface: Created IPredictor<TModelInput, TModelOutput> interface defining the standard prediction contract
   • Input/Output Records: Added strongly-typed input records for each model type:
   • DetectabilityPredictionInput
   • FragmentIntensityPredictionInput
   • RetentionTimePredictionInput
   • Unified API: Standardized prediction method signature: List Predict(List inputs) across all models
2. Enhanced Modification Handling
   • New Enum: Added IncompatibleModHandlingMode with 4 strategies:
   • RemoveIncompatibleMods: Strip unsupported modifications and predict with remaining mods
   • UsePrimarySequence: Ignore all modifications and use only base sequence
   • ThrowException(): Fail fast if incompatible modifications present
   • ReturnNull: Skip prediction for peptides with incompatible modifications
   • Configurable per Model: Each model can specify its preferred modification handling strategy via ModHandlingMode property
   • Warning System: Predictions now include WarningException? field to communicate modification handling to users
3. Improved Batching & Performance
   • Dynamic Timeout Calculation: Timeouts now scale based on batch count using benchmarked timing data (BenchmarkedTimeForOneMaxBatchSizeInMilliseconds)
   • Configurable Batching: Added MaxNumberOfBatchesPerRequest and ThrottlingDelayInMilliseconds properties to control request parallelism
   • Better Throttling: Implements chunked batch processing with configurable delays to prevent server overload
   • Edge Case Fix: Fixed bug where client timeout would be set to 0 when no valid inputs exist (commit 4244575)
4. Input Validation Improvements
   • ValidInputsMask: Added boolean mask to track which inputs passed validation
   • Pre-Validation: Input cleaning and validation now happens upfront before API calls
   • Better Alignment: Predictions are correctly aligned back to original input order, with null predictions for invalid inputs
   • Renamed Property: CanonicalAminoAcidPattern → AllowedAminoAcidPattern for clarity
5. Simplified Concrete Implementations
   • Concrete model classes (e.g., Prosit2020IntensityHCD, Prosit2019iRT, PFly2024FineTuned) significantly simplified
   • Removed constructor complexity and moved validation logic to base classes
   • Reduced code duplication across model implementations

📊 Files Changed
Core Abstract Classes:
• KoinaModelBase.cs (+72/-39)
• DetectabilityModel.cs (+153/-55)
• FragmentIntensityModel.cs (+241/-74)
• RetentionTimeModel.cs (+158/-45)
New Files:
• IPredictor.cs (new interface, +22 lines)
• IncompatibleModHandlingMode.cs (new enum, +13 lines)
Concrete Model Implementations:
• PFly2024FineTuned.cs (+30/-71)
• Prosit2020IntensityHCD.cs (+70/-154)
• Prosit2019iRT.cs (+85/-49)
• Prosit2020iRTTMT.cs (+71/-44)
Test Files: (All significantly refactored to test new functionality)
• PFly2024FineTunedTests.cs (+289/-191)
• Prosit2020IntensityHCDTests.cs (+292/-301)
• Prosit2019iRTTests.cs (+246/-249)
• Prosit2020iRTTMTTests.cs (+313/-555)

✅ Testing
• All existing tests updated to work with new API
• Tests now verify modification handling behavior for different modes
• Boundary condition tests enhanced
• Benchmark tests completed on 4M+ peptides/predictions for all four model types

🐛 Bug Fixes
• Fixed client timeout being set to 0 when no valid inputs exist
• Improved handling of edge cases with empty or all-invalid input lists
• Better error messaging and warning propagation

💡 Benefits

1. More Flexible: Users can now choose how to handle incompatible modifications instead of failing silently
2. Better Performance: Dynamic timeout calculation and optimized batching prevent timeouts on large datasets
3. Type Safety: Generic type parameters provide compile-time type checking
4. Maintainability: Reduced code duplication and clearer separation of concerns
5. User Experience: Warning system provides transparency about what happened during prediction

🔄 Breaking Changes
• Constructor signatures changed for all model classes
• PeptideSequences property removed in favor of input/output parameters
• Prediction method changed from RunInferenceAsync() to Predict(List)
• Some properties renamed (e.g., CanonicalAminoAcidPattern → AllowedAminoAcidPattern)

[codecov]

Codecov Report

❌ Patch coverage is 92.12454% with 43 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.43%. Comparing base (9ab5c1b) to head (e0c32ec).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
...ts/Koina/AbstractClasses/FragmentIntensityModel.cs	90.83%	12 Missing and 11 partials ⚠️
...lients/Koina/AbstractClasses/RetentionTimeModel.cs	91.20%	4 Missing and 4 partials ⚠️
...lients/Koina/AbstractClasses/DetectabilityModel.cs	91.66%	4 Missing and 3 partials ⚠️
.../FragmentIntensityModels/Prosit2020IntensityHCD.cs	90.74%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1029      +/-   ##
==========================================
- Coverage   81.46%   81.43%   -0.03%     
==========================================
  Files         315      315              
  Lines       42765    42993     +228     
  Branches     4814     4846      +32     
==========================================
+ Hits        34839    35012     +173     
- Misses       7051     7089      +38     
- Partials      875      892      +17     

Files with missing lines	Coverage Δ
...ionClients/Koina/AbstractClasses/KoinaModelBase.cs	89.70% <100.00%> (-10.30%)	⬇️
...portedModels/FlyabilityModels/PFly2024FineTuned.cs	100.00% <100.00%> (ø)
...pportedModels/RetentionTimeModels/Prosit2019iRT.cs	100.00% <100.00%> (ø)
...rtedModels/RetentionTimeModels/Prosit2020iRTTMT.cs	100.00% <100.00%> (ø)
.../FragmentIntensityModels/Prosit2020IntensityHCD.cs	91.89% <90.74%> (-2.18%)	⬇️
...lients/Koina/AbstractClasses/DetectabilityModel.cs	90.00% <91.66%> (-2.86%)	⬇️
...lients/Koina/AbstractClasses/RetentionTimeModel.cs	86.79% <91.20%> (+3.00%)	⬆️
...ts/Koina/AbstractClasses/FragmentIntensityModel.cs	86.87% <90.83%> (-7.25%)	⬇️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.