feat: ProForma 2.0 reader/writer and Layer-2 modification converter#1072
Open
trishorts wants to merge 24 commits into
Open
feat: ProForma 2.0 reader/writer and Layer-2 modification converter#1072trishorts wants to merge 24 commits into
trishorts wants to merge 24 commits into
Conversation
Adds the authoritative ProForma test set (123 strings: 26 from the v1.0 paper, 89 valid + 8 counter-examples from the v2.0 spec) as a TSV plus a CorpusLoaderTests fixture that guards its completeness and integrity. This is the foundation the reader/writer will be developed against, test-first. Test data loads via TestContext.CurrentContext.TestDirectory, matching the existing FileReadingTests convention. Phase 0: setup + corpus. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds ProFormaReader.Read and ProFormaWriter.Write — thin mzLib entry points that delegate Layer-1 parse/write to the TopDownProteomics reference parser, so callers depend on mzLib types rather than the SDK directly. Includes a shared ProFormaTestCorpus loader and base-level round-trip tests asserting canonical idempotence of write(parse(s)). Surfaces the first SDK coverage gap: ProFormaWriter throws on a range bearing multiple modifications (spec 4.5); parsing succeeds, only writing fails. The case is pinned by KnownGap_MultiModRange_ParsesButSdkWriterThrows so a future SDK fix is detected. Phase 1: Readers/ProForma facade. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Widens the round-trip sweep to every valid corpus example. The wrapped SDK round-trips all single-proteoform-term forms (99/110): single-chain crosslinks, disulfides, glycans, formulas/isotopes, ranges, localization groups, scored sites, unknown position, global mods, termini, and the common v1.0 syntaxes. Three gaps are pinned with dedicated tests so each is tracked: - v1.0 default-source prefix [SOURCE]+SEQ — intentionally unsupported (v2.0 removed it). - charge /z, chimeric +, multi-chain // (13 rows) — valid v2.0 features that live above a single term; planned facade work. - writer throws on a multi-mod range (gap #1, pinned earlier). 106 ProForma tests green. Phase 2-3: Layer-1 coverage map. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds ProFormaConverter mapping between a ProFormaTerm and mzLib's
(base sequence + AllModsOneIsNterminus) representation that
IBioPolymerWithSetMods consumes, mirroring the index convention and lookup
idiom of GetModificationDictionaryFromFullSequence/DetermineFullSequence
(N-terminus = 1, residue r = r + 2, C-terminus = length + 2).
This first slice resolves per-residue Name descriptors via the
"{name} on {residue}" IdWithMotif key and round-trips them back through the
writer. A guard throws MzLibException on features outside the Layer-2 subset
(tag groups/crosslinks, labile, unlocalized, global mods, ranges, sequence
ambiguity) and on modifications absent from the supplied database, so callers
never silently drop data.
Phase 10: Layer-2 converter (slice 1).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends ProFormaConverter to map Identifier descriptors (e.g. [UNIMOD:35], [MOD:00719]) to mzLib Modifications via a DatabaseReference accession index, and to emit accession descriptors on the inverse path when a modification carries a recognized ontology reference (else a Name). Resolution is motif-aware: a single accession such as UNIMOD:21 maps to several modifications differing only by target residue, so the residue at the tagged position disambiguates. Alias maps bridge mzLib's DatabaseReference keys and ProForma prefixes (Unimod<->UNIMOD, PSI-MOD<->MOD, RESID<->RESID). Phase 10: Layer-2 converter (slice 2, accessions). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Makes ProFormaConverter resolution terminus-aware: interior residues match on
target motif, while N-/C-terminal descriptors match on LocationRestriction
(N-terminal./Peptide N-terminal. and the C-terminal equivalents). The Name
path tries the usual "{name} on X" terminal id first, then any same-named
terminus-compatible modification.
Round-trips terminal mods by both name ([Acetyl]-...-[Amidation]) and
accession ([UNIMOD:214]-...), placing them at the AllModsOneIsNterminus
terminal keys (1 and length + 2).
Phase 10: Layer-2 converter (slice 3, terminal mods).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a fixture that loads the bundled Unimod database (~2700 modifications) via Loaders.LoadUnimod and exercises ProFormaConverter on real data rather than hand-built mods: resolves names (Oxidation, Phospho) and accessions (UNIMOD:35, UNIMOD:21), selects the correct motif amid real accession ambiguity (UNIMOD:21 maps to Phospho on S/T/Y), and round-trips. Confirms the per-residue/terminal, name/accession converter behaves on the actual modification dictionary MetaMorpheus loads, de-risking integration. Phase 10: Layer-2 converter (slice 4, real-Unimod validation). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds ToProFormaString/ToProFormaTerm extension methods on IBioPolymerWithSetMods so a scored peptide or proteoform converts to its ProForma 2.0 string in one call. This is the surface MetaMorpheus top-down output will consume; it composes the existing converter and writer. End-to-end tests build a PeptideWithSetModifications from its native mzLib FullSequence and assert the ProForma output for both a name modification ([Oxidation]) and an accession-bearing one ([UNIMOD:35]). Phase 10: Layer-2 converter (slice 5, peptide bridge). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the "ProForma" column constant to SpectrumMatchFromTsvHeader so search output can carry the ProForma 2.0 string alongside Full Sequence. The value is populated by MetaMorpheus's PsmTsvWriter via IBioPolymerWithSetMods .ToProFormaString(); the column appears on the standard PSM/proteoform path (crosslink/glyco use their own headers). Phase 10: ProForma output column. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds SpectrumMatchFromTsv.ProForma, registers the column in ParseHeader, and reads it via GetOptionalValue so it round-trips with the writer. The column is optional (IndexOf returns -1 for pre-ProForma files, treated as absent), so older result files still read unchanged. In the ambiguity-disambiguation constructor the parent value is carried as-is rather than split on '|', because ProForma uses '|' as an internal descriptor separator and cannot be split per candidate. Phase 10: ProForma reader round-trip. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
Author
|
Companion PR (consumes this): smith-chem-wisc/MetaMorpheus#2663 |
Add 7 NUnit tests for the fail-loud paths in ProFormaConverter that had no patch coverage: - the five EnsureLayer2Supported guards (labile, unlocalized, global, position-range, sequence-ambiguity modifications) - the unresolved-terminal-mod throw (N- and C-terminus) - the terminal name-resolution fallback (mod stored with a residue motif rather than the wildcard X) Lifts ProFormaConverter.cs patch coverage from ~80% well above 90%. The two lines left uncovered (SelectCandidate count==1 fallback, IsTerminusCompatible default arm) are provably unreachable from the public API. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…der-writer # Conflicts: # mzLib/Readers/Readers.csproj
Fixes from the PR smith-chem-wisc#1072 review of the ProForma Layer-2 converter: - Match the motif's modified residue instead of the whole motif string, so context-bearing motifs (e.g. the N-glyc sequon Nxs) resolve instead of throwing MzLibException for an otherwise-known modification. - Cache the accession index per allModsKnown instance (ConditionalWeakTable) rather than rebuilding it over the full mod database on every term, removing O(mods x references) work per converted PSM in batch consumption. - Fail loud (MzLibException) when two tags target the same residue index instead of silently overwriting one, per the converter's stated contract. - Throw a descriptive MzLibException, not a raw IndexOutOfRangeException, for an out-of-range tag index. - Emit ontology accessions in a fixed UNIMOD > MOD > RESID order so ToProFormaString output is deterministic when a mod has multiple references. - Guard null arguments in ToModificationDictionary/ToProFormaTerm. Addresses 6 findings (2 Major, 4 Minor) from the PR smith-chem-wisc#1072 review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fixes from the PR smith-chem-wisc#1072 review of the ProForma test suite: - Assert the exact v2-spec valid corpus count (EqualTo(89)) instead of a weak GreaterThan(70), matching the precise-count sibling tests so a deleted or mistyped row is caught. - Assert the crosslink guard's message ("tag group") so the test confirms the tag-group path fired rather than passing on an incidental missing-mod throw against the empty dictionary. - Use a unique per-test temp filename (Test.ID) instead of a fixed name in the shared test directory, removing a parallelization/leftover hazard. Addresses 3 Minor findings from the PR smith-chem-wisc#1072 review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-chain forms Add a layer above the single-term TopDownProteomics parser so ProForma 2.0 constructs that live above one proteoform term round-trip losslessly: chimeric peptidoforms (+), charge states (/z[adducts]), and inter-chain crosslink / branch chains (//). - ProFormaMultiTerm / ProFormaPeptidoform: model a chimeric set of peptidoforms, each one or more // chains plus an optional /charge[adducts]. - ProFormaMultiTermReader / ProFormaMultiTermWriter: split on the top-level +, /, and // operators (bracket-depth-aware), delegating each chain to the wrapped single-term ProFormaReader/ProFormaWriter and reassembling on write. Covers the 13 corpus forms previously pinned as "above a single term" (charge x7, chimeric x1, inter-chain crosslink x3, branch x2). 17 new multi-term round-trip tests pass; the single-term parser is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JNnpzBNwfnTbZxtZdtAzLG
The 0.0.299 ProFormaWriter now serializes a range bearing multiple modifications (spec 4.5), closing the only known SDK writer gap. The KnownGap_MultiModRange guard test fired as designed: drop it, remove v2-4.5-01 from the writer-gap exclusion set, and let that example flow through the standard round-trip suite. 144 ProForma tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Treat the wildcard "X" motif as matching any residue, so an interior mod stored against an X-motif Modification resolves on read instead of throwing (previously broke the round-trip for that whole class of mods). - Accept "Anywhere." LocationRestriction at a terminus so a terminal- position mod that ToProFormaTerm writes is also resolvable on read. - Reject IonAdducts without a Charge in the ProFormaPeptidoform ctor, since the writer can only emit adducts inside /z[...] and would otherwise drop them silently. Addresses 3 findings (1 Major, 2 Minor) from the PR smith-chem-wisc#1072 review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Assert exact input preservation for an inter-chain crosslink form and a branch // form, plus chain-sequence content for the chimeric and multi-chain splits; idempotence alone would miss a consistent drop/reorder of chains or crosslink labels. - ProFormaTestCorpus.Load() throws InvalidDataException on a malformed row instead of calling NUnit Assert, so a bad corpus fails cleanly at execution rather than as a TestCaseSource discovery error. Addresses 3 findings from the PR smith-chem-wisc#1072 review (fix_007/008/009 were already fixed in-branch; fix_010 informational, skipped). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
Manual unit-test reviewAll 36 unit test(s) added by this PR were reviewed individually by the author (not only by automated/AI-generated checks): 33 approved as written, 3 edited, 0 flagged. This satisfies the author's standing AI-use policy that AI-assisted tests receive human review before relying on them. Recorded 2026-06-30 15:14. |
Pins the fix that lets IsTerminusCompatible accept an "Anywhere." LocationRestriction at a terminus: ToProFormaTerm writes any position-1 mod as an N-terminal descriptor regardless of restriction, so the read path must resolve it. Fails before that change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bring PR smith-chem-wisc#1072 current with master (resolves DIRTY/CONFLICTING state). OpenMcdf 3.1.4 from master; TopDownProteomics stays 0.0.299.
…iter Bring PR smith-chem-wisc#1072 current with the true upstream base (the fork's origin/master was stale). Keep Newtonsoft.Json 13.0.4 from upstream; TopDownProteomics stays 0.0.299 (this PR's bump; upstream is still 0.0.297).
added 2 commits
July 13, 2026 23:49
mzLib gained a universal sequence-conversion framework in smith-chem-wisc#1035 (Omics/SequenceConversion: Parser -> CanonicalSequence -> Serializer, with pluggable IModificationLookup). ProForma was built alongside it rather than inside it. This plugs Layer 2 into that framework instead of standing beside it. Adds, in Readers (which is where the TopDownProteomics SDK is referenced; Omics does not reference it): - ProFormaSequenceFormatSchema - describes the format to the service. - ProFormaSequenceParser - ProForma string -> CanonicalSequence. - ProFormaSequenceSerializer - CanonicalSequence -> ProForma string. - ProFormaSequenceConversion.RegisterWith(service) - explicit, idempotent registration. Deliberately NOT a module initializer: touching SequenceConversionService.Default constructs GlobalModificationLookup and loads the modification databases, and that cost belongs to callers who convert sequences, not to every consumer of Readers. Both delegate their string handling to Layer 1 (ProFormaReader/ProFormaWriter), and the serializer emits descriptors through the existing ProFormaConverter.BuildDescriptor (now internal), so ProForma emission has one implementation, not two. The payoff is interop for free: mzLib -> ProForma now converts a tool-specific name to a community accession ("PEPC[Common Fixed:Carbamidomethyl on C]TIDE" -> "PEPC[UNIMOD:4]TIDE") through Nic's mzLib parser and our serializer, with no bespoke converter. It also gives the framework a parser for the accession notation its UnimodSequenceSerializer already emits. The flat-subset boundary is explicit and fails loud. CanonicalSequence is a flat model - one mod per position, no groups, ranges, or delocalization - so tag groups, position ranges, unlocalized, labile, and global mods, and sequence ambiguity are reported as IncompatibleModifications rather than silently dropped. Layer 1 still round-trips all of them losslessly; that is why Layer 1 keeps its own richer AST and is not routed through the canonical IR. Layer 2's resolution logic is intentionally left in place rather than replaced by ModificationLookupBase: ProFormaConverter resolves accessions through an exact index and understands context-bearing motifs (the N-glycosylation sequon "Nxs"), whereas ModificationLookupBase.FilterByUnimodId matches accession ids by substring Contains and FilterByMotif compares the whole motif string. Those look like precision gaps worth raising upstream, not behavior to adopt. Tests: 16 new (ProFormaSequenceConversionTests) covering registration, accession and terminal and mass tags, mzLib->ProForma and ProForma round trips, the five unrepresentable constructs, handling modes, and CanParse not stealing mzLib's own syntax during auto-detection. Full mzLib suite green: 5177 passed / 0 failed (baseline 5161; +16 = exactly the new tests).
Cover the changed lines that Codecov's patch check gates on. The SequenceConversion classes added in fcf6438 were exercised only along their happy paths - the serializer sat at 38% and RegisterWithDefault was never called at all. Adds 20 NUnit tests: - Serializer: terminal-mod round trip, the UNIMOD-id / name / mass / null descriptor fallbacks, fail-loud and RemoveIncompatibleElements handling of an undescribable mod, empty base sequence, and WithLookup. - Parser: null/empty input, CanParse on a malformed ProForma lookalike, two mods on one residue, sequence ambiguity, non-UNIMOD ontologies. - Registration: RegisterWithDefault idempotence; format schema. - Converter: null-argument guards, out-of-bounds tag index, two tags on one residue, the wildcard "X" motif, and the N-glycosylation sequon motif "Nxs" - the context-bearing-motif resolution that keeps ProFormaConverter stricter than ModificationLookupBase. Patch coverage 74.75% -> 91.83% (371/404 changed lines), target 90%. Full suite green: 5197 passed / 0 failed. Left uncovered by design: the inner check of the double-checked lock in RegisterWithDefault (unreachable single-threaded), and partial branches in MotifTargetResidue/ResolveName reachable only through modification shapes that do not occur in the real databases.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This PR adds HUPO-PSI ProForma 2.0 reading and writing to mzLib. Layer 1 (string ↔
ProFormaTerm) is delegated to the TopDownProteomics SDK — already a production dependency of mzLib — through a thinReaders/ProFormafacade (ProFormaReader,ProFormaWriter), so callers depend on stable mzLib entry points rather than the SDK directly. Layer 2 (ProFormaTerm↔IBioPolymerWithSetMods) is new work:ProFormaConvertermaps per-residue and N-/C-terminal modifications to and from mzLibModificationobjects, mirroring the existingGetModificationDictionaryFromFullSequence/DetermineFullSequenceidiom (N-term = 1, residue = index + 2, C-term = length + 2). Resolution is motif-aware for interior residues andLocationRestriction-aware at termini, handling both names ([Oxidation]) and ontology accessions ([UNIMOD:35]). A one-callIBioPolymerWithSetMods.ToProFormaString()bridge plus aProFormapsmtsv column (written and re-read) complete the round-trip. Unsupported constructs fail loud.What's included
Readers/ProForma/—ProFormaReader.Read,ProFormaWriter.Write(wrap the SDK parser/writer);ProFormaConverter(Layer-2, both directions);ProFormaExtensions.ToProFormaString()/ToProFormaTerm().SpectrumMatchFromTsvHeader.ProForma;SpectrumMatchFromTsv.ProFormaparsed viaParseHeader+GetOptionalValue(optional — pre-ProForma files still read).TopDownProteomicsPackageReference added toReaders.csproj(already transitive viaUsefulProteomicsDatabases).Design notes for reviewers
IResultFile: ProForma is an in-memory string codec, not a file-on-disk with an extension. Documented, not an oversight.MzLibException): crosslinks/tag groups, branches, labile mods, ambiguity groups, ranges, sequence ambiguity, global isotope mods.[SOURCE]+default-source prefix not parsed (removed in v2.0; intentionally unsupported); (3) charge/z, chimeric+, multi-chain//need a multi-term facade (planned).Testing
120 ProForma tests (corpus of 123 manuscript examples; Layer-1 round-trip idempotence; Layer-2 round-trips incl. real bundled Unimod validation). Full
FileReadingTestssuite green (716). Baseline unchanged.🤖 Generated with Claude Code