Enhance ACDC job creation logic to handle empty lumis and derive Firs…

[hassan11196]

Derive ACDC MCFakeFile FirstEvent from lumis so pLHE ACDCs number events correctly

Fixes #12502

Status

ready

Description

For MC (MCFakeFile) workflows, all failed-job ACDC documents of a task share a single LFN, so
DataCollectionService.mergeFilesInfo collapses them into one entry that keeps the first_event of an
arbitrary failed job (whichever the CouchDB view returns first) while unioning the lumi lists.
EventBased.createACDCJobs then stepped each job's FirstEvent sequentially from that merged base
(FirstEvent = merged_first_event + k * events_per_job), fully decoupled from the lumis the job was
actually assigned. Since each failed ACDC job re-uploads its (wrong) mask FirstEvent into the next
round's ACDC doc (CouchFileset.add), the error compounds on every ACDC round. For lheInputFiles
workflows this FirstEvent becomes LHESource skipEvents at runtime, so round 2+ seek past the end of
the LHE file and the recovery deadlocks.

Changes in src/python/WMCore/JobSplitting/EventBased.py (createACDCJobs):

• For MCFakeFile jobs, re-derive the event range from the lumis the job processes:
  FirstEvent = (currentLumi - 1) * events_per_lumi + 1, and re-set the event mask with it (job event
  count is unchanged). This makes the mask self-consistent and ACDC-of-ACDC idempotent — a job
  recovering a given set of lumis is numbered identically in every round, so the writeback loop can no
  longer drift.
• Guard against a merged doc whose declared event count exceeds what its lumi list can cover: warn and
  stop creating jobs instead of raising IndexError (this also covers the overrun that fix: JobSubmitter lumi list index out of range #12497 guards
  against).

The real-input ACDC path (distinct real-file LFNs, where first_event is a genuine skip into a real
file and mergeFilesInfo does not cross-contaminate) is untouched.

Added unit tests in test/python/WMCore_t/JobSplitting_t/EventBased_t.py for the previously-untested
createACDCJobs MC path: correct FirstEvent↔lumi mapping from a scrambled merged first_event,
non-contiguous lumis, and the events-exceed-lumis overrun guard.

Validated by replaying the real 198-doc ACDC collection of an affected pLHE workflow
(cmsunified_task_SMP-RunIISummer20UL17pLHEGEN-00061__v1_T_260701_190821_3994) through the patched
splitter: 198 jobs, FirstEvent = (FirstLumi-1)*1000+1 for 198/198, exact lumi coverage, events
conserved, max LastEvent = 992000 within the 1,000,000-event LHE file (the old code would have ended
at 1,194,000, past EOF).

Is it backward compatible (if not, which system it affects?)

YES

Only the event-range numbering of MCFakeFile ACDC job masks changes, and only to make it consistent
with the lumis being recovered (matching how the original EventBased MC splitting numbered them).
Non-ACDC splitting and real-input ACDC splitting are unchanged. No schema, API, or ACDC-document format
change (the acdc_version=2 documents are read as-is).

Related PRs

• fix: JobSubmitter lumi list index out of range #12497 (JobSubmitter lumi list index out of range, CMSPROD-410) — guards a symptom of the same
  merged-doc inconsistency; the overrun guard in this PR covers the same case at the splitting stage.
  Superseded by / overlaps with this change.

External dependencies / deployment changes

None.