v3.260709

This main purpose is to validate HTCondor 25.0.11, which expected to fix the refreshGSIProxy renewal failure in the HTC v2 interface w/o workaround flag.

What's Changed

• Make runner more robust by @belforte in #9339
• remove workaround flag, DELEGATE_JOB_GSI_CREDENTIALS. by @nausikt in #9344

Full Changelog: v3.260702...v3.260709

v3.260702

add a fix for MigrationAccounter.
Also gives a chance to try again pipeline after @nausikt fixed credentials and DNS

What's Changed

• lock migration account file when updating by @belforte in #9338

Full Changelog: v3.260701...v3.260702

Validation

• pipeline https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/15179215 - OK

Deployment

• July 3, 2026
  • on crab-prod-tw01 (production) as Publisher_schedd and Publisher_rucio (same code was already running as Publihser_schedd via ./start -g from current master
  • on crab-prod-tw02 as canary TW

v3.260701

Introduce FailedMigrationAccounter for Publisher and correctly count processed files when user provides an input file list other than LFN's.

What's Changed

• introduce migrationAccounter. Fix #9329 by @belforte in #9330
• Add migration accounter to rucio publisher 9331 by @belforte in #9334
• fix #9336 by @belforte in #9337
• fix #9332 by @belforte in #9335

Full Changelog: v3.260622...v3.260701

Validation

• pipeline https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/15169507 failed due to expired S3 keys
• anyhow we want to add a file lock. Will tag new release once pipeline is fixed

Deployment

• skipped

v3.260622

Main purpose of this release is to ensure that core dumps are discarded inside job wrapper, since we have at least one site where somehow telling condor not to retrieve them was not sufficient (#9324)

What's Changed

• enforce useGzip everywhere we instantiate DbsApi. by @nausikt in #9314
• remove core dumps and better msgs. Fix #9325 by @belforte in #9327

Full Changelog: v3.260616...v3.260622

Validation

• pipeline https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/15097482 - OK
  • one CCV job had an errore during ASO step. Just random EOS permission error (there is enough quota)

Deployment

• new TW in Canary - new Publishers in production : 23 June 2026
• new TW in production : tbd

v3.260616

This release incorporates the refactoring of the resubmission/retry logic by @aspiringmind-code to allow more flexibility.

What's Changed

• Remove unused panda(?) keyword since 2021, last use 2013 by @oljemark in #9322
• Use RRN and Exitcodes to overhaul resubmission and retry policy by @aspiringmind-code in #9319
• change to 3 by @aspiringmind-code in #9323

Full Changelog: v3.260606...v3.260616

Validation

• pipeline https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/15041226 : OK

NOTE: Same content as tag v3.260615 (in gitlab only) validated already in https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/15032685

v3.260606

The main purpose of this tag is to release mitigations for heavy/slow queries in response to the DB admin review.

What's Changed

• Fix heavy query by @nausikt in #9321

Full Changelog: v3.260514...v3.260606

v3.260514

The main purpose of this release is to try the ORA-00001 failed block-publication retry mechanism in production.

What's Changed

• Stop mounting CVMFS inside containers. Completes #9235 by @belforte in #9308
• update DebugFailedBlockPublication.py for #9309 by @belforte in #9310
• save blocks which fail to publish also in JSON format by @belforte in #9312
• Add DbsApi wrapper to catch and retry ORA-00001, failed block publication. by @nausikt in #9315

Full Changelog: v3.260428...v3.260514

v3.260428

Most urgent is fix for Publisher (limit at 100K lumis per block).
It is also good to deploy the fix for $FRONTIER_ID ( #9306 )

What's Changed

• add lsof and strace to containers by @belforte in #9301
• Revert add jitter delays to Publisher DBS API calls to prevent throttling. by @nausikt in #9304
• maxLumisPerBlock = 100k. Fix #9302 by @belforte in #9305
• CRAB_Reqname, not ReqName. Anyhow use grep -i. Fix #9306 by @belforte in #9307

Full Changelog: v3.260418...v3.260428

Validation

• https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/14639785 - OK

v3.260418

Hotfix for the current chaos: O(112k) publication files are stuck in Acquire, forming a baseline of terminally failed migrations. [#9299]

What's Changed

• Fix a few logging typos in gitlab cicd by @oljemark in #9298
• Mitigate terminally failed migration baseline by adding jitter delays between our Publisher DBS API calls. by @nausikt in #9300

Full Changelog: v3.260415...v3.260418

v3.260415

Main reason of the tag is to deploy the possibility for sites to require ASO via Rucio
as requested by Italian T2's

What's Changed

• Use python3 for wmc httpd by @belforte in #9291
• Add manual pipeline by @belforte in #9293
• deprecate (sidecar) crab-k8s-overlays, switching CI to mono-repo approach by @nausikt in #9279
• Allow sites to force ASO via Rucio. Fix #9294 by @belforte in #9297

Full Changelog: v3.260407...v3.260415

Validation

• validation pipeline https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/14533059 OK