forked from duckdb/duckdb-r
-
Notifications
You must be signed in to change notification settings - Fork 0
976 lines (878 loc) · 39.6 KB
/
Copy pathrevdep4.yaml
File metadata and controls
976 lines (878 loc) · 39.6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
# Containerized reverse-dependency check, queue engine:
# plan + base image (parallel) -> build -> universe image -> test (matrix) -> collect.
#
# revdep2 ran the two halves of a package -- the CRAN release and the dev
# build of this package -- as two simultaneous `R CMD check` processes on one
# host. That is not a supported mode of operation for the packages being
# checked: the PSOCK port collision needed a hand-rolled `R_PARALLEL_PORT`
# split, the halves shared one TMPDIR, and any singleton a check assumes it
# owns was a latent false `newly_broken`. This workflow removes the
# simultaneity instead of isolating it: a package's old and new halves run
# SEQUENTIALLY, and the lost concurrency is won back ACROSS packages -- a
# custom bash work queue runs REVDEPX_WORKERS packages at once, each check in
# its own container (see revdep4/queue.sh).
#
# The queue is two-ended over the shard's heaviest-first package list: one
# worker eats from the heavy end, so the longest checks start first and are
# never the straggler discovered last; the remaining workers eat from the
# light end, draining the long tail of cheap packages in parallel; the two
# ends meet in the middle. One container per package also isolates
# *different* packages from each other -- stronger than revdep2, which only
# ever separated the two halves' ports.
#
# Sequential halves make per-half durations real measurements -- revdep2
# could only ever record the pair's shared wall clock. Both halves always
# run fresh: where the plan finds a valid old-version result from an earlier
# run it rides along as a *second
# opinion* (`baseline_agrees` records whether the fresh old check reproduced
# it, and disagreement is reported as drift), never as a substitute for the
# check itself.
#
# The jobs:
#
# plan enumerates the revdeps, weighs them (measured timings, else
# CRAN's times scaled; sequential halves priced at two checks),
# deals cost-balanced shards sized by the worker count.
# base builds (or finds, by content-derived tag) the check platform:
# rocker/r-ver at the pinned R version (4.6.1 by default -- at
# least one reverse dependency needs R-release) plus qpdf,
# ghostscript, pandoc, TeX, Xvfb and pak.
# build builds the dev package binary INSIDE the base container -- the
# binary must load under the container's R, not the runner's.
# universe installs the whole dependency universe (plus every system
# requirement) into a container from the base image -- or,
# delta-style, from the previous universe image when one fresh
# enough exists -- and commits + pushes the result to GHCR.
# test one job per shard: pull the universe image, install the two
# igraph halves into bind-mounted single-package libraries, then
# run the queue over the shard's packages. Checks run in three
# slices with an upload after each, so a reclaimed runner loses
# at most a third of a shard.
# collect folds every shard artifact into the revdepcheck-style report,
# the manifest, the baseline for later runs, and the timings the
# next plan calibrates from.
#
# The scripts live in `.github/workflows/revdepx/` (once shared with the
# retired concurrent-pair sibling, revdep3, whose PR was closed unmerged);
# artifacts, manifest schema and the universe image lineage are unchanged,
# so history from the sibling's live runs still serves as baselines and
# timings.
#
# Results land in artifacts:
# revdepx-report the merged report -- fetch with
# `gh run download <run-id> --name revdepx-report`
# or ./.github/workflows/revdepx/fetch.sh <run-id>
# revdepx-baseline old-version results, read back by later runs of
# either workflow as a second opinion beside their
# fresh old checks, while the revdep's version, our
# CRAN version, the container R, the base image and
# the resolved dependency versions all still match
# revdepx-timings what the checks and shards actually cost; the
# next plan (of either workflow) sizes shards by it
# revdepx-universe-report what the universe image build installed, loaded
# and could not
#
# To re-check only what a run could not declare ok, dispatch again with
# `retry-run: <run-id>`; the new report carries the old run's good results
# over, so it is complete again.
#
# Dispatch-only, deliberately: nothing here runs on push. Check results never
# turn the run red -- the summary and the report artifact are the
# deliverable; only infrastructure failures fail jobs.
on:
workflow_dispatch:
inputs:
ref:
description: "Branch, tag, or commit SHA to check (the tree must contain the revdepx scripts); default: the dispatched ref"
type: string
default: ""
packages:
description: "Packages to check (comma/space separated), or 'broken' to take what the committed revdep/ report lists as broken or failed; default: all reverse dependencies"
type: string
default: ""
which:
description: "Which reverse dependencies to enumerate"
type: choice
options:
- strong
- most
default: strong
depth:
description: "Levels of reverse dependencies: 1 = direct, 2 = also revdeps of revdeps, ..., 'all' = the full transitive closure"
type: string
default: "1"
retry-run:
description: "Run id of an earlier revdep4 run; re-check only its not-ok packages"
type: string
default: ""
part:
description: "Check one G-th of the reverse dependencies, as 'i/G'; for a set too big for one run (the plan says when, and with which G)"
type: string
default: ""
r-version:
description: "R version the checks run under: an explicit version like 4.6.1 (the pinned default -- at least one reverse dependency needs R-release), or 'oldrel'/'release' to resolve at run time."
type: string
default: "4.6.1"
workers:
description: "Packages checked concurrently per shard by the queue (one heavy lane, the rest light); default: the runner's core count"
type: string
default: ""
shard-budget-minutes:
description: "Check-time target per shard; smaller buys wall clock with more shards, up to one wave of max-parallel"
type: string
default: ""
max-parallel:
description: "Shards to run concurrently, and so the wave size the plan cuts: set it to the concurrency the account really has, never more (GitHub queues past its own limit anyway)"
type: string
default: ""
refresh-baseline:
description: "Offer no stored old results as second opinions (the old half always runs fresh either way)"
type: boolean
default: false
baseline-max-age-days:
description: "Oldest baseline result worth reusing"
type: string
default: ""
not-cran:
description: "Run the tests CRAN skips: sets NOT_CRAN=true, so skip_on_cran() does not skip (slower, noisier, and includes packages' own spelling tests)"
type: boolean
default: false
dry-run:
description: "Plan only: report the shards and reuse decisions, start no checks"
type: boolean
default: false
name: revdep4
# GitHub does not show a run the inputs it was dispatched with, so the ones
# that decide how much work the run is go into the run's own title.
run-name: "revdep4 ${{ inputs.packages && format('({0})', inputs.packages) || format('({0}, depth {1})', inputs.which || 'strong', inputs.depth || '1') }}${{ inputs['retry-run'] && format(', retry of {0}', inputs['retry-run']) || '' }}${{ inputs['dry-run'] && ', dry run' || '' }}"
# Read-only by default; every job opts back into what it actually needs.
permissions:
contents: read
# One revdepx run per checked ref at a time -- the group is named revdepx,
# not after this file, a relic of the retired sibling workflow that shared
# it. Two runs interleaving on one ref would race the committed report, the
# baseline lineage and the universe image's `latest` tag. A queued run waits
# rather than killing shards mid-check.
#
# `github.ref_name`, not `github.ref`: the input holds a bare name ("main")
# while `github.ref` is fully qualified ("refs/heads/main"), and a group
# built from the mismatched pair would let a run dispatched with the input
# filled in interleave with one dispatched without it -- on the very same
# branch the group exists to serialize. (A SHA or tag typed into the input
# forms its own group, which is correct: it is its own checked tree.)
concurrency:
group: revdepx-${{ inputs.ref || github.ref_name }}
cancel-in-progress: false
env:
REVDEPX_ENGINE: queue
REVDEPX_PACKAGES: ${{ inputs.packages || '' }}
REVDEPX_WHICH: ${{ inputs.which || 'strong' }}
REVDEPX_DEPTH: ${{ inputs.depth || '1' }}
REVDEPX_RETRY_RUN: ${{ inputs.retry-run || '' }}
REVDEPX_PART: ${{ inputs.part || '' }}
REVDEPX_R_VERSION_INPUT: ${{ inputs.r-version || '4.6.1' }}
REVDEPX_WORKERS: ${{ inputs.workers || vars.REVDEPX_WORKERS || '4' }}
# Hard memory cap per check container; empty lets queue.sh derive (total
# minus 2 GiB) / workers. A hungry check then kills its own container --
# one package's result -- instead of the runner. 6g deliberately
# overcommits the 15.6 GiB runner across 4 workers: checks rarely peak
# together, and the derived ~3.4g cap OOM-killed cc1plus during Stan/TMB
# compiles in run 32158907637 (dmesg: anon-rss ~3.3 GiB) -- misreported as
# packages "failing to install".
REVDEPX_MEMORY_PER_CHECK: ${{ vars.REVDEPX_MEMORY_PER_CHECK || '6g' }}
# Compiler flags appended for the check's own compile of the package under
# test (default -g0 in check-half.sh: debug info is what a Stan/TMB
# translation unit spends its compiler gigabytes on; set '-g' for CRAN's
# own flags) and MAKEFLAGS inside the check container (default -j1 there).
REVDEPX_CHECK_FLAGS: ${{ vars.REVDEPX_CHECK_FLAGS || '' }}
REVDEPX_CHECK_MAKEFLAGS: ${{ vars.REVDEPX_CHECK_MAKEFLAGS || '' }}
REVDEPX_SHARD_BUDGET_MINUTES: ${{ inputs.shard-budget-minutes || vars.REVDEPX_SHARD_BUDGET_MINUTES || '45' }}
REVDEPX_MAX_PARALLEL: ${{ inputs.max-parallel || vars.REVDEPX_MAX_PARALLEL || '20' }}
# Check minutes one shard may be planned to hold. Only a batch too big for
# one wave of REVDEPX_MAX_PARALLEL shards ever hits it, and then it decides
# how many waves there are; empty means 80% of REVDEPX_DEADLINE_MINUTES.
REVDEPX_SHARD_CAPACITY_MINUTES: ${{ vars.REVDEPX_SHARD_CAPACITY_MINUTES || '' }}
REVDEPX_REFRESH_BASELINE: ${{ inputs.refresh-baseline && '1' || '' }}
REVDEPX_BASELINE_MAX_AGE_DAYS: ${{ inputs.baseline-max-age-days || vars.REVDEPX_BASELINE_MAX_AGE_DAYS || '30' }}
REVDEPX_DRY_RUN: ${{ inputs.dry-run && '1' || '' }}
# Whether the checks behave like CRAN's own machines or like a developer's.
# `skip_on_cran()` reads `NOT_CRAN`; CRAN-like is the default: the point is
# to find what a released igraph would break, and a test CRAN never runs
# cannot break on CRAN.
REVDEPX_NOT_CRAN: ${{ inputs.not-cran && 'true' || 'false' }}
# How old the shared universe image may grow before a run rebuilds it from
# the base image instead of installing a delta on top. The delta path keeps
# runs fast; the periodic full rebuild keeps the image from accreting
# layers and stale system packages forever.
REVDEPX_IMAGE_MAX_AGE_DAYS: ${{ vars.REVDEPX_IMAGE_MAX_AGE_DAYS || '14' }}
# Per-check timeout: factor times the package's CRAN check time, but never
# below the floor -- CRAN's machines are not these runners. The floor is
# 30 min because a saturated shard runs each check at roughly half speed
# (measured queue-vs-pair inflation: median 1.5x, p90 1.8x, and up to
# ~3x in the hottest windows): run 32574134229 killed four packages at
# the old 20-min floor that finish in 7-12 min uncontended.
REVDEPX_TIMEOUT_FACTOR: ${{ vars.REVDEPX_TIMEOUT_FACTOR || '1.5' }}
REVDEPX_TIMEOUT_MIN_MINUTES: ${{ vars.REVDEPX_TIMEOUT_MIN_MINUTES || '30' }}
REVDEPX_DEADLINE_MINUTES: ${{ vars.REVDEPX_DEADLINE_MINUTES || '300' }}
jobs:
plan:
runs-on: ubuntu-26.04
# Planning is minutes, not hours: the preflight that used to share this
# job lives in the universe job now.
timeout-minutes: 60
name: "Plan shards"
outputs:
# Fallbacks cover a planning step that did not run at all: `test` is
# guarded by `shards != '0'`, but its `strategy` must still parse.
matrix: ${{ steps.plan.outputs.matrix || '{"shard":["none"]}' }}
shards: ${{ steps.plan.outputs.shards || '0' }}
packages: ${{ steps.plan.outputs.packages || '0' }}
max_parallel: ${{ steps.plan.outputs.max_parallel || '1' }}
baseline_run: ${{ steps.plan.outputs.baseline_run || '0' }}
plan_hash: ${{ steps.plan.outputs.plan_hash || 'none' }}
universe_count: ${{ steps.plan.outputs.universe_count || '0' }}
r_version: ${{ steps.resolve.outputs.r_version }}
permissions:
contents: read
# To find the newest earlier run (of either workflow) with a baseline
# artifact, to read the retried run's report, and to fetch timings.
actions: read
steps:
- name: Check out the ref under test
uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
# The same resolution runs in the base and build jobs: the three run in
# parallel and each needs the answer before any of them could publish
# it. It is one cached HTTPS call; determinism within a run is what
# matters, and the alias moves at most once every few months.
- name: Resolve the R version
id: resolve
uses: cynkra/cynkratemplate/.github/actions/revdepx-resolve-r-version@main
with:
input: ${{ env.REVDEPX_R_VERSION_INPUT }}
- name: Set up R
uses: r-lib/actions/setup-r@v2
with:
use-public-rspm: true
- name: Install jsonlite
run: |
install.packages("jsonlite")
shell: Rscript {0}
- name: Plan shards
id: plan
env:
GH_TOKEN: ${{ github.token }}
OUT: ${{ runner.temp }}/plan.json
run: |
Rscript ./.github/workflows/revdepx/plan.R
shell: bash
- name: Upload the shard plan
uses: actions/upload-artifact@v6
if: steps.plan.outputs.shards != '0'
with:
name: revdepx-plan
path: ${{ runner.temp }}/plan.json
retention-days: 30
overwrite: true
base:
# No needs: the base image depends only on the recipe in the checkout and
# the resolved R version, so this runs in parallel with planning. A base
# that already exists in the registry is found by tag and costs seconds.
if: inputs.dry-run != true
runs-on: ubuntu-26.04
timeout-minutes: 60
name: "Base image"
outputs:
image: ${{ steps.base.outputs.image }}
permissions:
contents: read
# To push the base image.
packages: write
steps:
- name: Check out the ref under test
uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
# Same resolution as in the plan job; see the comment there.
- name: Resolve the R version
uses: cynkra/cynkratemplate/.github/actions/revdepx-resolve-r-version@main
with:
input: ${{ env.REVDEPX_R_VERSION_INPUT }}
- name: Log in to the container registry
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ github.token }}
- name: Build or find the base image
id: base
env:
REVDEPX_PUSH: "1"
run: |
set -eu
repo=$(printf '%s' "${GITHUB_REPOSITORY}" | tr '[:upper:]' '[:lower:]')
ref=$(./.github/workflows/revdepx/base-image.sh \
"${REVDEPX_R_VERSION}" "ghcr.io/${repo}/revdepx-base" | tail -n 1)
echo "Base image: ${ref}"
echo "image=${ref}" >> "${GITHUB_OUTPUT}"
shell: bash
build:
needs:
- base
if: inputs.dry-run != true
runs-on: ubuntu-26.04
timeout-minutes: 90
name: "Build the dev binary"
permissions:
contents: read
# To pull the base image.
packages: read
steps:
- name: Check out the ref under test
uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
- name: Log in to the container registry
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ github.token }}
# The build's own pak downloads (igraph's dependencies and their system
# requirements) are small next to the universe's, but caching them is
# one step.
- name: Cache the pak package cache
uses: actions/cache@v5
with:
path: ${{ runner.temp }}/pak-cache
key: revdepx-pak-build-${{ hashFiles('DESCRIPTION') }}
restore-keys: |
revdepx-pak-build-
# Inside the base container, because the binary must load under the
# container's R (the pinned oldrel), and a binary built by the runner's
# newer R would not.
- name: Build the binary in the base container
uses: cynkra/cynkratemplate/.github/actions/revdepx-build-binary@main
with:
base-image: ${{ needs.base.outputs.image }}
- name: Upload the package binary
uses: actions/upload-artifact@v6
with:
name: revdepx-pkg
path: ${{ runner.temp }}/pkg
retention-days: 30
overwrite: true
universe:
needs:
- plan
- base
if: >-
!cancelled()
&& needs.plan.result == 'success'
&& needs.base.result == 'success'
&& needs.plan.outputs.shards != '0'
&& inputs.dry-run != true
runs-on: ubuntu-26.04
# Installing a whole dependency universe; when that goes wrong it tends
# to go wrong slowly. Below the 6 h ceiling so it ends as a failure with
# logs rather than as a hard kill.
timeout-minutes: 300
name: "Universe image"
outputs:
image: ${{ steps.publish.outputs.image }}
pushed: ${{ steps.publish.outputs.pushed }}
saved: ${{ steps.publish.outputs.saved }}
permissions:
contents: read
# To pull the base image and push the universe image.
packages: write
steps:
- name: Check out the ref under test
uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
- name: Download the shard plan
uses: actions/download-artifact@v7
with:
name: revdepx-plan
path: ${{ runner.temp }}/plan
- name: Log in to the container registry
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ github.token }}
# These runners have ONE disk: there is no separate /mnt volume, and
# RUNNER_TEMP, /var/lib/docker and the checkout all share the root
# filesystem (run 32068290691's sampler shows / only). So this cannot
# move docker anywhere roomier -- what it does is make room (the
# preinstalled toolchains this job never uses are tens of gigabytes)
# and put the data-root under RUNNER_TEMP, so every heavy write of
# this job lands in one accounted place. The commit step below
# measures against that one disk before copying anything large.
- name: Make room for docker on the one disk
uses: cynkra/cynkratemplate/.github/actions/revdepx-docker-room@main
# The universe build downloads every dependency once; saving the pak
# cache under the plan's hash hands the next build a warm start even
# when the image itself cannot be reused.
- name: Cache the pak package cache
uses: actions/cache@v5
with:
path: ${{ runner.temp }}/pak-cache
key: revdepx-pak-${{ needs.plan.outputs.plan_hash }}
restore-keys: |
revdepx-pak-
- name: Start the resource sampler
run: |
mkdir -p "${RUNNER_TEMP}/universe-out"
watch=./.github/workflows/revdepx/watch-resources.sh
"${watch}" once "before the universe build"
RESOURCE_LOG="${RUNNER_TEMP}/universe-out/resources.log" \
RESOURCE_PHASE_FILE="${RUNNER_TEMP}/universe-out/phase" \
"${watch}" watch 30 "universe build" &
disown || true
shell: bash
- name: Build the universe image
id: build-universe
uses: cynkra/cynkratemplate/.github/actions/revdepx-build-universe@main
with:
base-image: ${{ needs.base.outputs.image }}
# Everything here is written and UPLOADED before the commit below
# runs, because two runs (32068779192, 32084560474) died in that
# commit ~34 minutes in, runner and all -- and a dead runner turns out
# to lose even the log lines it already streamed. The numbers a
# post-mortem needs must be on the artifact store before the copy
# starts, or they do not exist.
- name: Record the state before the commit
id: precommit
uses: cynkra/cynkratemplate/.github/actions/revdepx-record-precommit@main
- name: Publish the pre-commit state (survives a dying runner)
uses: actions/upload-artifact@v6
with:
name: revdepx-universe-precommit
path: ${{ runner.temp }}/universe-out/pre-commit-state.txt
retention-days: 30
overwrite: true
# `docker commit` writes a full copy of the container's rw layer -- the
# entire installed universe -- as a new image layer, on the same one
# disk everything else already occupies. Two runs died exactly there,
# with logs, artifacts and cache saves. So: commit only when the copy
# provably fits -- in bytes AND in inodes, since `df` can show free
# gigabytes while the inode table runs dry under millions of small
# package files -- and even then under a watchdog that stops the
# docker daemon before the disk starves the runner agent: a wrong
# measurement must never cost a third runner. A skipped or aborted
# commit is not a failed run, it is the shards' local-build fallback
# -- slower, but alive, and the same path a failed universe job
# already takes.
- name: Commit and push the universe image
id: publish
uses: cynkra/cynkratemplate/.github/actions/revdepx-publish-universe@main
- name: Upload the universe image artifact (registry fallback only)
uses: actions/upload-artifact@v6
if: steps.publish.outputs.saved == 'true'
with:
name: revdepx-universe-image
path: ${{ runner.temp }}/universe-image
retention-days: 7
- name: Report what the build consumed
if: always()
run: |
./.github/workflows/revdepx/watch-resources.sh once "after the universe build"
./.github/workflows/revdepx/watch-resources.sh oom
shell: bash
- name: Upload the universe build report
uses: actions/upload-artifact@v6
if: always()
with:
name: revdepx-universe-report
path: ${{ runner.temp }}/universe-out
if-no-files-found: ignore
retention-days: 30
overwrite: true
test:
needs:
- plan
- base
- build
- universe
# A universe job that failed does not stop the shards: shard-prep.sh
# falls back to building a shard-local image from the base. Only the plan
# and the binary are hard prerequisites.
if: >-
!cancelled()
&& needs.plan.result == 'success'
&& needs.base.result == 'success'
&& needs.build.result == 'success'
&& needs.plan.outputs.shards != '0'
&& inputs.dry-run != true
runs-on: ubuntu-26.04
# Below GitHub's 6 h ceiling, and above the shard's own deadline: the
# shard is meant to stop itself and defer what is left, not to be killed
# mid-check.
timeout-minutes: 350
name: "shard ${{ matrix.shard }} (${{ matrix.label }})"
permissions:
contents: read
# To download the baseline artifact, which lives on an earlier run.
actions: read
# To pull the universe image.
packages: read
strategy:
fail-fast: false
max-parallel: ${{ fromJSON(needs.plan.outputs.max_parallel) }}
matrix: ${{ fromJSON(needs.plan.outputs.matrix) }}
env:
# What the checks inside the containers see; check-half.sh forwards
# exactly these. CRAN-like check conditions for the revdeps, not the
# incoming gauntlet.
R_REMOTES_NO_ERRORS_FROM_WARNINGS: true
# prevent rgl issues because no X11 display is available
RGL_USE_NULL: true
_R_CHECK_CRAN_INCOMING_: false
_R_CHECK_SYSTEM_CLOCK_: false
_R_CHECK_FUTURE_FILE_TIMESTAMPS_: false
_R_CHECK_FORCE_SUGGESTS_: false
# `--as-cran` turns on `--run-donttest`, which is the most expensive
# thing a check does and the least useful thing for this workflow:
# `\donttest{}` is where packages put the examples too slow to run on
# CRAN, so it is where the runners spend hours and where the timeouts
# land.
_R_CHECK_DONTTEST_EXAMPLES_: false
# No spell checking: a misspelling in a revdep's DESCRIPTION is the
# same misspelling in both halves.
_R_CHECK_CRAN_INCOMING_USE_ASPELL_: false
# No timings in the check output. The two halves run at different
# times next to different queue neighbours, so per-stage
# `[user/elapsed]` stamps differ by construction and would be pure
# diff noise. What a stage cost is recorded by the elapsed stamping
# on the driver log instead -- which the salvage now carries into the
# results artifact for every failed half, and check-half.sh prints as
# a stage timeline in the job log when a half dies.
_R_CHECK_TIMINGS_: ""
_R_CHECK_EXAMPLE_TIMING_THRESHOLD_: 99999
# 300 lines of a failed test transcript in the check log, not R's
# default 13 -- thirteen routinely cuts off the failure itself. R still
# writes the complete transcript to `<file>.Rout.fail` regardless.
_R_CHECK_TESTS_NLINES_: 300
GH_TOKEN: ${{ github.token }}
SHARD: ${{ matrix.shard }}
TIMEOUT_FACTOR: ${{ vars.REVDEPX_TIMEOUT_FACTOR || '1.5' }}
TIMEOUT_MIN_MINUTES: ${{ vars.REVDEPX_TIMEOUT_MIN_MINUTES || '30' }}
DEADLINE_MINUTES: ${{ vars.REVDEPX_DEADLINE_MINUTES || '300' }}
steps:
- name: Check out the ref under test
uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
# The host runs only the driver: R with jsonlite and rcmdcheck to
# parse, compare and account. Everything a check needs -- TeX, pandoc,
# qpdf, the dependency library -- lives in the image. This is the whole
# of the host setup that revdep2 spent TinyTeX, pandoc and apt minutes
# on, once per shard.
- name: Set up R
uses: r-lib/actions/setup-r@v2
with:
use-public-rspm: true
- name: Install jsonlite and rcmdcheck
run: |
install.packages(c("jsonlite", "rcmdcheck"))
shell: Rscript {0}
- name: Download the shard plan
uses: actions/download-artifact@v7
with:
name: revdepx-plan
path: ${{ runner.temp }}/plan
- name: Download the package binary
uses: actions/download-artifact@v7
with:
name: revdepx-pkg
path: ${{ runner.temp }}/pkg
# The baseline lives on an earlier run of either workflow; absence is
# not an error, the shard just records drift for nothing.
- name: Download the baseline results
uses: actions/download-artifact@v7
if: needs.plan.outputs.baseline_run != '0'
continue-on-error: true
with:
name: revdepx-baseline
run-id: ${{ needs.plan.outputs.baseline_run }}
github-token: ${{ github.token }}
path: ${{ runner.temp }}/baseline
# Only when the universe job could not push but could save: the image
# travelled as an artifact instead, and shard-prep.sh docker-loads it.
- name: Download the universe image artifact
uses: actions/download-artifact@v7
if: needs.universe.outputs.saved == 'true'
continue-on-error: true
with:
name: revdepx-universe-image
path: ${{ runner.temp }}/universe-image
- name: Log in to the container registry
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ github.token }}
# One disk on these runners (no /mnt): free what the preinstalled
# toolchains hold before the universe image unpacks into the data-root,
# and put that data-root under RUNNER_TEMP beside the checks' work
# directories, so everything heavy is in one accounted place.
- name: Make room for docker on the one disk
uses: cynkra/cynkratemplate/.github/actions/revdepx-docker-room@main
# Both driver phases need these and neither can name them itself: the
# `runner` context does not reach the job's `env`, so they are set once
# here.
#
# `NOT_CRAN` is written here because `r-lib/actions/setup-r` writes
# `NOT_CRAN=true` into `$GITHUB_ENV`, and a later write is what
# reliably overrides an earlier one.
- name: Point the driver at this runner's directories
uses: cynkra/cynkratemplate/.github/actions/revdepx-driver-env@main
with:
not-cran-input: ${{ env.REVDEPX_NOT_CRAN }}
# Emit machine state while the work runs: a reclaimed runner never
# executes its `if: always()` steps, so only what was already streamed
# to the log survives.
- name: Start the resource sampler
run: |
mkdir -p "${RUNNER_TEMP}/results"
watch=./.github/workflows/revdepx/watch-resources.sh
"${watch}" once "shard ${SHARD} before the prepare phase"
RESOURCE_LOG="${RUNNER_TEMP}/results/resources.log" \
"${watch}" watch 30 "shard ${SHARD}" &
disown || true
shell: bash
# Pull the universe image (or load the artifact, or -- when the
# universe job died -- build a shard-local image from the base), and
# extract its library index for the driver's dependency screen.
- name: Prepare the check image
uses: cynkra/cynkratemplate/.github/actions/revdepx-check-image@main
with:
universe-image: ${{ needs.universe.outputs.image }}
revdepx-base-image: ${{ needs.base.outputs.image }}
plan: ${{ runner.temp }}/plan/plan.json
- name: Prepare the shard
env:
PHASE: prepare
run: |
Rscript ./.github/workflows/revdepx/shard.R
shell: bash
# The checks run in three slices, each followed by an upload: a
# reclaimed runner loses at most a third of a shard, and the interim
# artifacts say `deferred` for the packages not yet reached, which is
# truer than the `missing` the collector would otherwise reconcile
# them into.
#
# `!cancelled()` rather than the default `success()`: a prepare step
# that failed still leaves a shard full of packages to account for, and
# the driver reports what it can, which is a result.
- name: Check the shard (1 of 3)
if: '!cancelled()'
env:
PHASE: check
CHECK_SLICE: 1/3
run: |
# Streamed from inside the step -- a sampler backgrounded in an
# earlier step stops reaching the job log when that step ends, and
# the checks are where the minutes (and the memory) actually go.
./.github/workflows/revdepx/watch-resources.sh watch 60 "shard ${SHARD} checks (1/3)" &
sampler=$!
trap 'kill "${sampler}" 2> /dev/null || true' EXIT
Rscript ./.github/workflows/revdepx/shard.R
shell: bash
- name: Upload the shard results (1 of 3)
uses: actions/upload-artifact@v6
if: always()
with:
name: revdepx-results-${{ matrix.shard }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/results
if-no-files-found: ignore
overwrite: true
retention-days: 30
- name: Check the shard (2 of 3)
if: '!cancelled()'
env:
PHASE: check
CHECK_SLICE: 2/3
run: |
./.github/workflows/revdepx/watch-resources.sh watch 60 "shard ${SHARD} checks (2/3)" &
sampler=$!
trap 'kill "${sampler}" 2> /dev/null || true' EXIT
Rscript ./.github/workflows/revdepx/shard.R
shell: bash
- name: Upload the shard results (2 of 3)
uses: actions/upload-artifact@v6
if: always()
with:
name: revdepx-results-${{ matrix.shard }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/results
if-no-files-found: ignore
overwrite: true
retention-days: 30
- name: Check the shard (3 of 3)
if: '!cancelled()'
env:
PHASE: check
CHECK_SLICE: 3/3
run: |
./.github/workflows/revdepx/watch-resources.sh watch 60 "shard ${SHARD} checks (3/3)" &
sampler=$!
trap 'kill "${sampler}" 2> /dev/null || true' EXIT
Rscript ./.github/workflows/revdepx/shard.R
shell: bash
# Before the final upload, so the last sample and the OOM verdict ride
# in the artifact.
- name: Report what the shard consumed
if: always()
run: |
./.github/workflows/revdepx/watch-resources.sh once "shard ${SHARD} after the checks"
./.github/workflows/revdepx/watch-resources.sh oom
df -BG / "${RUNNER_TEMP}" 2>/dev/null || true
shell: bash
# Named per attempt: a re-run of one shard must not overwrite the
# results the other shards uploaded in the first attempt; the collector
# reads every attempt and lets the later one win per package.
- name: Upload the shard results
uses: actions/upload-artifact@v6
if: always()
with:
name: revdepx-results-${{ matrix.shard }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/results
if-no-files-found: ignore
overwrite: true
retention-days: 30
collect:
needs:
- plan
- base
- build
- test
# `always()` so a run with red shards still reports -- a broken revdep is
# the result this workflow exists to surface. Guarded on the plan and the
# binary, because without those there is nothing to collect.
if: >-
always()
&& needs.plan.result == 'success'
&& needs.build.result == 'success'
&& needs.plan.outputs.shards != '0'
&& inputs.dry-run != true
runs-on: ubuntu-26.04
timeout-minutes: 120
name: "Collect results and report"
permissions:
# To commit the report back to the checked branch.
contents: write
# To download the retried run's report artifact, which lives on that
# run, and to read shard job durations for the timings.
actions: read
steps:
- name: Check out the ref under test
uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
- name: Set up R
uses: r-lib/actions/setup-r@v2
with:
use-public-rspm: true
- name: Install pak and jsonlite
run: |
install.packages("pak", repos = sprintf("https://r-lib.github.io/p/pak/stable/%s/%s/%s", .Platform$pkgType, R.Version()$os, R.Version()$arch))
install.packages("jsonlite")
shell: Rscript {0}
# The reports come out of revdepcheck itself; when this fails the
# collector still writes a manifest-derived summary.
- name: Install revdepcheck
continue-on-error: true
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
run: |
pak::pkg_install("krlmlr/revdepcheck")
shell: Rscript {0}
- name: Download the shard plan
uses: actions/download-artifact@v7
with:
name: revdepx-plan
path: ${{ runner.temp }}/plan
# Tolerated because a run in which no shard uploaded anything is still
# worth a report: the collector reconciles against the plan and names
# every package it never heard about, which is what `retry-run` needs.
- name: Download the results of all shards
uses: actions/download-artifact@v7
continue-on-error: true
with:
pattern: revdepx-results-*
path: ${{ runner.temp }}/results
- name: Download the report of the run being retried
uses: actions/download-artifact@v7
if: env.REVDEPX_RETRY_RUN != ''
continue-on-error: true
with:
name: revdepx-report
run-id: ${{ env.REVDEPX_RETRY_RUN }}
github-token: ${{ github.token }}
path: ${{ runner.temp }}/retry-report
- name: Collect results
id: collect
env:
# To read how long the shard *jobs* took -- the minutes before
# their driver starts, which is what an extra shard really costs.
GH_TOKEN: ${{ github.token }}
PLAN: ${{ runner.temp }}/plan/plan.json
RESULTS_DIR: ${{ runner.temp }}/results
RETRY_DIR: ${{ runner.temp }}/retry-report
OUT_DIR: revdep
BASELINE_OUT: ${{ runner.temp }}/baseline
TIMINGS_OUT: ${{ runner.temp }}/timings
run: |
Rscript ./.github/workflows/revdepx/collect.R
shell: bash
- name: Upload the report
uses: actions/upload-artifact@v6
if: always()
with:
name: revdepx-report
path: revdep
retention-days: 90
overwrite: true
# The report is also the repository's own record: `revdep/README.md`
# and friends have been committed here since revdepcheck wrote them,
# and `packages: broken` reads them back. Only the generated files and
# the manifest are staged.
#
# `!cancelled()` rather than `always()`: a cancelled run has no shard
# results, so its report is every package `missing`. `compared > 0`
# covers the rest of that family: a run in which *no* package produced
# a comparison learnt nothing, and must not overwrite the good record.
- name: Commit the report to the checked ref
if: >-
!cancelled()
&& vars.REVDEPX_COMMIT_REPORT != 'false'
&& steps.collect.outputs.compared != '0'
continue-on-error: true
uses: cynkra/cynkratemplate/.github/actions/revdep-commit-report@main
with:
ref: ${{ inputs.ref }}
workflow: revdep4
- name: Upload the new baseline
uses: actions/upload-artifact@v6
if: always()
with:
name: revdepx-baseline
path: ${{ runner.temp }}/baseline
retention-days: 90
overwrite: true
# What the run cost, per package and per shard. Small on purpose and
# separate from the report: the next plan (of either workflow)
# downloads it to calibrate its cost model, and should not have to
# fetch a report to do so.
- name: Upload the measured timings
uses: actions/upload-artifact@v6
if: always()
with:
name: revdepx-timings
path: ${{ runner.temp }}/timings
if-no-files-found: ignore
retention-days: 90
overwrite: true