Merge branch 'main' into agent/fix-nemotron-runtime-config #18539
cicd-main.yml
on: push
pre-flight
/
pre-flight
14s
Matrix: cicd-container-build
Launch_Unit_Tests_Core
9m 16s
Launch_Unit_Tests_Diffusion
2m 54s
Matrix: cicd-functional-tests-l0
Matrix: cicd-functional-tests-l1
Matrix: cicd-functional-tests-l2
Matrix: cicd-functional-tests-flaky
Matrix: cicd-functional-tests-gb200-l0
Matrix: cicd-functional-tests-gb200-l1
Matrix: cicd-functional-tests-gb200-l2
Matrix: cicd-functional-tests-gb200-flaky
Coverage_Fake
0s
Matrix: Coverage
Deployment protection rules
Reviewers, timers, and other rules protecting deployments in this run
| Event | Environments |
|---|---|
|
svcnvidia-nemo-ci
approved
|
test |
Annotations
1 error, 4 warnings, and 27 notices
|
cicd-container-build (aws, 766267172432.dkr.ecr.us-east-1.amazonaws.com, nemo-ci-aws-gpu-x2)
WARNING: The directory '/home/runner/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
|
|
pre-flight / pre-flight
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@v4, step-security/changed-files@v45.0.1. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
cicd-container-build (aws, 766267172432.dkr.ecr.us-east-1.amazonaws.com, nemo-ci-aws-gpu-x2)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: docker/build-push-action@v5, docker/setup-buildx-action@v3. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
Coverage (e2e)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
Coverage (unit-test)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
Result
✅ Launch_Unit_Tests_Diffusion — PASSED
|
|
Result
✅ Launch_Unit_Tests_Core — PASSED
|
|
Result
✅ L0_Launch_converter_generate — PASSED
|
|
Result
✅ L0_Launch_converter_fsdp — PASSED
|
|
Result
✅ L0_Launch_models_deepseek — PASSED
|
|
Result
✅ L0_Launch_recipes_perf_config — PASSED
|
|
Result
✅ L0_Launch_megatron_fsdp — PASSED
|
|
Result
✅ L0_Launch_models_gemma4_vl — PASSED
|
|
Result
✅ L0_Launch_recipes_deepseek — PASSED
|
|
Result
✅ L0_Launch_models_nemotron_omni — PASSED
|
|
Result
✅ L0_Launch_models_flux — PASSED
|
|
Result
✅ L0_Launch_recipes_llama_1b — PASSED
|
|
Result
✅ L0_Launch_training_callbacks — PASSED
|
|
Result
✅ L0_Launch_recipes_gemma4_vl — PASSED
|
|
Result
✅ L0_Launch_data — PASSED
|
|
Result
✅ L0_Launch_models_nemotronh — PASSED
|
|
Result
✅ L0_Launch_models_qwen3 — PASSED
|
|
Result
✅ L0_Launch_converter — PASSED
|
|
Result
✅ L0_Launch_training_distributed_save — PASSED
|
|
Result
✅ L0_Launch_training_decentralized_pg — PASSED
|
|
Result
✅ L0_Launch_training_local_checkpointing — PASSED
|
|
Result
✅ L0_Launch_training_finetune — PASSED
|
|
Result
✅ L0_Launch_training_pretrain — PASSED
|
|
Result
✅ L0_Launch_training_tensor_inspect — PASSED
|
|
Result
✅ L0_Launch_utils — PASSED
|
|
Result
✅ L0_Launch_training_resiliency — PASSED
|
|
Result
✅ L0_Launch_wan_ckpt_conversion — PASSED
|
Artifacts
Produced during runtime
| Name | Size | Digest | |
|---|---|---|---|
|
coverage-e2e-29957368596-05bddd4f-59e5-4405-b999-1f5faf19bd4c
|
315 KB |
sha256:08f836ffb4b58e17e778b57b8da336f31694de341b35253bb41fbf48d7e6eeca
|
|
|
coverage-e2e-29957368596-130411cc-b156-416b-b8fd-840f98ec7fc0
|
384 KB |
sha256:ced6b64bd1e059a58b57e624c0a3b525f655b3a3cf156ccd774c0fdddede114a
|
|
|
coverage-e2e-29957368596-2309a3ca-db5c-44fd-9054-cf9f375c338f
|
742 KB |
sha256:d2565bc08ec0e0e88c01fd508eedbbf73513960875f980f49a9e323e1bc7b365
|
|
|
coverage-e2e-29957368596-38a0261c-d1f1-42c9-9376-e07841c927cb
|
759 KB |
sha256:76e50ccbb9a902fd1cfa298f40eb9c854a7f0706db6480093baa10fa46be1982
|
|
|
coverage-e2e-29957368596-3a12804e-c150-476c-bcc1-8c3459633ae8
|
249 KB |
sha256:c4f05d0fc5ac49807f41027802571be5a1ef2c0ab95687b4eb52879c9937e443
|
|
|
coverage-e2e-29957368596-3de6d3d5-876f-4c0a-8bff-8dac1bd59182
|
316 KB |
sha256:ae78857db7324a953d95f72af21be84bddacfddbd96e145f934818fb3324592c
|
|
|
coverage-e2e-29957368596-48c9c1ec-9117-435e-ba1e-c0b18009fdfd
|
760 KB |
sha256:3ce025738c355dfcb12782bc6c8b348fa00a94aa246e596cc9127ef5c8ff8ba5
|
|
|
coverage-e2e-29957368596-50864a38-6d71-4a6c-bb67-243f1c2bf4da
|
808 KB |
sha256:cef2a955787ae5bd85788073c4cac16f5bb4590dee0fd4699039eb14f6956b5b
|
|
|
coverage-e2e-29957368596-57fc5ddc-c6d1-427f-9fff-f5e29589c16f
|
809 KB |
sha256:fded2f75e78c6a63921b2912645537e3574924e883950db05e14db3e265c0983
|
|
|
coverage-e2e-29957368596-69d09ec5-ec41-4dc1-a7c0-18ca01aef87b
|
339 KB |
sha256:1af9fe5f3654e01a4b50ecca2a377f7a23d4c7c4e36fab498df47d4264f522e2
|
|
|
coverage-e2e-29957368596-69ee86fb-2873-45b6-925b-334e0041ebca
|
744 KB |
sha256:a13514d2a65d2867e704b5fe13ae048b33415b46367752919e2785c3879152ca
|
|
|
coverage-e2e-29957368596-73180f91-f4d9-4c61-a764-6e0f6793dab5
|
249 KB |
sha256:0c26d535000b239bfe4ca4846ef769f8ed5cedf05e16e22125f168b754fcbffa
|
|
|
coverage-e2e-29957368596-834c4ac5-6138-4ba6-9a9f-292d39c3a057
|
742 KB |
sha256:3b1ca8efc6e5670f5d444098332e576771bca19487bbe77a4e8a5bdce16a391c
|
|
|
coverage-e2e-29957368596-86790b12-0840-4fcb-b864-25b898963dbd
|
287 KB |
sha256:77535bce8b2ea06624ac4b8d3d165831cc188063bd2b3bea27c817daa426ff49
|
|
|
coverage-e2e-29957368596-89b1776e-50e4-4e9d-ad9a-fec7805ae305
|
1.29 KB |
sha256:c4409f8c5ca352a3ae9493724d999f3d0df5c7ce2b30dd4a53b5ebe587421a2a
|
|
|
coverage-e2e-29957368596-89f3c63b-113b-48b8-9877-edfc5a27df75
|
460 KB |
sha256:f138b13549711f7e846e30727d754a6e8cf2a1460c3154fda04833231acb2aa9
|
|
|
coverage-e2e-29957368596-8d5454e1-8250-4f32-a305-954782b571c6
|
351 KB |
sha256:953109e5bf4b41e38a53121dc62be6b4cf9b5b16152da2e4fbbda294e09c5e5b
|
|
|
coverage-e2e-29957368596-9be0e183-d7c3-462f-9847-2d4701636597
|
316 KB |
sha256:040a4b99f1bb3eabbff7fba6f7c13df4d0687cfd6e6079448875add059dba491
|
|
|
coverage-e2e-29957368596-b15cf21e-de03-4581-a897-32bd58021b6d
|
332 KB |
sha256:7f492e2ef526add22ae6adaa3a4a675ee27bbb2c74867cfc4f67462d1979f789
|
|
|
coverage-e2e-29957368596-d7e7630e-8fb0-4402-aabb-2fee94537dfd
|
282 KB |
sha256:6220e178800de60e0d06bc48afad4a553e938b27a84ddd6908bf90d29d548ab6
|
|
|
coverage-e2e-29957368596-ddf99060-9530-4026-8d02-f8f2ac924117
|
288 KB |
sha256:5135e884d2733156695c47995e88d756a4269c17bb2a337a03f2ae5bf747f424
|
|
|
coverage-e2e-29957368596-ed9e2d5c-f019-4cfa-8e1a-d38db5719245
|
371 KB |
sha256:ea536f1e630dda4032c60666d6fc75821047f4846d50b97ba33e470b670ee8a5
|
|
|
coverage-e2e-29957368596-f144dac8-bbce-40c8-87d0-6c2832ebe048
|
512 KB |
sha256:ded6abf7875047cdac8b59e5e60bf123574589352a78a896f46e584425236b0e
|
|
|
coverage-e2e-29957368596-f394abbe-5bb5-4936-894b-94cb2e7c6634
|
742 KB |
sha256:59cca6a06cf7b0c90a85dea0834f01c2016f28e6283f89fe145e2ba40360da04
|
|
|
coverage-e2e-29957368596-fb5f0423-af6a-4c84-86b5-0becbba77f3b
|
816 KB |
sha256:71b022605b8c9db1d045385836dd9103f9b3faff11e2e1753ebeb403a6f27629
|
|
|
coverage-e2e-aggregated
|
145 KB |
sha256:6f9ac640952e85baca56e0736b1b007cb3ac9763dfdc1211c66b45cb815225c2
|
|
|
coverage-unit-test-29957368596-b2977812-08bd-4886-a956-db4b3330e1c6
|
337 KB |
sha256:4dce1946b4de9a9dc1503295f1a3de80697dec66e19fbd885a070925af15c88f
|
|
|
coverage-unit-test-29957368596-fbecccde-18d7-4034-8420-349faf957daa
|
863 KB |
sha256:2286c5914daf7539c371ede2d09abe4e9db6f62d807dbda16dc70e0f35f742fe
|
|
|
coverage-unit-test-aggregated
|
151 KB |
sha256:5962d3a6ce190d7834887ac1d6f954d679fe2e884d6c30cfa321a6af489cd331
|
|