DAOS-19005 test: Fix dfuse daos_build tests by phender · Pull Request #18363 · daos-stack/daos

phender · 2026-05-26T16:20:58Z

Resolve issues installing elftools and hatchling for daos_build tests.

Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: DaosBuild DaosBuildVM

Steps for the author:

Commit message follows the guidelines.
Appropriate Features or Test-tag pragmas were used.
Appropriate Functional Test Stages were run.
At least two positive code reviews including at least one code owner from each category referenced in the PR.
Testing is complete. If necessary, forced-landing label added and a reason added in a comment.

After all prior steps are complete:

Gatekeeper requested (daos-gatekeeper added as a reviewer).

Resolve issues installing elftools and hatchling for daos_build tests. Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daltonbohning · 2026-05-26T16:33:31Z

+            f'{sys.executable} -m pip install pip --upgrade',
+            f'{sys.executable} -m pip install -r {build_dir}/requirements-build.txt',


I wonder if something got missed somewhere because above is

f'{sys.executable} -m venv {mount_dir}/venv'

And even higher is

remote_env['VIRTUAL_ENV'] = os.path.join(mount_dir, 'venv')

But it doesn't seem that we ever source that virtual env?
Maybe we should remove the venv and VIRTUAL_ENV stuff from this test now and just use sys.executable as you're doing

github-actions · 2026-05-26T16:54:34Z

Ticket title is 'dfuse/daos_build_vm.py:DaosBuildVM.test_dfuse_daos_build_* - elftools / hatchling issues'
Status is 'In Progress'
Labels: '2.8.0rc1,ci_master_weekly,triaged,weekly_test'
https://daosio.atlassian.net/browse/DAOS-19005

daosbuild3 · 2026-05-26T18:43:48Z

Test stage Test RPMs on Leap 15.5 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18363/2/execution/node/591/log

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-05-27T00:07:44Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/3/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-05-27T17:36:05Z

Test stage Functional on EL 9 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18363/5/execution/node/635/log

daosbuild3 · 2026-05-29T02:09:53Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/6/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-05-29T22:26:33Z

Test stage Functional on EL 9 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18363/9/execution/node/573/log

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-01T14:01:16Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/10/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-01T14:51:51Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/11/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-01T17:15:40Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/12/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-01T19:07:26Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/13/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-01T22:11:26Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/14/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-01T23:33:07Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/15/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-02T06:52:55Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/16/testReport/

daosbuild3 · 2026-06-11T02:12:24Z

Test stage Functional on Leap 15 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18363/40/execution/node/710/log

daosbuild3 · 2026-06-11T02:22:04Z

Test stage Functional on EL 8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/40/testReport/

daosbuild3 · 2026-06-11T03:05:07Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/40/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Skip-func-test-el8: false Skip-func-test-leap15: false Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-11T04:26:19Z

Test stage Functional on SLES 15 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/43/testReport/

daosbuild3 · 2026-06-11T04:34:36Z

Test stage Functional on Leap 15 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/43/testReport/

daosbuild3 · 2026-06-11T04:35:31Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/43/testReport/

daosbuild3 · 2026-06-11T04:37:20Z

Test stage Functional on EL 8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/43/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Skip-func-test-el8: false Skip-func-test-leap15: false Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-11T19:00:46Z

Test stage Functional on SLES 15 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18363/44/execution/node/700/log

daosbuild3 · 2026-06-11T19:30:59Z

Test stage Functional on Leap 15 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18363/44/execution/node/710/log

daosbuild3 · 2026-06-11T19:46:24Z

Test stage Functional on EL 8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/44/testReport/

daosbuild3 · 2026-06-11T20:34:59Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/44/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Skip-func-test-el8: false Skip-func-test-leap15: false Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-12T00:39:40Z

Test stage Functional on EL 8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/45/testReport/

daosbuild3 · 2026-06-12T02:54:29Z

Test stage Functional on Leap 15 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18363/45/execution/node/710/log

daosbuild3 · 2026-06-12T05:37:14Z

Test stage Functional on SLES 15 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18363/45/execution/node/851/log

daosbuild3 · 2026-06-12T06:41:49Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/45/testReport/

Skip-unit-tests: true Skip-fault-injection-test: true Skip-func-test-el8: false Skip-func-test-leap15: false Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Cancel-prev-build: false Skip-unit-tests: true Skip-fault-injection-test: true Skip-func-test-el8: false Skip-func-test-leap15: false Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daosbuild3 · 2026-06-12T21:18:22Z

Test stage Functional on SLES 15 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18363/46/execution/node/677/log

daosbuild3 · 2026-06-12T21:55:28Z

Test stage Functional on Leap 15 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18363/46/execution/node/692/log

daosbuild3 · 2026-06-12T22:00:52Z

Test stage Functional on EL 8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/46/testReport/

daosbuild3 · 2026-06-12T22:51:29Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/46/testReport/

daosbuild3 · 2026-06-13T00:15:09Z

Test stage Functional on EL 8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/47/testReport/

daosbuild3 · 2026-06-13T00:45:27Z

Test stage Functional on SLES 15 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18363/47/execution/node/761/log

daosbuild3 · 2026-06-13T00:47:52Z

Test stage Functional on Leap 15 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18363/47/execution/node/731/log

daosbuild3 · 2026-06-13T01:56:15Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18363/47/testReport/

phender added 2 commits May 26, 2026 12:16

DAOS-19005 test: Fix dfuse daos_build tests

99a763e

Resolve issues installing elftools and hatchling for daos_build tests. Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Merge branch 'master' into hendersp/DAOS-19005

efadc77

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

daltonbohning reviewed May 26, 2026

View reviewed changes

Add sourcing the virtual environment.

768faf8

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

phender added 2 commits May 27, 2026 12:05

Merge branch 'master' into hendersp/DAOS-19005

09b88c6

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Fix command join.

2bf2f64

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

phender added 4 commits May 29, 2026 15:30

Merge branch 'master' into hendersp/DAOS-19005

6baaec0

Run a build commands in a remote script

deec53b

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Cleanup

55aa48a

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Make new file executable

fea782c

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Fix joining strings

2baa9c6

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

FIx path to daos_build.sh

3f8eec0

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Update

dbc7808

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Make daos_build.sh executable

aaf5698

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Updated daos_build.sh

ac704bb

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Updates

0cd12fd

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Bump timeouts

6f14600

Skip-unit-tests: true Skip-fault-injection-test: true Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

phender added 3 commits June 10, 2026 23:15

Debug

fffbf54

Skip-unit-tests: true Skip-fault-injection-test: true Skip-func-test-el8: false Skip-func-test-leap15: false Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Invoke the venv interpreter explicitly everywhere

d4ed390

Skip-unit-tests: true Skip-fault-injection-test: true Skip-func-test-el8: false Skip-func-test-leap15: false Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Cleanup

c5d3868

Skip-unit-tests: true Skip-fault-injection-test: true Skip-func-test-el8: false Skip-func-test-leap15: false Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Debug

b584033

Skip-unit-tests: true Skip-fault-injection-test: true Skip-func-test-el8: false Skip-func-test-leap15: false Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Trying forced tty allocation

20ca59b

Skip-unit-tests: true Skip-fault-injection-test: true Skip-func-test-el8: false Skip-func-test-leap15: false Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

phender added 2 commits June 12, 2026 14:40

Try scons --enable-virtualenv again

dfdf0e5

Skip-unit-tests: true Skip-fault-injection-test: true Skip-func-test-el8: false Skip-func-test-leap15: false Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

Trying SCONS_ENV=full

e8eff8a

Cancel-prev-build: false Skip-unit-tests: true Skip-fault-injection-test: true Skip-func-test-el8: false Skip-func-test-leap15: false Test-tag: DaosBuild DaosBuildVM Signed-off-by: Phil Henderson <phillip.henderson@hpe.com>

		f'{sys.executable} -m pip install pip --upgrade',
		f'{sys.executable} -m pip install -r {build_dir}/requirements-build.txt',

Conversation

phender commented May 26, 2026

Steps for the author:

After all prior steps are complete:

Uh oh!

daltonbohning May 26, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

daosbuild3 commented May 26, 2026

Uh oh!

daosbuild3 commented May 27, 2026

Uh oh!

daosbuild3 commented May 27, 2026

Uh oh!

daosbuild3 commented May 29, 2026

Uh oh!

daosbuild3 commented May 29, 2026

Uh oh!

daosbuild3 commented Jun 1, 2026

Uh oh!

daosbuild3 commented Jun 1, 2026

Uh oh!

daosbuild3 commented Jun 1, 2026

Uh oh!

daosbuild3 commented Jun 1, 2026

Uh oh!

daosbuild3 commented Jun 1, 2026

Uh oh!

daosbuild3 commented Jun 1, 2026

Uh oh!

daosbuild3 commented Jun 2, 2026

Uh oh!

daosbuild3 commented Jun 11, 2026

Uh oh!

daosbuild3 commented Jun 11, 2026

Uh oh!

daosbuild3 commented Jun 11, 2026

Uh oh!

daosbuild3 commented Jun 11, 2026

Uh oh!

daosbuild3 commented Jun 11, 2026

Uh oh!

daosbuild3 commented Jun 11, 2026

Uh oh!

daosbuild3 commented Jun 11, 2026

Uh oh!

daosbuild3 commented Jun 11, 2026

Uh oh!

daosbuild3 commented Jun 11, 2026

Uh oh!

daosbuild3 commented Jun 11, 2026

Uh oh!

daosbuild3 commented Jun 11, 2026

Uh oh!

daosbuild3 commented Jun 12, 2026

Uh oh!

daosbuild3 commented Jun 12, 2026

Uh oh!

daosbuild3 commented Jun 12, 2026

Uh oh!

daosbuild3 commented Jun 12, 2026

Uh oh!

daosbuild3 commented Jun 12, 2026

Uh oh!

daosbuild3 commented Jun 12, 2026

Uh oh!

daosbuild3 commented Jun 12, 2026

Uh oh!

daosbuild3 commented Jun 12, 2026

Uh oh!

daosbuild3 commented Jun 13, 2026

Uh oh!

daosbuild3 commented Jun 13, 2026

Uh oh!

daosbuild3 commented Jun 13, 2026

Uh oh!

daosbuild3 commented Jun 13, 2026

github-actions Bot commented May 26, 2026 •

edited

Loading