need help: test fails on darwin x86-64 (but curiously not aarch64)

[donn]

The test test_tch1dn/zch1dn fails on x86-64 versions of macOS, but not aarch64.

The residual errors are worse overall on the former, but it is only large enough to tip over into a failure for zch1dn.

Is this too serious of a problem? Is there a way for me to adjust the threshold?

x86-64 test log

Output:
----------------------------------------------------------

 testing Cholesky rank-1 downdate routines.
 All residual errors are expected to be small.

 sch1dn test:
      residual error =              0.572204589844E-05       PASS
 dch1dn test:
      residual error =              0.888178419700E-14       PASS
 cch1dn test:
      residual error =              0.953972266871E-05       PASS
 zch1dn test:
      residual error =              0.497379915032E-13       FAIL
----------------------------------------------------------------------
 total:     PASSED   3     FAILED   1

aarch64 test log

Output:
----------------------------------------------------------

 testing Cholesky rank-1 downdate routines.
 All residual errors are expected to be small.

 sch1dn test:
      residual error =              0.476837158203E-05       PASS
 dch1dn test:
      residual error =              0.106581410364E-13       PASS
 cch1dn test:
      residual error =              0.152587890625E-04       PASS
 zch1dn test:
      residual error =              0.284217094304E-13       PASS
----------------------------------------------------------------------
 total:     PASSED   4     FAILED   0

[grisuthedragon]

Can you give some more details about the used BLAS library? Since the must be a reason. Nevertheless I seem to be safe to adjust the tolerance a bit.
Just change the the factor 2D2 in

qrupdate-ng/test/utils.f

Line 258 in 44a34de

if (rnrm < 2d2*dlamch('p')) then

to 1D3 ,

[doronbehar]

Hey, on Nixpkgs (Which also distributes software for both Darwin platforms) we experience exactly the same issue with x86_64-darwin (and not aarch64-darwin):

 1/13 Test  #1: test_tch1dn ......................***Failed   16.07 sec
 testing Cholesky rank-1 downdate routines.
 All residual errors are expected to be small.
 sch1dn test:
      residual error =              0.572204589844E-05       PASS
 dch1dn test:
      residual error =              0.106581410364E-13       PASS
 cch1dn test:
      residual error =              0.953972266871E-05       PASS
 zch1dn test:
      residual error =              0.497379915032E-13       FAIL
----------------------------------------------------------------------
 total:     PASSED   3     FAILED   1
Note: The following floating-point exceptions are signalling: IEEE_UNDERFLOW_FLAG IEEE_DENORMAL
STOP 1

I don't have available the floating points we had on aarch64-darwin unfortunately (because there the tests passed). Our blas and lapack implementations are both based on openblas version 0.3.27. The build log of it is available here (for x86_64-darwin):

https://cache.nixos.org/log/vdk8dns4jvy1n7w1djhdy7i1a3ph37p0-openblas-0.3.27.drv

I don't have personally an x86_64-darwin machine, so I am able to only use our CI which is very slow for these platforms unfortunately, so I won't be able to help much in debugging. I hope the debugging information I provided helps a bit.

[donn]

It's the same blas version FWIW. I am using Nix to build qrupdate.

[grisuthedragon]

It seems that the tolerances need to be adjusted a bit more. I'll prepare a patch during the next days. But the solution seems to adjust this line

qrupdate-ng/test/utils.f

Line 258 in 44a34de

if (rnrm < 2d2*dlamch('p')) then

again.

[barracuda156]

FWIW, on macOS PowerPC all 13 tests pass normally for v. 1.1.5.

[Alessandro-Barbieri]

Tests fail for me on nixos linux x86_64 while using blis as dependency

[grisuthedragon]

@Alessandro-Barbieri Can you show the test outputs?

[Alessandro-Barbieri]

Running phase: checkPhase
@nix { "action": "setPhase", "phase": "checkPhase" }
check flags: SHELL=/nix/store/ih68ar79msmj0496pgld4r3vqfr7bbin-bash-5.2p37/bin/bash VERBOSE=y test
�[36mRunning tests...�[0m
/nix/store/29ax4k0a83zhz43lb73cv610d95wdsx1-cmake-3.31.6/bin/ctest --force-new-ctest-process 
Test project /build/source/build
      Start  1: test_tch1dn
      Start  2: test_tch1up
      Start  3: test_tchdex
      Start  4: test_tchinx
      Start  5: test_tchshx
      Start  6: test_tlu1up
      Start  7: test_tlup1up
      Start  8: test_tqr1up
      Start  9: test_tqrdec
      Start 10: test_tqrder
      Start 11: test_tqrinc
      Start 12: test_tqrinr
 1/13 Test  #6: test_tlu1up ......................   Passed    0.46 sec
      Start 13: test_tqrshc
 2/13 Test  #7: test_tlup1up .....................   Passed    0.46 sec
 3/13 Test #12: test_tqrinr ......................   Passed    1.99 sec
 4/13 Test #10: test_tqrder ......................   Passed    2.26 sec
 5/13 Test  #2: test_tch1up ......................***Failed    3.34 sec

 testing Cholesky rank-1 update routines.
 All residual errors are expected to be small.

 sch1up test:
      residual error =              0.163398780823E+02       FAIL
 dch1up test:
      residual error =              0.181205456641E+02       FAIL
 cch1up test:
      residual error =              0.152587890625E-04       PASS
 zch1up test:
      residual error =              0.285361874708E-13       PASS
----------------------------------------------------------------------
 total:     PASSED   2     FAILED   2

Note: The following floating-point exceptions are signalling: IEEE_DENORMAL
STOP 1

 6/13 Test #11: test_tqrinc ......................   Passed    3.16 sec
 7/13 Test  #9: test_tqrdec ......................   Passed    3.51 sec
 8/13 Test  #1: test_tch1dn ......................***Failed    3.67 sec

 testing Cholesky rank-1 downdate routines.
 All residual errors are expected to be small.

 sch1dn test:
      residual error =              0.163398742676E+02       FAIL
 dch1dn test:
      residual error =              0.181205456641E+02       FAIL
 cch1dn test:
      residual error =              0.114596032290E-04       PASS
 zch1dn test:
      residual error =              0.213162820728E-13       PASS
----------------------------------------------------------------------
 total:     PASSED   2     FAILED   2

Note: The following floating-point exceptions are signalling: IEEE_UNDERFLOW_FLAG IEEE_DENORMAL
STOP 1

 9/13 Test  #4: test_tchinx ......................***Failed    3.84 sec

 testing Cholesky symmetric insert routines.
 All residual errors are expected to be small.

 schinx test:
      residual error =              0.163398742676E+02       FAIL
 dchinx test:
      residual error =              0.181135123196E+02       FAIL
 cchinx test:
      residual error =              0.768876816437E-05       PASS
 zchinx test:
      residual error =              0.142385832386E-13       PASS
----------------------------------------------------------------------
 total:     PASSED   2     FAILED   2

Note: The following floating-point exceptions are signalling: IEEE_UNDERFLOW_FLAG IEEE_DENORMAL
STOP 1

10/13 Test  #3: test_tchdex ......................***Failed    4.03 sec

 testing Cholesky symmetric delete routines.
 All residual errors are expected to be small.

 schdex test:
      residual error =              0.163398742676E+02       FAIL
 dchdex test:
      residual error =              0.181135123196E+02       FAIL
 cchdex test:
      residual error =              0.954865663516E-05       PASS
 zchdex test:
      residual error =              0.284217094304E-13       PASS
----------------------------------------------------------------------
 total:     PASSED   2     FAILED   2

Note: The following floating-point exceptions are signalling: IEEE_UNDERFLOW_FLAG IEEE_DENORMAL
STOP 1

11/13 Test #13: test_tqrshc ......................   Passed    3.53 sec
12/13 Test  #8: test_tqr1up ......................   Passed    4.03 sec
13/13 Test  #5: test_tchshx ......................***Failed    4.36 sec

 testing QR column shift routines.
 All residual errors are expected to be small.

 schshx test (left shift):
      residual error =              0.163398742676E+02       FAIL
 dchshx test (left shift):
      residual error =              0.181135123196E+02       FAIL
 cchshx test (left shift):
      residual error =              0.114440917969E-04       PASS
 zchshx test (left shift):
      residual error =              0.177760540151E-13       PASS
 schshx test (right shift):
      residual error =              0.173504104614E+02       FAIL
 dchshx test (right shift):
      residual error =              0.178075760678E+02       FAIL
 cchshx test (right shift):
      residual error =              0.953841936280E-05       PASS
 zchshx test (right shift):
      residual error =              0.179622969087E-13       PASS
----------------------------------------------------------------------
 total:     PASSED   4     FAILED   4

STOP 1


62% tests passed�[0;0m, �[0;31m5 tests failed�[0;0m out of 13

Total Test time (real) =   4.41 sec

The following tests FAILED:
	�[0;31m  1 - test_tch1dn (Failed)�[0;0m
	�[0;31m  2 - test_tch1up (Failed)�[0;0m
	�[0;31m  3 - test_tchdex (Failed)�[0;0m
	�[0;31m  4 - test_tchinx (Failed)�[0;0m
	�[0;31m  5 - test_tchshx (Failed)�[0;0m
Errors while running CTest
make: *** [Makefile:71: test] Error 8

[grisuthedragon]

That's far away from roundoff errors. Can you provide the following additional information:

• used fortran compiler
• used compiler flags
• used BLAS and LAPACK library

[Alessandro-Barbieri]

That's far away from roundoff errors. Can you provide the following additional information:

* used fortran compiler

gfortran 14.2.1

* used compiler flags

-- CMAKE_Fortran_FLAGS: -std=legacy -frecursive -fPIC -fimplicit-none
-- CMAKE_Fortran_FLAGS_RELEASE: -O3

* used BLAS and LAPACK library

amd-blis 5.1 and amd-libflame 5.1

[grisuthedragon]

Can you try it with a "standard" blis and a standard "libflame"? On which architecture did you try the amd blis?

[Alessandro-Barbieri]

Can you try it with a "standard" blis
and a standard "libflame"?

libflame isn't packaged in nixos

I can try blis and amd-libflame

On which architecture did you try the amd blis?

x86_64 zen2

[grisuthedragon]

Then give blis and amd-libflame a try. Otherwise everything with OpenBLAS.

I do an experiment on a zen2 box as well.