Add cmake preset to compile 4C on Darwin

[vryy]

Description and Context

This PR adds cmake preset and necessary compilation scripts for the dependencies to compile 4C on the Mac/ARM system.

Related Issues and Pull Requests

#1771, #1779

[vryy]

I have tested the preset on a clean system. For the dependencies there are few remarks:

• homebrew does not provide parmetis, so parmetis has to be compile manually.
• cmake from homebrew is too modern (4.x.x), so cmake3 from botantony/cmake3 is recommended. Or you can also compile yourself.
• The installation script for trilinos assumes other dependencies (suitesparse, superlu_dist) located in a normal folder (not system folder), so the installation script of trilinos needs additional parameter to that folder, e.g., sh install.sh /path/to/install/dir path/to/dependency/dir. This can be changed later for the workflow.
• The compilation guide is summarized in a script, i.e., compile_dependencies.sh, which is self-explanatory and compile all the dependencies at once. Then 4C can be compiled using the preset darwin.

Lastly, the tools required to compile on the Mac is at least: brew install openmpi git wget gcc@14 hdf5 ninja llvm boost cln.

[sebproell]

cmake from homebrew is too modern

Are there any issues when using CMake 4.x.x?

[sebproell]

What are the future plans for adding all these Darwin-specific install scripts? Do you want to set up a docker build? These files should be tested in some way or they will stop working quickly. It would be OK to do this in a second step, I think.

I am not thrilled to see a new CMake preset, as I believe we should rather work to remove more and more of these. They are specific to your system, and the LNM and IMCS presets are here for legacy reasons.

[sebproell]

Standard JSON does not have comments.

Suggested change

	"FOUR_C_WITH_GOOGLETEST": "OFF" // set to off to disable unittest compilation
	"FOUR_C_WITH_GOOGLETEST": "OFF"

[vryy]

What are the future plans for adding all these Darwin-specific install scripts? Do you want to set up a docker build? These files should be tested in some way or they will stop working quickly. It would be OK to do this in a second step, I think.

I am not thrilled to see a new CMake preset, as I believe we should rather work to remove more and more of these. They are specific to your system, and the LNM and IMCS presets are here for legacy reasons.

Hey @sebproell that makes sense, the preset is only useful for the workflow. I will prepare to create a workflow for that.

[ppraegla]

What are the future plans for adding all these Darwin-specific install scripts? Do you want to set up a docker build?

Unfortunately, it is not possible to create a Darwin Docker image. Docker on MacOS runs a Linux virtual machine which then runs Linux-based images. The alternative is to use the GitHub-hosted runners with macOS. The free macos runner comes with 7 GB of memory (16 GB for Linux runner), which could pose a problem. Also, we would need to install all the dependencies during the workflow.

There is also the possibility to create custom VM images. But this requires a paid runner and, from what I have read, is only possible for Linux- and Windows-based images.

In summary, we could try to set up a workflow that uses a macos runner, install the dependencies during that workflow and build 4C. Due to the long runtime, I think we can only run this workflow once a week. But this should be fine.

[vryy]

What are the future plans for adding all these Darwin-specific install scripts? Do you want to set up a docker build?

Unfortunately, it is not possible to create a Darwin Docker image. Docker on MacOS runs a Linux virtual machine which then runs Linux-based images. The alternative is to use the GitHub-hosted runners with macOS. The free macos runner comes with 7 GB of memory (16 GB for Linux runner), which could pose a problem. Also, we would need to install all the dependencies during the workflow.

There is also the possibility to create custom VM images. But this requires a paid runner and, from what I have read, is only possible for Linux- and Windows-based images.

In summary, we could try to set up a workflow that uses a macos runner, install the dependencies during that workflow and build 4C. Due to the long runtime, I think we can only run this workflow once a week. But this should be fine.

Hi @ppraegla, thanks for the useful information. Would you be so kind to add some commits to setup a priori the macos workflow, since you seem to be the one to know it best? Then I will add scripts to install the dependencies and test.

In addition, the checkcode workflow throws the error: The hashes differ. The computed hash is 5244e650 while the container dependencies hash is 9ffec977. You probably need to adapt the dependencies hash in the workflow. How should I generate these dependency hash?

[ppraegla]

Hi @ppraegla, thanks for the useful information. Would you be so kind to add some commits to setup a priori the macos workflow, since you seem to be the one to know it best? Then I will add scripts to install the dependencies and test.

I can look into it some time next week.

How should I generate these dependency hash?

This is described here. You basically compute the hash, update the hash in the workflows (e.g. see #1813), open a PR to build the new docker image, and then rerun the actions in the PR where you need the new dependencies.

I suggest that we first set up the workflow for the macos runner. This will likely take some trial and error. Then we will see what we need to adapt in the dependency scripts.

[ppraegla]

@vryy I added the stub for the macos workflow.

In order to test the changes of this PR, you need to commit the workflow file to the main branch in your fork. Otherwise, the new workflow cannot be run. You will test the changes of this PR in your fork. Once the macos.yml workflow is on the main branch in your fork. You can go to Actions -> macOS build test -> Run workflow -> select your branch. Have a look what I did in my fork https://github.com/ppraegla/4C. The output of the workflow is here https://github.com/ppraegla/4C/actions/runs/23536820844/job/68514027187

I suggest that you try to get the workflow to pass in your fork. Once the macos workflow works, we can take care of the checks in this PR, i.e. updating the Docker image.

[ppraegla]

@vryy You need to adapt this part to install the dependencies. The build and test step should work. But may need some adapting as well.

[ppraegla]

I think we should aim not to duplicate the install scripts. There are only minimal changes compared to the Linux version. This also holds for the other install scripts.

However, I would be fine to keep them separate and think later about how we can unify the installation process.

Something like dealii candi looks interesting, but may be too much for us.

[ppraegla]

Also add a debug configuration with assertions that we can use for the workflow. I think it makes sense to test debug instead of release.

[vryy]

The problem is that the debug test took a lot of time, it may exceed the allowance of 6 hours. I switch to release test for now and also enable caching.

[amgebauer]

I think caching will not help here since we only have 20 GB of caches available on Github. Chances are very low that a cache survives a whole day.

[ppraegla]

The macos runner (M processor) only has 3 processors. So, the following test may fail preconditioner/tutorial_prec_solid_chebyshev.4C.yaml NP 4. Maybe this can be avoided with MPI --oversubscribe.

[vryy]

Hi @ppraegla thanks for providing the initial setup. I have added the compilation scripts and enabled the workflow. The current result is:

  95% tests passed, 131 tests failed out of 2852

The corresponding log is here: https://github.com/vryy/4C/actions/runs/23640661288/job/68860275643

[vryy]

Most of the tests failed because of unmatched tolerance, something we all knew about. Some differences are very closed, some are definitely something I will care about. Is there a way to specifically mark them as skipped for macos workflow?

[amgebauer]

Nice work!

Is there a way to specifically mark them as skipped for macos workflow?

No, there is not yet such a functionality. However, if it are just slight adaptions to the test tolerances, we should just adapt them instead of skipping the tests.

A further note, I would also try to activate all optional dependencies (like ArborX, VTK, GMSH) so that all tests are tested on MacOS

[davidrudlstorfer]

I just had a quick look into the results of a browndyn test and the results are completely different. I guess this has something to do again with our random number generation. Could we have some way to add results for different platforms? I think dealii does something similar right?

[maxfirmbach]

@vryy Most of the third-party library packages will be optional (also including parmetis, Umfpack, Superlu_dist, etc.). Would it make sense to try this with a minimal, no-dependency build first?

[maxfirmbach]

The macos runner (M processor) only has 3 processors. So, the following test may fail preconditioner/tutorial_prec_solid_chebyshev.4C.yaml NP 4. Maybe this can be avoided with MPI --oversubscribe.

I would rather opt to reduce the test to three procs.