Port pony to Haiku

[ahwayakchih]

Hi,

This is my attempt to port Pony to Haiku operating system (https://www.haiku-os.org/).

It passes all the tests (but i had to force regression-1118 to use just 2 threads on Haiku, otherwise it times out, and disable error same way as Windows has it disabled), all examples work:
pony-test.log

Runtime tracing works, except for crash when actor behavior category is enabled (i'm attaching crash log, maybe you could help me fix that?):
net-5509-debug-17-05-2026-16-58-34.report.txt.

I did get some crashes with systematic_testing, but i have to retest it yet.

I did not test LTO, pony-lint and pony-lsp (i mean, besides checking if executable works).
pony-doc seems to work too.

libponyc and libponyrt benchmarks work, but they warn about not being able to set thread affinity (but i think that may be because of benchmark library?):
pony-benchmark.txt

If you think this has a chance to be merged, i can try to prepare CI config for running Haiku, but it would have to be a bit... hacky. Haiku is not Linux or BSD, so running it means running QEmu, and building pony requires over 10 GB of space on disk. My idea is to prepare install image, and then separate data disk (qcow) for pony. Disks could be cached between CI runs. First build run would be horribly long (it took over half a day on my laptop ;).

[redvers]

I will tell you that there are BeOS fans among the ponyc maintainers ;)

I found inspiration in https://www.haiku-os.org/legacy-docs/bebook/TheInterfaceKit_Overview_Introduction.html

[SeanTAllen]

Hi. Thanks for taking this on. I haven't looked at all. Some general comments.

I have a couple of large changes coming that I would want to land before these and would then ask you to incorporate and update again.

We need to talk about if we want to take this on as platform we support in the codebase. I love BeOS dearly. I learned to "really C++" on it. You will find my name in the old BeOS newsletter. But I am hesitant.

I'd like to see things like:

        ifdef haiku then
          // Workaround haiku bug with parsing "nan(123)": https://dev.haiku-os.org/ticket/20092
          if (errno == 0) and (endp == ptr.offset(_size - 1)) and (this.at_offset(_size.isize() - 1)? == 0x29) then
            res

addressed upstream and not carry the cost of this in the codebase if we were to accept it.

I notice there are at least a few such things.

---

Linking would need to be made to work with the new embedded linker that we are moving all platforms to before merging. I'm not going to take on doing that work on Haiku. It would need to be done before merging. All the Linuxes, MacOS, and Windows currently use it. The BSDs have yet to be ported over.

---

I note that there are comment like "under pressure example consistently crashes". This and other issues would need to be resolved before merging.

---

Lastly, it would need a tier 3 CI job like what OpenBSD, FreeBSD, and DragonFly have.

Before you undertake any work related to the above, I'd suggest waiting for our decision if we would want to take this on to maintain long term.

[SeanTAllen]

I will tell you that there are BeOS fans among the ponyc maintainers ;)

I found inspiration in https://www.haiku-os.org/legacy-docs/bebook/TheInterfaceKit_Overview_Introduction.html

Sylvan was also a BeOS developer. Both of us were.

[ahwayakchih]

Thanks for checking and answering :).

I will tell you that there are BeOS fans among the ponyc maintainers ;)

I found inspiration in https://www.haiku-os.org/legacy-docs/bebook/TheInterfaceKit_Overview_Introduction.html

Great! BeOS was my favorite OS, unfortunately i learned about it only around BeOS R5 PE, so not long before the end. But i was lucky enough to use it as my one and only OS for a few years, before i had to change hardware :).

Sylvan was also a BeOS developer. Both of us were.

Even better :D! Thanks!
Now i know why, when i've read about pony, i thought "this could be perfect for Haiku, with all it's multi-threading wrapped by actors, it could make writing UI easier for newcommers (or lazy people like me ;)".

I have a couple of large changes coming that I would want to land before these and would then ask you to incorporate and update again.
addressed upstream and not carry the cost of this in the codebase if we were to accept it.

Sure, i already saw change about strtof/d, so the ugly workaround i did was mainly to pass the tests and i did plan to remove it eventually. I guess it would be better to just ifdef the only two test cases that fail on Haiku? I reported the problem, but i don't think they'll work on this anytime soon.

The other workaround is for socket shutdown, and i plan to prepare test case and report it (unless it's just something that i messed up in asio implementation and i'll be able to fix it myself).

Two smaller things are in tests:

• one in regression-1118 (with more threads it times out when tests run, but it does not fail or crash when i run it separately, it's just slow),
• error.cc test, but that's already ifdefed for Windows too.

I was trying to keep changes minimal, so most of it is just adding "or haiku" or "and not haiku" or error/signal constants in places where platform is already checked in code.

Two "bigger" chunks of code are in:

• cpu.c: i would prefer to add some platform-specific file and call to it from cpu.c, but i was not sure if you'd like more files or not (Linux code there is also "big" for that file, but i understand that's one of main platforms, so it has "privileges" :),
• wfo.c: new file for asio on Haiku, but other implementations have own files, so i suspect that's ok.
• edited genexe.cc: forgot about this one: bigger chunk of code is to support rdef files, but i kept it in separate functions, so hopefully it's not too annoying for others.

I plan to remove custom logging from wfo.c (LOG and LOGLIST stuff). It's suboptimal, it just helped me a lot while implementing asio. I was thinking about adding stuff to runtime tracing, but decided to postpone it, because that could mess with other platforms (either it would be strange that there are functions that only one platform uses, or i would have to implemented it for all of them, some of which i would not have a way to test besides waiting for CI).

There is kevent implementation for Haiku, but it's missing some of the stuff used in pony. I wanted to see if i can write Haiku-specific implementation anyway ;).

I was also thinking about implementing threading using native threads and ports, but that would require more changes, because pthread* stuff is not contained in a single file. Also messaging could be slower then, because it would need to pass through Haiku kernel then, and AFAIK it's all on pony side now.

Linking would need to be made to work with the new embedded linker that we are moving all platforms to before merging. I'm not going to take on doing that work on Haiku. It would need to be done before merging. All the Linuxes, MacOS, and Windows currently use it. The BSDs have yet to be ported over.

Hmm... i could try it, but i never wrote any linking (or compiler, for that matter) code before, so... i would need some pointers regarding what's needed and where, or maybe i'll wait for BSDs and follow their steps?

BTW i'm not sure yet about static libs. BeOS and Haiku are all for shared libraries, and i did not have a look yet to see how to make sure pony stuff links dynamically and then generate ponylang package with correct content (to add it to HaikuDepot).

I note that there are comment like "under pressure example consistently crashes". This and other issues would need to be resolved before merging.

Yes, of course. But under pressure works fine now. Actually nothing crashes AFAIK, except for runtime tracing actor behaviors. Not sure why. It looks like the tracing thread calls a function chain that ends up checking if tracing is enabled and then it crashes. It's kinda similar to how timers were crashing, only there thread ctx (like on Windows) was needed, and it looked like thread local variable access was crashing. My guess is that, it's because it's Haiku's thread that calls timer function. Once i changed it to just "ping" asio thread, which in turn sends event, timers started working ok 100% of time.

Lastly, it would need a tier 3 CI job like what OpenBSD, FreeBSD, and DragonFly have.

Yeah, that's why i was already thinking about how to implement that.

Before you undertake any work related to the above, I'd suggest waiting for our decision if we would want to take this on to maintain long term.

OK, thanks for at least considering it anyway. I know Haiku is way smaller in popularity than other platforms, and i understand well that maintaining project takes precious time. So, if it means too much of a burden for pony team (for now?), i can keep fork updated from time to time and maybe you can reconsider merging in the future :).

[SeanTAllen]

I'm completely ok if we decide to move forward with Haiku support of treating it as a real option and leaning into its own APIs rather than doing things like riding on a kqueue compatibility layer. So long as those Haiku APIs dont bring an oversized burden for their relative merit. Just had to caveat that. If we take on Haiku, we should take on Haiku.

I was also thinking about implementing threading using native threads and ports, but that would require more changes, because pthread* stuff is not contained in a single file. Also messaging could be slower then, because it would need to pass through Haiku kernel then, and AFAIK it's all on pony side now.

I'm not sure where this would fall on the line I put above. I would encourage doing a rough and dirty spike for what it would look like to get early feedback before investing a ton of time. That would probably be the answer with everything "like this". "Can you show us what it might look like and we can tell you how likely we would be to accept?"

I really feel bad when people put a lot of work into something and then we reject it. I'd much rather they talk with us early on and rejections come with much less effort that won't get merged (I dont want to say the effort is wasted as people often learn a ton in the process).

Re the linking: you dont need to write linker. You "just" need to use the embedded LLD linker like the MacOS, Windows, and Linux versions are. Basically, instead of calling out to an external linker, you call into the "lld as a library" that exists in the pony compiler.

BTW i'm not sure yet about static libs. BeOS and Haiku are all for shared libraries, and i did not have a look yet to see how to make sure pony stuff links dynamically and then generate ponylang package with correct content (to add it to HaikuDepot).

And again, thank you for doing this. BeOS is still my favorite OS. I still have 2 early rev BeBoxes and I still check in on Haiku once a year to see if there is any way I could use it as a daily driver. It's a ritual I have been undertaking for at least 10 years and almost assuredly longer.

I would model after MacOS on this front as it "doesn't really do static libs".

[ahwayakchih]

I would encourage doing a rough and dirty spike for what it would look like to get early feedback before investing a ton of time. That would probably be the answer with everything "like this". "Can you show us what it might look like and we can tell you how likely we would be to accept?"

Hmm... then maybe i'll try it, after i'm sure everything else is 100% OK. I know that sending a message to native thread/port means it's copied, so i guess i could send pointer to pony message data instead of whole data... anyway, i suspect it will be slower than just passing pointer all on pony side. Still, it could be interesting to see how it would look like anyway.

I really feel bad when people put a lot of work into something and then we reject it. I'd much rather they talk with us early on and rejections come with much less effort that won't get merged (I dont want to say the effort is wasted as people often learn a ton in the process).

No worry :). It's on me - i got curious and carried away ;). And i did learn stuff and got reminded about a lot more (i'm not working with C/C++, and never really did professionally, so my knowledge is... spotty and/or rusty).

Basically, instead of calling out to an external linker, you call into the "lld as a library" that exists in the pony compiler.

OK, i'll try that :).

BeOS is still my favorite OS. I still have 2 early rev BeBoxes and I still check in on Haiku once a year to see if there is any way I could use it as a daily driver. It's a ritual I have been undertaking for at least 10 years and almost assuredly longer.

Nice! I'm checking Haiku from time to time too, but less regularly (usually when i'm between work/projects or on holidays) and in meantime i just try to be up to date on the stuff happening there.

I would model after MacOS on this front as it "doesn't really do static libs".

OK, thanks!

[SeanTAllen]

@ahwayakchih please proceed if you want. our caveat would be. we don't see ourselves supporting haiku going forward but we are happy to take into mainline with all the previous caveats if you are willing to commit to doing ongoing support and if breakages start piling up and you are no longer around, we would almost assuredly remove support.

Sound good?

[ahwayakchih]

@SeanTAllen that's great! and very fair, thanks!

[ahwayakchih]

Hi,

I've rebased Haiku port onto latest ponyc source and applied additional changes:

• use embedded LLD on Haiku,
• disable IPv6 tests on Haiku (they keep failing there, and i'm not sure yet why, most probably because i can't configure IPv6 on Haiku - it has to be set with static IP there, and then on Linux host i'm using passt for network, and... anwyay... a lot of places to fail),
• add very thin ntohs and ntohl wrappers around builtin byte swaps, because they are pure defines on Haiku, so there no symbols in libraries when Pony's net package tries to load them.

I added ntohs and ntohl in socket.c but i'm not sure about that. I tried to keep them with "network stuff", but maybe they should go into separate file in platform directory?

[SeanTAllen]

I added ntohs and ntohl in socket.c but i'm not sure about that. I tried to keep them with "network stuff", but maybe they should go into separate file in platform directory?

socket.c seems fine to me.

[SeanTAllen]

@ahwayakchih i apologize for the fact that the build system is going to change out from underneath you soon. hopefully it won't be painful.

[ahwayakchih]

@ahwayakchih i apologize for the fact that the build system is going to change out from underneath you soon. hopefully it won't be painful.

@SeanTAllen oh well, i'll try to bring it back then :).
Right now i'm struggling a bit with process/* tests (like process/expect or process/stderr): they sometimes fail. I thought it's because of timeouts, but it's not. It looks like process does not get child process' output (so, child process exits without error, but parent does not get stdout and/or stderr content). What's more, it fails only when running all tests. When i run specific test alone it never fails.
As for the qemu setup, i'm trying to avoid building whole haiku (which allows to define what should be in the final image), but it looks like i'll have to write qemu script that will do things like "move mouse to x,y; click; wait X time..." to go through installation process.
There's a docker prepared by 3rd-party, but i'm trying to avoid adding dependencies.

[SeanTAllen]

@ahwayakchih that is odd with the process tests. they run in an exclusion group so they should be running sequentially no matter what. do they fail when you do --sequential to make sure only 1 test is running at a time?

[SeanTAllen]

@ahwayakchih oof. no headless way to install haiku?

[ahwayakchih]

@SeanTAllen

[...] do they fail when you do --sequential to make sure only 1 test is running at a time?

I did try running make test-stdlib-release directly (IIRC, not sure about exact name at the moment) with and without --sequential, also with filtering to run just process/* tests. It still failed. Although, which is even weirder, it looked like it failed less when running without --sequential, but that could be a fluke.

My wild guess is that maybe Haiku closes and cleans up everything before IO on pony's side handles reading output of child process. AFAIK, OS should not remove anything while something is still reading, but maybe there's a bug somewhere (most probably my code, or maybe something on Haiku's side).

oof. no headless way to install haiku?

Unfortunately no, not that i know of :(. There was experimental app_server over http, but that would require even more complications to set it up.
Good news is that most of the qemu monitor script could just use sendkey command. Maybe we'll not need mouse movement and fragile x,y coordinated after all :).

[ahwayakchih]

@SeanTAllen rpath is supported on Haiku, but software distributed for Haiku does not use it. For building and testing it might be ok, but then it's very inconvenient to remove or modify it later. Leaving build-time environment paths in distributed packages is not tidy.

When executables/libraries are packaged and distributed for Haiku, they should not depend on totally custom paths. They should depend on libraries placed in one of the default places (currently there are 4 main, 2 additional for dev/compiler stuff, + "lib" subdirectory next to an executable, but the 6 default ones preferably should not be hardcoded - there's an API to get them, which i used in codegen files) - where Haiku looks for them by default.

I wouldn't say that this is an issue with any of the systems. It's more like an issue with where the generated *-additional libraries are stored while tests that depend on them are built and run. Unless there's something else that depends on some custom-path libraries, and i just did not happen to stumble upon it, and that something else is distributed later (but in that case, custom-path libraries should end up in one of the default paths anyway, or in "lib" subdirectory of distributed executables)? Why a few tests should force all generated executables to include rpaths that are very build-time environment specific (and will almost always be incorrect for any other run-time environment)?

In general, having somewhat standardized paths for libraries is easier for user to monitor and manage.
If software needs libraries, that are needed only for that software and nothing else will use them, it can put them in "lib" subdirectory. A simple symlink/subdir is more visible and easier to modify for the user.

[ahwayakchih]

@SeanTAllen i think i found the original issue you mentioned: #651. Looks like it was about custom builds, but solution impacts everyone. Maybe there could be an additional option to NOT to add rpath for all paths, e.g., "--no-rpath"? I think it would be better if option was opt-in instead of opt-out, but that would break compatibility.

[SeanTAllen]

@ahwayakchih I'm not comfortable with haiku being different here. It will end up being surprising to pony users and maintainers.

[ahwayakchih]

@SeanTAllen ok then, i'll change it back so rpath is generated on Haiku too (even though it may be surprising for Haiku users), but then... do you want to keep rpath in distributed executables too (i mean, after i'll add script to create ponyc.pkg for Haiku)? it does not make sense on Haiku, no one will have any libraries in /Data/ponyc/[whatever] (path used in provisioned Haiku VM for building and testing ponyc).

[ahwayakchih]

I've modified builder repo and now it builds Pony both for Haiku nightly and recently released r1beta6 version. It also publishes packages, so if you'd like to test how pony works on r1beta6, you don't have to even build it, just download package ready to install:
https://github.com/ahwayakchih/haiku-build-ponyc/releases

It builds Pony with system installed LLVM instead of statically linked one. I did not include changes to use shared LLVM in this PR, because i suspect you will not want it. But just in case you'd consider it, here's a diff:
ahwayakchih@9c945f1
(it enables option for Linux too, but has location hardcoded, because that was just for testing. it would need more changes to make it real for Linux and other OS-es).

[SeanTAllen]

@ahwayakchih we dont support using system llvm and won't be doing that. back when we did, chasing down llvm bugs and weird interactions was almost all of our time. its definitely not something we are going to do. so yes, don't want. it was just a massive massive timesink.

[SeanTAllen]

@ahwayakchih let me know if/when you are ready for CI to kick off for anything.

[ahwayakchih]

@SeanTAllen regarding shared LLVM: i suspected that, since you pin specific version in repo.

Thing is, i have similar reasons to avoid building LLVM: it takes very long time and sometimes just stops hanging forever and never finishing (that most probably is a problem with Haiku, not LLVM, but from my point of view result is the same). Locally, i can kill the process, start build again, and sometimes it works in the end, other times i end up with non-working/crashing binaries and have to repeat whole process again. In CI i have to cancel workflow (if i notice this, or wait until GitHub fails it automatically after a very long time) and restart from zero again.
I thought about writing some wrapper scripts, or heartbeat monitoring that would restart build... but then i thought that fighting with LLVM builds is not what interests me in porting Pony to Haiku, and not what i want to spend my free time on.
I checked that Pony's patches to LLVM do not seem to be needed at all on Haiku, implemented option to use system-installed LLVM and worked with that from then on. Building from zero is a lot faster, (re-)linking ponyc after every small change is a lot faster, even running ponyc (as in: compiling pony examples) seems to be faster (my guess is that it's because shared lib is already cached by Haiku, and loading ~14 MB executable is a lot faster than loading over 100 MB executable, but that may be just my imagination).

I re-enabled building with static LLVM in my repository today and will see what will happen this time. There were two changes in Haiku that might affect it: one regarding io handling, which i did test on nightly and it did not seem to help then, and recently there was a fix for timer handling that mentioned haiku crashing after ~5 hours (which kinda fits to what was happening in CI, although it looks more like freezing then to me) that i did not check yet. Both are in r1beta6 AFAIK, so we'll see... but as of this moment it looks like nightly build is already hanging doing nothing at 27%, while r1beta6 build is at 59% and looks like it's still alive and moving on.

LLVM available in HaikuPorts (installable from HaikuDepot applicaiton provided by Haiku) is built with some patches (https://github.com/haikuports/haikuports/blob/f40c16e59aa091c55ad427b54b75f87704c2fea8/sys-devel/llvm/patches/llvm22-22.1.8.patchset) but it looks like they're only for default system paths (which do not matter much, since Pony is instructing LLVM anyway) and adding 32 vs 64 bit build handling (which we do not care about AFAIK, requiring 64 bit anyway).

It worked much better a few months ago, so it's something with Haiku changing, or maybe updated LLVM... or both. Anyway, if it keeps happening, this port will have to be dropped :(.
Or i'll try to keep it up-to-date, but keep building with shared LLVM in my repo until Haiku is fixed (or drop it after Pony starts requiring some additional patches to LLVM, thus requiring building custom version of LLVM).

[ahwayakchih]

@ahwayakchih let me know if/when you are ready for CI to kick off for anything.

Thanks, i'll let you know, but i think LLVM builds have to start working again first :(.

[ahwayakchih]

So LLVM build for r1beta6 got stuck at [ 78%] Building CXX object llvm/src/llvm/lib/Transforms/IPO/CMakeFiles/LLVMipo.dir/BarrierNoopPass.cpp.o.
LLVM for nightly got stuck at [ 27%] Building CXX object llvm/src/llvm/lib/MC/CMakeFiles/LLVMMC.dir/StringTableBuilder.cpp.o.
But the file and percent are not meaningful. It "forgets to work" at various, seemingly random points.
Looks like there might be couple of separate issues:

• Haiku indeed kept crashing after ~5 hours (actually it looks like freezing locally, i never got Kernel Debug Land screen). Builds are at over 5 hours 19 minutes now, but who knows what is happening there... they got stuck way earlier than that (nightly got stuck at 27% sometime around 1 hour 30 minutes IIRC after less than 30 minutes).
• Not sure about CI, but locally it was looking like if build process (and all subprocesses, like shell, compiler, etc...) just stopped working, but was not ended. Maybe deadlock of some kind, i don't know. It did not use CPU almost at all, just nothing was happening and build was not getting stopped/cancelled. Probably something else with Haiku, or cmake, or gcc, or whatever else can break like that without error/crash.

So i guess this PR is on hold, or maybe should be closed and re-opened at some point in future, when LLVM can be built in one go and without issues, like it used to :(.

[SeanTAllen]

@ahwayakchih i dont think we need to close. do you want to see what happens with building LLVM here in CI? Probably the same thing, but... maybe not.

[ahwayakchih]

@SeanTAllen well, sure we can try, but i wanted to spare you spending time on this :). Especially when i did not actually run workflows prepared for Pony repository. I tried to keep it close to what i used in my "builder" repo, but never actually run it, so there might be errors in there that will fail before it even gets to building anything ;).

[ahwayakchih]

I noticed that pr workflow does not include BSDs, so i did not add Haiku there. Which means that PR workflow could be run for this. AFAIK it would check if my changes did not break anything for main platforms, and lint anything i did change.

[SeanTAllen]

@ahwayakchih yes. Tier 3 gets run a few times a week and can be manually kicked off as well when needed for given PRs

[ahwayakchih]

Some good news: deadlocking problem got fixed in Haiku nightly hrev60080. Static LLVM builds seems fine there now. Unfortunately i don't know when this will get backported to recently released r1beta6.
Tried workaround for r1beta6: setting -DJOBS=1 for building static LLVM, and it seems to work (so far almost 3 finished builds, 3rd one is at 77% now). Bad news is that after LLVM is built, whole thing can deadlock soon after when building pony's stuff (happened once), or when test is run (every time it got to that point, including runs when LLVM libs were already built and downloaded from cache).
Weird that it does not deadlock that often when using shared LLVM on r1beta6 - test should work the same in both cases, but maybe because of large binaries IO access patterns change.

I want to make sure builds are ok for nightly today, and if all goes well, i'll switch workflows in this PR back to using nightly (and then i guess we could try running CI for the PR?).

[SeanTAllen]

@ahwayakchih that's awesome news. let me know when you are ready to "try turning it on".

[ponylang-main]

Hi @ahwayakchih,

The changelog - added label was added to this pull request; all PRs with a changelog label need to have release notes included as part of the PR. If you haven't added release notes already, please do.

Release notes are added by creating a uniquely named file in the .release-notes directory. We suggest you call the file 5358.md to match the number of this pull request.

The basic format of the release notes (using markdown) should be:

## Title

End user description of changes, why it's important,
problems it solves etc.

If a breaking change, make sure to include 1 or more
examples what code would look like prior to this change
and how to update it to work after this change.

Thanks.

[ahwayakchih]

@SeanTAllen building LLVM works on Haiku nightly, so i switched workflows back to it. I guess you'll prefer to target r1beta6 (or next one) in future, since that's more like a "LTS" release of Haiku, but for now we have to target nightly.
There's still one problem tough: it may timeout. In my "builder" repo, custom workflow timed out by juuuust a minute or two, but i do create package for end-users there (it timed out while installing to temp directory, after all builds and tests were OK). Here we don't have that (yet? i did not add it, because Haiku is to be a "testing only" tier), so with a bit of luck it will make it to the finish line ;).

Building and testing is in a separate workflow file for now: ponyc-haiku-wip.yml and has to be triggered manually.
Building just LLVM libs is added to update-lib-cache.yml workflow (i hope i did not break anything there).

[ahwayakchih]

0.71.0 built ok both in static and shared versions for Haiku nightly:
https://github.com/ahwayakchih/haiku-build-ponyc/releases/tag/v0.71.0-1

It also got through build and test process for Haiku r1beta in shared version. Static version keeps timing out now (so i was very lucky before, when LLVM got built with -DJOBS=1 before).