LibreSSL is misidentified as OpenSSL 0.9.0, disabling functionality LibreSSL supports

[SeanTAllen]

LibreSSL users currently build with -Dopenssl_0.9.0, which forces LibreSSL through a code path designed for ancient OpenSSL. This silently disables functionality that LibreSSL actually supports (ALPN, modern init, modern EVP API) and runs unnecessary legacy code (the CRYPTO_set_locking_callback threading ceremony, which is a no-op in LibreSSL).

LibreSSL's actual API compatibility

LibreSSL's API is a hybrid — mostly OpenSSL 1.1.x compatible, with a few 0.9.x/1.0.x holdovers. It does not fit cleanly into any of the three existing ifdef paths (openssl_0.9.0, openssl_1.1.x, openssl_3.0.x).

Functions LibreSSL has (1.1.x compatible)

Function	Status
TLS_method()	Available (since LibreSSL 2.2.2)
OPENSSL_init_ssl()	Available (with pthread_once semantics)
EVP_MD_CTX_new() / EVP_MD_CTX_free()	Available (old names also still work)
SSL_CTX_set_alpn_select_cb()	Available
SSL_CTX_set_alpn_protos()	Available
SSL_get0_alpn_selected()	Available

Functions LibreSSL does NOT have

Function	LibreSSL alternative	Notes
OPENSSL_sk_pop() / OPENSSL_sk_free()	sk_pop() / sk_free()	Old naming convention only
SSL_get1_peer_certificate()	SSL_get_peer_certificate()	3.0.x rename not adopted
SSL_has_pending()	Not available	SSL_pending() exists but SSL_has_pending() (added in OpenSSL 1.1.0) is not
SSL_CTX_set_options() as function	Macro calling SSL_CTX_ctrl()	OpenSSL 1.1.x made this a real function; LibreSSL kept the macro

Threading

CRYPTO_set_locking_callback() exists in LibreSSL for API compatibility but is a no-op. LibreSSL handles threading internally using pthread mutexes. The current 0.9.0 code path calls @ponyint_ssl_multithreading and @CRYPTO_set_locking_callback unnecessarily.

Impact of the current misidentification

Because LibreSSL is forced through the openssl_0.9.0 path:

1. ALPN is disabled — the 0.9.0 path returns false from alpn_set_resolver() and alpn_set_client_protocols(), even though LibreSSL fully supports ALPN.
2. Legacy init ceremony runs — SSL_library_init(), SSL_load_error_strings(), CRYPTO_num_locks(), @ponyint_ssl_multithreading(), and CRYPTO_set_locking_callback() all run. LibreSSL doesn't need any of this; OPENSSL_init_ssl() works fine.
3. Legacy API names used — SSLv23_method(), EVP_MD_CTX_create()/EVP_MD_CTX_destroy(), SSL_CTX_ctrl() for options. LibreSSL supports the modern equivalents.
4. SSL_has_pending() not called — this one is actually correct since LibreSSL doesn't have it, but for the wrong reason (the 0.9.0 path skips it because 0.9.0 doesn't have it, not because LibreSSL doesn't).

Proposed fix

Add a new define (-Dlibressl or similar) that gives LibreSSL its own code path. This path should be based on the openssl_1.1.x path with targeted overrides for the four differences:

1. Use sk_pop()/sk_free() instead of OPENSSL_sk_pop()/OPENSSL_sk_free()
2. Use SSL_get_peer_certificate() instead of SSL_get1_peer_certificate()
3. Handle SSL_CTX_set_options() being a macro (this may already work transparently)
4. Skip SSL_has_pending() (use the SSL_pending() fallback)

Once LibreSSL has its own path, the actual OpenSSL 0.9.0 code path can be removed — OpenSSL 0.9.x has been EOL since 2015.

Downstream projects that have Makefiles with ssl= options (e.g., lori) will need to add the new define as a build option so users can build against LibreSSL properly.

[SeanTAllen]

Implementation Plan

This will be done as two separate PRs:

PR 1: Add -Dlibressl define

Change type: Changed (breaking — LibreSSL users must switch from ssl=0.9.0 to ssl=libressl)

Approach: Add "libressl" to existing ifdef conditions where LibreSSL matches the 1.1.x behavior. Only create separate branches at the three divergence points from 1.1.x:

1. sk_pop/sk_free — LibreSSL uses old names (no OPENSSL_ prefix)
2. SSL_CTX_set_options/SSL_CTX_clear_options — macros in LibreSSL, not real functions. Pony FFI can't call macros, so LibreSSL must use SSL_CTX_ctrl (same as the current 0.9.0 approach).
3. SSL_has_pending — not available in LibreSSL

File-by-file changes

Makefile — Add ssl=libressl option mapping to -Dlibressl, alongside the existing ssl=0.9.0 option (retained in PR 1, removed in PR 2).

make.ps1 — Change --define openssl_0.9.0 to --define libressl (line 70). Windows always builds against LibreSSL.

ssl/net/_ssl_init.pony — Add or "libressl" to the 1.1.x/3.0.x init condition (line 28). No FFI declaration changes needed — OPENSSL_init_ssl, OPENSSL_INIT_new, OPENSSL_INIT_free are already declared without version guards. Contingency: If CI reveals LibreSSL lacks OPENSSL_INIT_new/OPENSSL_INIT_free, give LibreSSL its own init branch that calls @OPENSSL_init_ssl(0, Pointer[_OpenSslInitSettings]) directly, skipping settings allocation/free.

ssl/net/ssl_context.pony:

• FFI declarations: Add or "libressl" to @TLS_method, @SSL_CTX_set_alpn_select_cb, @SSL_CTX_set_alpn_protos. Leave @SSL_CTX_set_options/@SSL_CTX_clear_options as 1.1.x/3.0.x only.
• create() (line 86): Add or "libressl" to the 1.1.x/3.0.x branch (uses TLS_method + set_min_proto_version/set_max_proto_version, which work via SSL_CTX_ctrl)
• _set_options/_clear_options: Add or "libressl" to the 0.9.0 branch (uses SSL_CTX_ctrl)
• alpn_set_resolver/alpn_set_client_protocols: Add or "libressl" to the 1.1.x/3.0.x branch
• Update ALPN method docstrings to note LibreSSL support (replacing "Requires OpenSSL >= 1.0.2")

ssl/net/ssl.pony:

• FFI declarations: Add or "libressl" to @SSL_get0_alpn_selected and @SSL_get_peer_certificate. Leave @SSL_has_pending and @SSL_get1_peer_certificate unchanged.
• alpn_selected(): Add or "libressl"
• read() SSL_has_pending check: No change (LibreSSL doesn't have it)
• _verify_hostname(): Add or "libressl" to the "openssl_1.1.x" or "openssl_0.9.0" branch (uses SSL_get_peer_certificate)

ssl/net/x509.pony:

• FFI declarations: Add or "libressl" to @sk_pop and @sk_free
• Three ifdef blocks in all_names(): Add or "libressl" to each 0.9.0 branch

ssl/crypto/digest.pony:

• FFI declarations: Add or "libressl" to @EVP_MD_CTX_new and @EVP_MD_CTX_free
• Seven hash constructors (md5 through sha512): Add or "libressl" to the EVP_MD_CTX_new branch
• shake128/shake256: Leave gated to 1.1.x/3.0.x only. Update compile_error messages to "shake128/256 is only supported with OpenSSL 1.1.x or 3.0.x" (removing stale 0.9.x reference)
• final() EVP_MD_CTX_free branch: Add or "libressl"

ssl/crypto/_test.pony: No changes needed. Shake test guard stays gated to 1.1.x/3.0.x.

CI workflows (pr.yml and breakage-against-ponyc-latest.yml):

• Change ssl=0.9.0 to ssl=libressl on all four LibreSSL test jobs

Follow-up issue

File an issue to verify and add SHAKE128/256 support for LibreSSL.

---

PR 2: Remove -Dopenssl_0.9.0 define

Change type: Changed (breaking — removes a build option)

After PR 1 lands, no users should be on the 0.9.0 path.

• Makefile — Remove ssl=0.9.0 option
• make.ps1 — No change needed (already updated in PR 1)
• _ssl_init.pony — Remove 0.9.0 FFI declarations and @ponyint_ssl_multithreading. Remove elseif "openssl_0.9.0" init branch.
• ssl_context.pony — Remove @SSLv23_method declaration. Remove elseif "openssl_0.9.0" from create(), alpn_set_resolver, alpn_set_client_protocols. In _set_options/_clear_options, change "openssl_0.9.0" or "libressl" to just "libressl".
• ssl.pony — Remove "openssl_0.9.0" from @SSL_get_peer_certificate guard and _verify_hostname branch.
• x509.pony — Remove "openssl_0.9.0" from @sk_pop/@sk_free guards and from ifdef blocks in all_names().
• digest.pony — Remove @EVP_MD_CTX_create/@EVP_MD_CTX_destroy declarations (this also resolves a pre-existing operator precedence bug where if not "openssl_1.1.x" or "openssl_3.0.x" parses as (not "openssl_1.1.x") or "openssl_3.0.x" rather than the intended exclusion). In hash constructors, the else branch becomes the compile_error catch-all. Similarly in final().
• CI workflows — No changes needed (already updated in PR 1).

[SeanTAllen]

we will need to handle different libressl abi versions. it isnt just "libressl".