Add-ons sometimes fail to install because one of the bundles referenced by their Karaf features is not available from the openHAB JFrog Maven repository.
This is not necessarily caught by the build today because Maven can resolve the artifact directly from Maven Central, while an openHAB runtime intentionally only resolves such artifacts through the openHAB-controlled JFrog repository.
A recent example was org.jsoup:jsoup:1.23.1, discussed here:
https://community.openhab.org/t/org-jsoup-jsoup-1-23-1-missing-from-jfrog/170026
Rationale
There are good reasons for keeping the openHAB repository as the only remote Maven repository used by openHAB installations:
- An openHAB installation should not depend directly on an external service that is outside the control of the openHAB project.
- Maven Central should not effectively become a CDN for deployed openHAB installations. Sonatype explicitly states that Maven Central should not be in your runtime path and recommends putting a repository manager/cache between production systems and Maven Central.
- Having every openHAB installation download runtime dependencies directly from Maven Central would create unnecessary traffic, especially because dependencies may need to be downloaded again after an openHAB upgrade.
Adding Maven Central to org.ops4j.pax.url.mvn.repositories therefore works as a temporary user workaround, but should not be the actual solution.
JFrog proxy/cache
JFrog is already configured with a remote repository that proxies Maven Central and caches requested artifacts.
The official openhab-distro snapshot, milestone, and release builds can use this proxy configuration and have sufficient access to cause previously uncached artifacts to be fetched from Maven Central and added to the JFrog cache.
This gives us a controlled way to automatically populate JFrog with dependencies needed by the distribution instead of having to upload missing artifacts manually whenever an add-on dependency changes.
An artifact does not need to be explicitly deployed into the remote repository. When one of these distro builds requests an uncached artifact through the proxy, JFrog fetches it from Maven Central and stores it in the remote cache.
The cached artifacts can then be served through the JFrog repository used by openHAB installations, so production installations continue to depend only on openHAB infrastructure and never directly on Maven Central.
Keep the proxy scoped to official distro builds
The Maven Central proxy should deliberately only be used for the official openhab-distro snapshot, milestone, and release builds that are intended to populate the runtime dependency cache.
It should not be used by:
- pull request builds;
- normal
openhab-addons, openhab-core, or other repository builds;
- regular developer builds.
Those builds should continue resolving their normal build dependencies directly from Maven Central.
Otherwise the openHAB JFrog cache would effectively become a general Maven Central proxy and would gradually fill with every dependency, Maven plugin, test dependency, and transient artifact encountered by CI or development builds.
Restricting cache population to the official distro builds keeps the cache reasonably close to the dependency set actually needed to produce an openHAB distribution.
It will still not be a perfect representation of the production runtime dependency set. A distro build can resolve artifacts that are only needed while building, testing, or packaging the distribution, so the cache may become somewhat larger than strictly necessary for production installations.
That seems like an acceptable trade-off compared with either manually maintaining the repository or proxying Maven Central for all openHAB builds.
Proposed fix
The build and JFrog configuration should work together as follows:
- Continue using the JFrog Maven Central proxy/cache from the official
openhab-distro snapshot, milestone, and release builds.
- Allow those builds to populate the JFrog cache when previously unseen dependencies are encountered.
- Do not use the Maven Central proxy for PR builds, regular developer builds, or normal builds of the individual openHAB repositories.
- Make the cached artifacts required at runtime available through the JFrog repository used by openHAB installations.
- Verify the generated Karaf features against the same repository view available to a normal openHAB runtime, without Maven Central as a fallback.
The last step is important: after the distro build has had the opportunity to populate the cache, feature verification should fail if a runtime dependency still cannot be resolved from the repository that an actual openHAB installation uses.
This would make missing dependencies either automatically enter the controlled JFrog cache during an official distro build or fail that build, instead of first being discovered when a user tries to install an add-on.
Until a missing artifact is available from JFrog, users can install the offline openhab-addons KAR, which already contains the add-on dependencies.
Add-ons sometimes fail to install because one of the bundles referenced by their Karaf features is not available from the openHAB JFrog Maven repository.
This is not necessarily caught by the build today because Maven can resolve the artifact directly from Maven Central, while an openHAB runtime intentionally only resolves such artifacts through the openHAB-controlled JFrog repository.
A recent example was
org.jsoup:jsoup:1.23.1, discussed here:https://community.openhab.org/t/org-jsoup-jsoup-1-23-1-missing-from-jfrog/170026
Rationale
There are good reasons for keeping the openHAB repository as the only remote Maven repository used by openHAB installations:
Adding Maven Central to
org.ops4j.pax.url.mvn.repositoriestherefore works as a temporary user workaround, but should not be the actual solution.JFrog proxy/cache
JFrog is already configured with a remote repository that proxies Maven Central and caches requested artifacts.
The official
openhab-distrosnapshot, milestone, and release builds can use this proxy configuration and have sufficient access to cause previously uncached artifacts to be fetched from Maven Central and added to the JFrog cache.This gives us a controlled way to automatically populate JFrog with dependencies needed by the distribution instead of having to upload missing artifacts manually whenever an add-on dependency changes.
An artifact does not need to be explicitly deployed into the remote repository. When one of these distro builds requests an uncached artifact through the proxy, JFrog fetches it from Maven Central and stores it in the remote cache.
The cached artifacts can then be served through the JFrog repository used by openHAB installations, so production installations continue to depend only on openHAB infrastructure and never directly on Maven Central.
Keep the proxy scoped to official distro builds
The Maven Central proxy should deliberately only be used for the official
openhab-distrosnapshot, milestone, and release builds that are intended to populate the runtime dependency cache.It should not be used by:
openhab-addons,openhab-core, or other repository builds;Those builds should continue resolving their normal build dependencies directly from Maven Central.
Otherwise the openHAB JFrog cache would effectively become a general Maven Central proxy and would gradually fill with every dependency, Maven plugin, test dependency, and transient artifact encountered by CI or development builds.
Restricting cache population to the official distro builds keeps the cache reasonably close to the dependency set actually needed to produce an openHAB distribution.
It will still not be a perfect representation of the production runtime dependency set. A distro build can resolve artifacts that are only needed while building, testing, or packaging the distribution, so the cache may become somewhat larger than strictly necessary for production installations.
That seems like an acceptable trade-off compared with either manually maintaining the repository or proxying Maven Central for all openHAB builds.
Proposed fix
The build and JFrog configuration should work together as follows:
openhab-distrosnapshot, milestone, and release builds.The last step is important: after the distro build has had the opportunity to populate the cache, feature verification should fail if a runtime dependency still cannot be resolved from the repository that an actual openHAB installation uses.
This would make missing dependencies either automatically enter the controlled JFrog cache during an official distro build or fail that build, instead of first being discovered when a user tries to install an add-on.
Until a missing artifact is available from JFrog, users can install the offline
openhab-addonsKAR, which already contains the add-on dependencies.