[yamlcomposer] YAML Composer Add-on for Enhanced YAML Preprocessing by jimtng · Pull Request #20305 · openhab/openhab-addons

jimtng · 2026-02-28T11:24:54Z

Initially submitted as openhab/openhab-core#4818 but now refactored as an add-on.

YAML Composer supports extended YAML features that make openHAB configuration more modular, reusable, and maintainable. These features let you structure configuration as composable building blocks rather than large, repetitive files.

The add-on loads enhanced-syntax YAML files from OPENHAB_CONF/yamlcomposer/ and compiles them into fully resolved plain YAML written to OPENHAB_CONF/yaml/composed/.

See the README.md for a brief example. Complete docs are included in this PR.

Copilot

Pull request overview

Introduces a new YAML Composer IO add-on that preprocesses “enhanced” YAML from OPENHAB_CONF/yamlcomposer/ into plain YAML output under OPENHAB_CONF/yaml/composed/, including documentation and OSGi integration tests.

Changes:

Adds the org.openhab.io.yamlcomposer bundle (processing pipeline, placeholders/processors, watch service, logging aggregation).
Registers the add-on in build/reactor, Karaf feature metadata, i18n, and CODEOWNERS.
Adds a dedicated OSGi integration test module for the watch service behavior.

Reviewed changes

Copilot reviewed 64 out of 65 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tools/static-code-analysis/spotbugs/suppressions.xml	Suppresses a SpotBugs SLF4J format warning for the new BufferedLogger wrapper.
itests/pom.xml	Adds the YAML Composer integration-test module to the itests reactor.
itests/org.openhab.io.yamlcomposer.tests/pom.xml	New Maven module for YAML Composer OSGi integration tests.
itests/org.openhab.io.yamlcomposer.tests/itest.bndrun	New bndrun defining the OSGi test runtime composition for YAML Composer tests.
itests/org.openhab.io.yamlcomposer.tests/NOTICE	Notice file for the new itests module.
itests/org.openhab.io.yamlcomposer.tests/src/main/java/org/openhab/io/yamlcomposer/internal/YamlComposerWatchServiceOSGiTest.java	OSGi integration tests for watch-triggered compilation and delete behavior.
bundles/pom.xml	Adds the YAML Composer bundle to the add-ons bundles reactor.
bundles/org.openhab.io.yamlcomposer/pom.xml	New bundle POM including dependencies (SnakeYAML Engine, Jinjava, etc.).
bundles/org.openhab.io.yamlcomposer/src/main/feature/feature.xml	New Karaf feature definition for installing the YAML Composer bundle.
bundles/org.openhab.io.yamlcomposer/src/main/resources/OH-INF/addon/addon.xml	Add-on metadata registration (type/name/description/service-id).
bundles/org.openhab.io.yamlcomposer/src/main/resources/OH-INF/i18n/yamlcomposer.properties	Adds i18n name/description for the add-on.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/YamlComposerWatchService.java	WatchService listener to compile sources into composed YAML and track include dependencies.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/YamlComposer.java	Core load/compile pipeline: parse, variables/templates, includes/inserts, packages, overrides, output model.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/ComposerConfig.java	Centralizes config/source/output paths and composer constants/limits.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/ComposerUtils.java	YAML load/dump utilities, alias breaking, formatting, and compiled output writing.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/StringInterpolator.java	`${...}` interpolation and expression evaluation plumbing with configurable delimiters.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/ExpressionEvaluator.java	Jinjava-backed expression evaluation with custom filters and warning tracking.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/BufferedLogger.java	Warning aggregation wrapper that routes warnings through a LogSession.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/LogSession.java	Session-scoped warning consolidation and flush-on-close behavior.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/IncludeRegistry.java	Tracks main↔include relationships to support recompiles on include changes.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/ModelResolver.java	SnakeYAML scalar resolver customization (defer merge-key handling; avoid ENV tag collisions).
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/constructors/ModelConstructor.java	SnakeYAML constructor extension adding custom tags and substitution tracking.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/constructors/ConstructSub.java	Tracks `!sub` pattern suffix to select delimiter patterns.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/constructors/ConstructStr.java	Constructs STR scalars as either literal strings or substitution placeholders based on state.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/constructors/ConstructLiteral.java	Implements `!literal` handling to disable substitution within a subtree.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/constructors/ConstructInterpolablePlaceholder.java	Generic constructor for placeholders whose values are subject to interpolation/transformation.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/core/RecursiveTransformer.java	Recursive tree transformer applying placeholder processors, merge keys, and removal semantics.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/core/ProcessingPhase.java	Defines handler subsets for phased processing (substitution/includes/standard/overrides).
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/core/VariableLoader.java	Extracts variables and injects special variables into evaluation context.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/core/TemplateLoader.java	Extracts templates while deferring substitutions until insertion time.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/core/SourceLocator.java	Finds approximate key positions in YAML source for diagnostics.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/core/PackageProcessor.java	Merges “packages” into the main model, injecting `package_id` and supporting overrides.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/core/MergeKeyProcessor.java	Resolves deferred merge-key placeholders (composer-level merge handling).
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/core/RemovalSignal.java	Semantic signal for intentional removal distinct from null-on-error.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/processors/PlaceholderProcessor.java	Common interface for placeholder processing.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/processors/SubstitutionProcessor.java	Resolves substitution placeholders with optional custom delimiter patterns.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/processors/IfProcessor.java	Implements `!if` conditional branching in models.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/processors/IncludeProcessor.java	Implements `!include` with variable overlays and include caching.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/processors/InsertProcessor.java	Implements `!insert` template expansion with per-invocation variable overrides.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/processors/ReplaceProcessor.java	Implements `!replace` placeholder behavior.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/processors/RemoveProcessor.java	Implements `!remove` placeholder behavior.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/processors/FragmentUtils.java	Shared parameter parsing for `!include`/`!insert` (string and map forms).
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/placeholders/Placeholder.java	Base placeholder abstraction.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/placeholders/InterpolablePlaceholder.java	Placeholder interface supporting value recreation and eager/lazy argument processing.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/placeholders/SubstitutionPlaceholder.java	Record representing deferred string interpolation (`!sub`).
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/placeholders/IfPlaceholder.java	Record representing conditional nodes (`!if`).
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/placeholders/IncludePlaceholder.java	Record representing include nodes (`!include`).
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/placeholders/InsertPlaceholder.java	Record representing template insertion nodes (`!insert`).
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/placeholders/ReplacePlaceholder.java	Record representing override replacement nodes (`!replace`).
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/placeholders/RemovePlaceholder.java	Record representing override removal nodes (`!remove`).
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/placeholders/MergeKeyPlaceholder.java	Record representing deferred merge-key occurrences.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/expression/filters/LabelFilter.java	Adds a custom `label` filter for expression rendering.
bundles/org.openhab.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/expression/filters/DigFilter.java	Adds a custom `dig` filter for safe nested map/list access.
bundles/org.openhab.io.yamlcomposer/README.md	Adds README + examples and links to detailed docs.
bundles/org.openhab.io.yamlcomposer/doc/basics.md	Documentation: YAML basics used by the composer docs.
bundles/org.openhab.io.yamlcomposer/doc/variables.md	Documentation: variables, substitution, expressions, filters.
bundles/org.openhab.io.yamlcomposer/doc/include.md	Documentation: `!include` syntax, scoping, and path resolution.
bundles/org.openhab.io.yamlcomposer/doc/templates.md	Documentation: `!insert` templates and evaluation model.
bundles/org.openhab.io.yamlcomposer/doc/packages.md	Documentation: packages and merge behavior/overrides.
bundles/org.openhab.io.yamlcomposer/doc/merge-keys.md	Documentation: merge keys and integration with composer features.
bundles/org.openhab.io.yamlcomposer/doc/conditionals.md	Documentation: `!if` behavior, forms, and truthiness.
bundles/org.openhab.io.yamlcomposer/doc/anchors.md	Documentation: anchors/aliases usage and best practices.
bundles/org.openhab.io.yamlcomposer/NOTICE	Notice file for the new YAML Composer bundle.
CODEOWNERS	Adds code ownership for the new YAML Composer bundle.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

openhab-bot · 2026-04-18T12:53:17Z

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/yaml-file-based-pages-and-widgets/169079/19

Copilot

Pull request overview

Copilot reviewed 64 out of 65 changed files in this pull request and generated 7 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

jimtng · 2026-04-18T13:36:36Z

rebased to main to pick up markdownlint, then applied the changes that it required.

jimtng · 2026-04-18T14:25:04Z

the ci is so slow! That parallel build PR would've sped this up.

jimtng · 2026-04-19T06:35:00Z

All checks have passed!

lsiepel · 2026-04-28T08:28:32Z

Would be nice if @lolodomo can have a look at this as yaml expert ;-)

Copilot

Pull request overview

Copilot reviewed 64 out of 65 changed files in this pull request and generated 3 comments.

Signed-off-by: Jimmy Tanagra <jcode@tanagra.id.au>

Copilot

Pull request overview

Copilot reviewed 63 out of 64 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 63 out of 64 changed files in this pull request and generated no new comments.

jimtng · 2026-05-28T15:21:04Z

@lsiepel, copilot had run out of suggestions now

Signed-off-by: Jimmy Tanagra <jcode@tanagra.id.au>

Copilot

Pull request overview

Copilot reviewed 63 out of 64 changed files in this pull request and generated 1 comment.

Signed-off-by: Jimmy Tanagra <jcode@tanagra.id.au>

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Signed-off-by: Jimmy Tanagra <jcode@tanagra.id.au>

lsiepel · 2026-06-06T10:05:28Z

Not sure if @lolodomo is available, but would be nice to see his review on this.

lolodomo · 2026-06-06T13:33:35Z

Sorry I will not be available for this very big PR in the next weeks. My priority is to finish the review of few PRs I already started.

Nadahar · 2026-06-06T20:06:19Z

Copilot has had many, many rounds on this code. It's quite complicated and comprehensive, so my guess is that doing a thorough review is a mammoth task. It might be that the "next step" is to have users actually try to use it, but it's a bit hard to do with the official add-ons because of the slow release cycle.

@jimtng Have you considered making it available on the community marketplace for "beta testing"? I think that would probably be the best way forward, because you could address issues rapidly, but alternatively, I think it must just be merged, and the feedback dealt with as it arrives.

jimtng · 2026-06-06T22:54:17Z

Even if we made this available on the marketplace first, we'll eventually end up right here again.

So I'd rather just have it merged now and fix any bugs as they come later. I've worked on this for more than a year.

lsiepel

Thanks, I have seen most of the code and looked up some details. This is not an easy review, but from what I have seen it gives a lot of confidence it is weel designed and written.
I can spend another day on reviewing, unlikely to find bugs that will surface quickly when used.

LGTM

…penhab#20305) * [yamlcomposer] New Yaml Composer add-on Signed-off-by: Jimmy Tanagra <jcode@tanagra.id.au>

jimtng · 2026-06-22T10:35:41Z

This is missing the 5.2 tag - didn't make it into the release notes

Nadahar · 2026-06-22T11:01:55Z

This is missing the 5.2 tag - didn't make it into the release notes

That can't be too late to rectify?

jimtng · 2026-06-22T11:08:26Z

I don't know how it works, but maybe it can still show up in the final release notes?

Nadahar · 2026-06-22T11:11:55Z

I don't know how it works, but maybe it can still show up in the final release notes?

I don't know for sure either, but I was able to add the milestone...

lsiepel · 2026-06-22T16:32:34Z

It will be in the final notes, sorry I did not add it.

…penhab#20305) * [yamlcomposer] New Yaml Composer add-on Signed-off-by: Jimmy Tanagra <jcode@tanagra.id.au>

…penhab#20305) * [yamlcomposer] New Yaml Composer add-on Signed-off-by: Jimmy Tanagra <jcode@tanagra.id.au> Signed-off-by: olemr <olemr@olemr.com>

jimtng added the new io label Feb 28, 2026

jimtng force-pushed the yaml-preprocessor branch 5 times, most recently from 4f4bab4 to 824a362 Compare March 6, 2026 13:21

jimtng marked this pull request as ready for review March 6, 2026 13:28

jimtng requested a review from a team as a code owner March 6, 2026 13:28

lsiepel requested a review from Copilot March 6, 2026 14:09

Copilot started reviewing on behalf of lsiepel March 6, 2026 14:13 View session

Copilot AI reviewed Mar 6, 2026

View reviewed changes

Comment thread ...amlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/YamlComposerWatchService.java Outdated

jimtng changed the title ~~[yamlcomposer] Yaml Composer add-on~~ [yamlcomposer] YAML Composer Add-on for Enhanced YAML Preprocessing Mar 13, 2026

jimtng mentioned this pull request Apr 18, 2026

Add file-based YAML support for UI pages and widgets openhab/openhab-core#5493

Merged

lsiepel requested a review from Copilot April 18, 2026 13:02

Copilot started reviewing on behalf of lsiepel April 18, 2026 13:03 View session

Copilot AI reviewed Apr 18, 2026

View reviewed changes

jimtng force-pushed the yaml-preprocessor branch from 215619e to caa132f Compare April 18, 2026 13:35

jimtng force-pushed the yaml-preprocessor branch from 23974fb to e9b69ea Compare April 18, 2026 22:11

This was referenced Apr 24, 2026

Device Template openhab/openhab-core#3808

Closed

Support local packages in the enhanced YAML parser openhab/openhab-core#5240

Closed

lsiepel requested a review from lolodomo May 16, 2026 09:28

lsiepel requested a review from Copilot May 25, 2026 20:03

Copilot started reviewing on behalf of lsiepel May 25, 2026 20:03 View session

Copilot AI reviewed May 25, 2026

View reviewed changes

[yamlcomposer] New Yaml Composer add-on

66aae29

Signed-off-by: Jimmy Tanagra <jcode@tanagra.id.au>

Copilot AI reviewed May 28, 2026

View reviewed changes

Comment thread ...b.io.yamlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/StringInterpolator.java

jimtng requested a review from Copilot May 28, 2026 15:12

Copilot started reviewing on behalf of jimtng May 28, 2026 15:12 View session

Copilot AI reviewed May 28, 2026

View reviewed changes

only write readme.txt when we had to create the composed dir

daade99

Signed-off-by: Jimmy Tanagra <jcode@tanagra.id.au>

jimtng requested a review from Copilot May 28, 2026 15:26

Copilot started reviewing on behalf of jimtng May 28, 2026 15:27 View session

Copilot AI reviewed May 28, 2026

View reviewed changes

Comment thread ...amlcomposer/src/main/java/org/openhab/io/yamlcomposer/internal/YamlComposerWatchService.java

spotless

51a03e1

Signed-off-by: Jimmy Tanagra <jcode@tanagra.id.au>

jimtng requested a review from Copilot May 28, 2026 23:30

Copilot AI reviewed May 28, 2026

change source/output root for clean up

a1be4ed

Signed-off-by: Jimmy Tanagra <jcode@tanagra.id.au>

lsiepel approved these changes Jun 10, 2026

View reviewed changes

lsiepel merged commit 0a26a04 into openhab:main Jun 10, 2026
2 checks passed

jimtng deleted the yaml-preprocessor branch June 10, 2026 23:45

Nadahar added this to the 5.2 milestone Jun 22, 2026

jimtng mentioned this pull request Jul 4, 2026

openHAB 5.2 release blog post openhab/openhab-website#564

Merged

darkscout pushed a commit to darkscout/openhab-addons that referenced this pull request Jul 5, 2026

[yamlcomposer] YAML Composer Add-on for Enhanced YAML Preprocessing (o…

b300f31

…penhab#20305) * [yamlcomposer] New Yaml Composer add-on Signed-off-by: Jimmy Tanagra <jcode@tanagra.id.au>

Uh oh!

Uh oh!

Conversation

jimtng commented Feb 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

openhab-bot commented Apr 18, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jimtng commented Apr 18, 2026

Uh oh!

jimtng commented Apr 18, 2026

Uh oh!

jimtng commented Apr 19, 2026

Uh oh!

lsiepel commented Apr 28, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

jimtng commented May 28, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

lsiepel commented Jun 6, 2026

Uh oh!

lolodomo commented Jun 6, 2026

Uh oh!

Nadahar commented Jun 6, 2026

Uh oh!

jimtng commented Jun 6, 2026

Uh oh!

lsiepel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jimtng commented Jun 22, 2026

Uh oh!

Nadahar commented Jun 22, 2026

Uh oh!

jimtng commented Jun 22, 2026

Uh oh!

Nadahar commented Jun 22, 2026

Uh oh!

lsiepel commented Jun 22, 2026

jimtng commented Feb 28, 2026 •

edited

Loading