[MODFQMMGR-1034] Handle entity type inheritance when migrating queries, custom entities by ncovercash · Pull Request #1144 · folio-org/mod-fqm-manager

ncovercash · 2026-01-28T18:55:43Z

Purpose

Custom entity types need love, too! (And this explicitly breaks the assumption used for earlier migrations that we'd only need to define changes in terms of the public, composite entities).

Approach

Migration logic now takes a Map<UUID, Map<String, UUID>> of source mappings (basically et.getSources().filter(is entity source).toMap(alias, targetId)) which will be used to iteratively attempt migration on each level of nesting:
- For example, an entity with a field like wrapping_entity.instance.id would result in migration being attempted on each of:
  - entityTypeId=base, fieldPrefix="", field=wrapping_entity.instance.id;
  - entityTypeId=wrapping-id, fieldPrefix="wrapping_entity.", field=instance.id; and
  - entityTypeId=simple-instance-id, fieldPrefix="wrapping_entity.instance.", field=id.
- This unwrapping will continue until either something was done (field/op/value change, warning emitted, etc) or there is nothing left to unwrap;
- This allows migration logic to only worry about the simplest entityTypeId and field themselves without worrying about parent structure.
These source mappings are defined as part of the migration (as, if we're defining that composite_instances has source instance pointing to simple_instances, there is no guarantee that this source will later have the same target, alias, or even exist altogether;
For custom entity types, these source mappings are generated based on their current state at the time of migration.
- This does present the drawback that changes to custom entity types which modify the source entity's ID and/or alias` (either due to the user or FQM changes) will lose their ability to have old queries migrated, however, the cases of FQM causing this should be very limited;
  - IMO this is a reasonable drawback and far below the "best effort" I initially thought we'd have
- These mappings are stored in the DB for optimization purposes (we don't want to recompute this for every migration) using a dedicated table with a single value (there's no need to use DB indexes/etc when we just grab the whole blob)
Custom entity types will be themselves migrated on tenant install, see the MD doc file for the most details about that.

Why don't we need to update old migrations to support composites like this?

Custom entity types will debut in Trillium, so there's no chance (outside of dev envs) that custom entities would need to be migrated (and we've been providing simples on a "use at your own risk" philosophy).

V23UserCreatedUpdatedDateFieldDeprecation defines it just for testing/example purposes.

…migration

…er into modfqmmgr-1034p2

ncovercash · 2026-01-30T14:40:50Z

  @CheckForNull String fqlQuery,
  List<String> fields,
  @Singular List<Warning> warnings,
-  String version,


this was never used?

Not that I know of

ncovercash · 2026-01-30T14:41:36Z

-    BiPredicate<F, SingleFieldMigrationResult<F>> didModify
+    BiPredicate<F, SingleFieldMigrationResult<F>> didModify,
+    Map<UUID, Map<String, UUID>> sourceMappings
  ) {
    SingleFieldMigrationResult<F> transformed = handler.apply(original);
+    int fieldDelimiterIndex = original.field().indexOf('.');

-    // stub for follow-up ticket
-    didModify.test(original, transformed);
+    if (didModify.test(original, transformed) || fieldDelimiterIndex == -1) {
+      return transformed;
+    }

-    return transformed;
+    Map<String, UUID> sourceMap = sourceMappings.get(original.entityTypeId());
+    String source = original.field().substring(0, fieldDelimiterIndex);
+    String remainder = original.field().substring(fieldDelimiterIndex + 1);
+    if (sourceMap == null || sourceMap.get(source) == null) {
+      return transformed;
+    }
+
+    return handleSingleFieldWithNesting(
+      original.dereferenced(sourceMap.get(source), source, remainder),
+      handler,
+      didModify,
+      sourceMappings
+    );


glad I did all the migration refactoring in that previous PR — this is nearly all the logic necessary for the actual composite traversal

ncovercash · 2026-01-30T14:54:49Z

+
+@Repository
+@Log4j2
+public class CustomEntityTypeMigrationMappingRepository {


Basically a key/value store with a single row and the only key mapping, mirrors the idea of mod-lists's latest_migrated_version. We might update this later with more stuff, but for now it's just this.

ncovercash · 2026-01-30T14:56:17Z

+    /**
+     * We must traverse the custom entity types from the outermost-in, to ensure source
+     * references work as we traverse. If A -> B -> simple_instance, both A and B may refer
+     * to fields inside simple_instance. However, if simple_instance changes (e.g. entity ID change)
+     * and B is migrated before A, B's migration will change how simple_instance is referred to,
+     * breaking the ability for A's migration to "know" that simple_instance is being referred to.
+     *
+     * By doing this from the outermost entities inwards, we can ensure that earlier migrations will
+     * not break future ones.
+     *
+     * Traversal as a DFS is a bit easier, though, so we do that to build a queue inside-out;
+     * since we know children appear before parents in this queue, we can also know parents will
+     * appear after children. Flip it et voilà, we've got a parent-first queue without too much complexity!
+     */


This traversal is a little counter-intuitive at first, but handles every case I could think of

ncovercash · 2026-01-30T14:56:50Z

+        migrateEntitySources(et, strategy, updatedMappings, warnings);
+        migrateEntityGroupByFields(et, strategy, updatedMappings, warnings);
+        migrateEntityDefaultSort(et, strategy, updatedMappings, warnings);


All of these methods are incredibly granular, however, it makes testing easier so 🤷

ncovercash · 2026-01-30T14:57:57Z

+  static List<Arguments> migrationOrderTestCases() {
+    // ensure we try every possible case here (only 24). for relationships between these,
+    // see comments in consuming test.
+    Iterator<List<String>> permutations = new PermutationIterator<>(
+      List.of(
+        "aaaaaaaa-fb77-5995-8ead-8b1efd81fd10",
+        "bbbbbbbb-7b59-522f-9b85-7208291def17",
+        "cccccccc-56d1-573d-9639-3b006ed8f953",
+        "dddddddd-5523-5c2b-8372-3c3787ed9c23"
+      )
+    );
+
+    return StreamSupport
+      .stream(Spliterators.spliteratorUnknownSize(permutations, Spliterator.ORDERED), false)
+      .map(Arguments::of)
+      .toList();
+  }


laziest way to test a traversal 🤣

sonarqubecloud · 2026-01-30T15:55:26Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
98.6% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

ncovercash added 2 commits January 28, 2026 13:45

[MODFQMMGR-1034] Add custom entity type mapping collection

aa793fa

[MODFQMMGR-1034] Use source mappings to resolve inherited fields for …

b42508c

…migration

ncovercash changed the title ~~Modfqmmgr 1034p2~~ [MODFQMMGR-1034] Resolve composite entity type sources when migrating queries Jan 28, 2026

ncovercash changed the title ~~[MODFQMMGR-1034] Resolve composite entity type sources when migrating queries~~ [MODFQMMGR-1034] Handle entity type inheritance when migrating queries, custom fields Jan 28, 2026

ncovercash added 19 commits January 28, 2026 13:59

fix test

51ad039

fix test, the sequel

3781ba7

sonar nits

5ee7f70

Add custom entity type support

f11da3a

test fixes, updates

7635f46

fix testdb?

94b8ac2

fix test2

cdf6f3f

doh

112d143

fix test

bec32cc

utils coverage

38bc686

coverage bump

c24891f

add tests

7b4dd83

oop

b35b8cd

finish coverage

8080406

compiling helps

d91063e

fix field prefixing for source modification

8aba472

sonar nits

d2c6d8f

documentify

7e9d63c

Merge branch 'master' into modfqmmgr-1034p2

f2e94a5

ncovercash changed the title ~~[MODFQMMGR-1034] Handle entity type inheritance when migrating queries, custom fields~~ [MODFQMMGR-1034] Handle entity type inheritance when migrating queries, custom entities Jan 30, 2026

ncovercash added 2 commits January 30, 2026 09:31

more dock

ac04e39

Merge branch 'modfqmmgr-1034p2' of github.com:folio-org/mod-fqm-manag…

6ab725b

…er into modfqmmgr-1034p2

ncovercash commented Jan 30, 2026

View reviewed changes

ncovercash marked this pull request as ready for review January 30, 2026 14:58

ncovercash requested a review from a team as a code owner January 30, 2026 14:58

bvsharp approved these changes Jan 30, 2026

View reviewed changes

mweaver-ebsco reviewed Jan 30, 2026

View reviewed changes

Comment thread docs/Migration.md Outdated

mweaver-ebsco approved these changes Jan 30, 2026

View reviewed changes

ncovercash added 3 commits January 30, 2026 10:49

docblock

8107caf

nit

6ac9c34

Merge branch 'master' into modfqmmgr-1034p2

8e9d390

ncovercash enabled auto-merge (squash) January 30, 2026 15:51

ncovercash merged commit 6b50d17 into master Jan 30, 2026
15 checks passed

ncovercash deleted the modfqmmgr-1034p2 branch January 30, 2026 15:54

Navigation Menu

Uh oh!

[MODFQMMGR-1034] Handle entity type inheritance when migrating queries, custom entities - #1144

[MODFQMMGR-1034] Handle entity type inheritance when migrating queries, custom entities#1144
ncovercash merged 26 commits into
masterfrom
modfqmmgr-1034p2

ncovercash commented Jan 28, 2026 •

edited

Loading

Uh oh!

ncovercash Jan 30, 2026

Uh oh!

mweaver-ebsco Jan 30, 2026

Uh oh!

ncovercash Jan 30, 2026

Uh oh!

ncovercash Jan 30, 2026

Uh oh!

ncovercash Jan 30, 2026

Uh oh!

ncovercash Jan 30, 2026

Uh oh!

ncovercash Jan 30, 2026

Uh oh!

Uh oh!

Uh oh!

sonarqubecloud Bot commented Jan 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

ncovercash commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Approach

Why don't we need to update old migrations to support composites like this?

Uh oh!

ncovercash Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

mweaver-ebsco Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

ncovercash Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

ncovercash Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

ncovercash Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

ncovercash Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

ncovercash Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sonarqubecloud Bot commented Jan 30, 2026

Quality Gate passed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ncovercash commented Jan 28, 2026 •

edited

Loading