Adding support for caching inChangesetReader api

common/api/core-backend.api.md

@@ -1030,10 +1030,10 @@ export class ChangesetReader implements Disposable, ChangeSource {
     clearTableNameFilters(): void;
     close(): void;
     readonly db: AnyDb;
-    deleted?: ChangeInstance;
+    get deleted(): ChangeInstance | undefined;
     disableStrictMode(): void;
     enableStrictMode(): void;
-    inserted?: ChangeInstance;
+    get inserted(): ChangeInstance | undefined;
     get isECTable(): boolean;
     get isIndirectChange(): boolean;
     get op(): SqliteChangeOp;
@@ -1058,6 +1058,7 @@ export class ChangesetReader implements Disposable, ChangeSource {
         txnId: Id64String;
         spillThresholdInBytes?: number;
     }): ChangesetReader;
+    setBatchSize(batchSize: number): void;
     setClassNameFilters(classNames: Set<string>): void;
     setOpCodeFilters(ops: Set<SqliteChangeOp>): void;
     setTableNameFilters(tableNames: Set<string>): void;

common/changes/@itwin/core-backend/soham-adding-batching-in-changesetreader_2026-06-24-10-29.json

@@ -0,0 +1,10 @@
+{
+  "changes": [
+    {
+      "packageName": "@itwin/core-backend",
+      "comment": "Added caching behaviour to ChangesetReader api",
+      "type": "none"
+    }
+  ],
+  "packageName": "@itwin/core-backend"
+}

docs/changehistory/NextVersion.md

@@ -16,6 +16,8 @@ publish: false
     - [Graphics no longer disappear when a new category is inserted](#graphics-no-longer-disappear-when-a-new-category-is-inserted)
   - [@itwin/core-geometry](#itwincore-geometry)
     - [`CurveFactory.createFilletsInLineString` expanded options](#curve-factory-create-fillets-in-line-string-expanded-options)
+  - [@itwin/core-backend](#itwincore-backend)
+    - [ChangesetReader.setBatchSize](#changesetreadersetbatchsize)
   - [@itwin/map-layers-formats](#itwinmap-layers-formats)
     - [Azure Maps basemap support is available through map-layers-formats](#azure-maps-basemap-support-is-available-through-map-layers-formats)
   - [@itwin/build-tools](#itwinbuild-tools)
@@ -205,6 +207,36 @@ The cache now keeps serving the previously-loaded data and instead marks the aff
 
 [CreateFilletsInLineStringOptions.cuspTolerance]($core-geometry) is used when [CreateFilletsInLineStringOptions.allowCusp]($core-geometry) is `true` to determine whether to suppress large cusps in the output. A cusp segment whose length exceeds `cuspTolerance` will be eliminated in the output `Path` by the removal of one or both of its constituent fillet arcs. The default value of this option is [Geometry.smallMetricDistance]($core-geometry), which is a slight deviation from previous default behavior. The new default behavior allows only miniscule cusps, whereas the old default behavior allowed cusps of any size. The old default behavior is considered to be a bug.
 
+## @itwin/core-backend
+
+### ChangesetReader.setBatchSize
+
+[ChangesetReader]($backend) now exposes a `setBatchSize(n: number)` method that controls how many change rows are cached in the reader. It is a performance improvement parameter that can be tweaked as per user's choice. Increasing the batch size increases the number of rows read at once and cached in the reader, thereby improving throughput when iterating large changesets but it also increases memory consumption; decreasing it reduces peak memory use. The method must be called before the first [ChangesetReader.step]($backend) call.
+
+Default batch sizes (unchanged behaviour when `setBatchSize` is not called):
+
+| Active configuration | Default |
+|---|---|
+| `propFilter: InstanceKey` | 100 |
+| `propFilter: BisCoreElement` | 20 |
+| `propFilter: All`, `abbreviateBlobs: false` | 5 |
+| `propFilter: All` (blobs abbreviated or unset) | 10 |
+
+```ts
+using reader = ChangesetReader.openFile({ db, fileName: changeset.pathname });
+reader.setBatchSize(10);
+while (reader.step()) { /* ... */ }
+```
+
+**Performance improvement with new caching behaviour in ChangesetReader`**:
+
+| Cache type | Inserts | Before (s) | After (s) | Improvement |
+|---|---|---|---|---|
+| InMemoryCache | 1,000 | 0.220 | 0.204 | 7.3% |
+| InMemoryCache | 10,000 | 2.213 | 1.402 | 36.6% |
+| SqliteBackedCache | 1,000 | 0.399 | 0.207 | 48.1% |
+| SqliteBackedCache | 10,000 | 3.342 | 1.981 | 40.7% |
+
 ## @itwin/map-layers-formats
 
 ### Azure Maps basemap support is available through map-layers-formats

docs/learning/backend/ChangesetReader.md

@@ -16,7 +16,7 @@ A single EC entity may typically map to multiple tables or a single table.
 [[include:ChangesetReader.BasicPipeline]]
 ```
 
-After draining the reader, `pcu.instances` yields one entry per (ECInstanceId + stage) pair, with properties merged across all contributing tables.
+After draining the reader, `pcu.instances` yields one entry per (ECInstanceId + stage) pair, with properties merged across all contributing tables. Use `pcu.instanceCount` to check how many merged instances were accumulated before iterating.
 
 ### [ChangeInstance]($backend) shape
 
@@ -121,6 +121,14 @@ OR order them appropriately if you are sure only the last disposal might throw:
 [[include:ChangesetReader.BasicPipeline]]
 ```
 
+### `invert` — reading a changeset in reverse
+
+All `open*` methods accept an optional `invert: true` argument. When set, every operation is flipped: Inserts become Deletes, Deletes become Inserts, and for Updates the `"New"` and `"Old"` stages are swapped. This is useful when you need to *undo* the effect of a changeset — for example, rolling back to a previous state for auditing.
+
+```ts
+[[include:ChangesetReader.InvertChangeset]]
+```
+
 ### [ChangesetReader.openGroup]($backend) — read multiple changesets as a single stream
 
 [ChangesetReader.openGroup]($backend) concatenates multiple changeset files into one logical stream. The unifier merges them across the whole group — an element that was inserted in changeset 1 and updated in changeset 2 surfaces as a single `"Inserted"` `"New"` instance reflecting its final state.
@@ -288,12 +296,14 @@ Each setter accepts a `Set<>`. Passing an empty `Set` is equivalent to calling t
 [[include:ChangesetReader.FilterClassNames]]
 ```
 
-### Clearing filters at runtime
+### Clearing filters
 
-All three filters can be cleared individually without reopening the reader:
+All three filters can be cleared individually. Like the setters, the clear methods must be called **before** the first successful [ChangesetReader.step]($backend) call:
 
 ```ts
-reader.clearTableNameFilters();
+// All filter configuration (set or clear) must happen before step() returns true.
+reader.setTableNameFilters(new Set(["bis_Element"]));
+reader.clearTableNameFilters(); // removes the table filter again
 reader.clearOpCodeFilters();
 reader.clearClassNameFilters();
 ```
@@ -332,7 +342,28 @@ To return to the default lenient behaviour at any time:
 reader.disableStrictMode();
 ```
 
-Both methods can be called between [ChangesetReader.step]($backend) calls to toggle the mode mid-stream.
+> **Important:** Both `enableStrictMode` and `disableStrictMode` must be called **before** the first successful [ChangesetReader.step]($backend) call. Calling either method after iteration has begun will throw an `IModelError`.
+
+---
+
+## Batch size — tuning native fetch performance
+
+[ChangesetReader]($backend) exposes a `setBatchSize(n: number)` method that controls how many change rows are fetched and cached per native call. Increasing the batch size improves throughput when iterating large changesets at the cost of higher peak memory usage; decreasing it keeps memory consumption lower. Like filters and strict mode, `setBatchSize` must be called **before** the first [ChangesetReader.step]($backend) call.
+
+| Active configuration | Default |
+|---|---|
+| `propFilter: InstanceKey` | 100 |
+| `propFilter: BisCoreElement` | 20 |
+| `propFilter: All`, `abbreviateBlobs: false` | 5 |
+| `propFilter: All` (blobs abbreviated or unset) | 10 |
+
+Call `setBatchSize` before the first [ChangesetReader.step]($backend) call:
+
+```ts
+[[include:ChangesetReader.SetBatchSize]]
+```
+
+> **Note:** `setBatchSize` throws if called after the first successful `step()` call, or if the supplied value is not a positive integer.
 
 ---
 
@@ -361,15 +392,15 @@ The following example imports a custom schema, inserts an element, pushes a seco
 > **See also:** the test suite `"ChangesetReader: behaviour in case imodel is not in sync with change file or transaction being read"` in
 > `core/backend/src/test/standalone/ChangesetReader.test.ts`.
 
-[ChangesetReader]($backend) uses the **live iModel** in two ways: to resolve `ECClassId` in case it is not part of the changeset or transaction(very common in cases of `Update` because generally only element props are updated not the class of the instance), and to fill in the non-changed components of compound property values. For compound types — `Point2d`, `Point3d`, and navigation properties - when a changeset records a change to only one component, the reader must fetch the remaining components from the live iModel to reconstruct the full value. For example, if only `X` changes in a `Point2d` property, `Y` is read from the current live database state. This means the reader's output quality depends on the current state of the iModel — specifically whether the entity being read still exists in the database at the time of reading, and whether subsequent transactions have already modified the components that were not part of the recorded changeset delta.
+[ChangesetReader]($backend) uses the **live iModel** in two ways: to resolve `ECClassId` when it is absent from the changeset (this is typical for `Update` operations, where only changed properties are recorded, not the class identity), and to fill in the non-changed components of compound property values. For compound types — `Point2d`, `Point3d`, and navigation properties — when a changeset records a change to only one component, the reader fetches the remaining components from the live iModel to reconstruct the full value. For example, if only `X` changes in a `Point2d` property, `Y` is read from the current live database state. This means the reader's output quality depends on the current state of the iModel — specifically whether the entity still exists at the time of reading, and whether subsequent transactions have already modified the compound property components that were not part of the recorded delta.
 
 Two concrete failure modes arise:
 
 ### 1. Deleted instance — class identity lost
 
 **Scenario:** An element is inserted, updated, then deleted across three changesets. The update changeset (the "middle" one) is read **after** the element has already been deleted from the live iModel.
 
-**What happens:** As in the update changeset obviously the ECClassId was not updated for the instance, only some properties might have been updated so `ECClassId` was not part of the changeset. So when the reader resolves the `ECClassId` for a row, it performs a lookup in the live iModel's table. Because the element no longer exists, the native layer cannot determine which leaf domain class the row belongs to. It falls back to the per-table base class (`BisCore.Element` for `bis_Element`, `BisCore.GeometricElement2d` for `bis_GeometricElement2d`). The per-table instances are **not merged** into a single `TestDomain.Test2dElement` instance; instead they appear as separate entries under their base-class identities.
+**What happens:** In an update changeset the `ECClassId` is typically not included in the change data — only the modified properties are recorded. When the reader resolves the `ECClassId` for a row, it performs a lookup in the live iModel's table. Because the element no longer exists, the native layer cannot determine which leaf domain class the row belongs to. It falls back to the per-table base class (`BisCore.Element` for `bis_Element`, `BisCore.GeometricElement2d` for `bis_GeometricElement2d`). The per-table instances are **not merged** into a single `TestDomain.Test2dElement` instance; instead they appear as separate entries under their base-class identities.
 
 ```ts
 // After push 2 (insert), push 3 (update), push 4 (delete):
@@ -390,7 +421,7 @@ using reader = ChangesetReader.openFile({
 //   { ECClassId: "BisCore.GeometricElement2d",   stage: "Old" }
 ```
 
-**Rule of thumb:** To read a changeset reliably, the iModel's current state should be **at** the change being read or the consumers of the api must be sure that, the instance was not deleted and for the compound properties of the instance like `Point2d`, `Point3d` or `NavProps`, no change was done to them subsequently after. In practice this means: read changesets in order and keep the iModel at the point being inspected.
+**Rule of thumb:** For reliable results, the iModel's state should match the point at which the change was recorded. In practice: read changesets in order and keep the iModel at the state being inspected.
 
 ### 2. Subsequent unsaved transaction pollutes property values
 
@@ -422,7 +453,7 @@ const newX = elementNew.s.X; // 100 — correct
 
 ### Summary
 
-It doesnot depend on whether a change group or a changeset or a transaction is opened. It might happen when the iModel's state is not in sync with the change being read. In other words it might happen when the iModel's state is not **at** the change being read.
+This risk is not specific to changesets — it arises any time the iModel's state is not synchronized with the change being read.
 
 | Scenario | Risk | Mitigation |
 |---|---|---|

example-code/snippets/src/backend/ChangesetReader-examples.test.ts

@@ -122,6 +122,9 @@ describe("ChangesetReader Examples", () => {
       pcu.appendFrom(reader);
     }
 
+    // pcu.instanceCount tells you how many merged instances were accumulated.
+    expect(pcu.instanceCount).to.be.greaterThan(0);
+
     for (const instance of pcu.instances) {
       expect(instance.ECInstanceId).to.exist;
       expect(instance.$meta.op).to.exist;
@@ -185,6 +188,30 @@ describe("ChangesetReader Examples", () => {
     // __PUBLISH_EXTRACT_END__
   });
 
+  it("invert — reading a changeset in reverse", () => {
+    // __PUBLISH_EXTRACT_START__ ChangesetReader.InvertChangeset
+    // Pass invert: true to flip every operation:
+    //   Inserts become Deletes, Deletes become Inserts,
+    //   and for Updates the "New" and "Old" stages are swapped.
+    // Useful for rolling back or auditing what a changeset *undid*.
+    using reader = ChangesetReader.openFile({
+      db,
+      fileName: insertChangesetPath,
+      invert: true,
+    });
+    using pcu = new PartialChangeUnifier(ChangeUnifierCache.createInMemoryCache());
+    while (reader.step()) pcu.appendFrom(reader);
+
+    for (const instance of pcu.instances) {
+      // The original changeset was an insert, so with invert:true op becomes "Deleted"
+      // and every instance appears in the "Old" stage.
+      if (instance.ECInstanceId !== elementId) continue;
+      expect(instance.$meta.op).to.equal("Deleted");
+      expect(instance.$meta.stage).to.equal("Old");
+    }
+    // __PUBLISH_EXTRACT_END__
+  });
+
   it("openGroup — multiple changesets as one stream", () => {
     // __PUBLISH_EXTRACT_START__ ChangesetReader.OpenGroup
     // openGroup merges insert + update into a single logical stream.
@@ -416,10 +443,26 @@ describe("ChangesetReader Examples", () => {
       expect(instance.$meta.op).to.exist;
       expect(instance.ECInstanceId).to.exist;
       expect(instance.ECClassId).to.exist;
+      // The active filter is recorded on every instance's $meta:
+      expect(instance.$meta.propFilter).to.equal(PropertyFilter.InstanceKey);
     }
     // __PUBLISH_EXTRACT_END__
   });
 
+  it("setBatchSize — tuning native fetch performance", () => {
+    // __PUBLISH_EXTRACT_START__ ChangesetReader.SetBatchSize
+    // setBatchSize controls how many rows are fetched and cached per native call.
+    // Must be called before the first step().
+    using reader = ChangesetReader.openFile({ db, fileName: insertChangesetPath });
+    reader.setBatchSize(10); // fetch 10 rows per native batch
+
+    using pcu = new PartialChangeUnifier(ChangeUnifierCache.createInMemoryCache());
+    while (reader.step()) pcu.appendFrom(reader);
+
+    expect(pcu.instanceCount).to.be.greaterThan(0);
+    // __PUBLISH_EXTRACT_END__
+  });
+
   it("SQLite-backed cache for large changesets", () => {
     // __PUBLISH_EXTRACT_START__ ChangesetReader.CacheStrategies
     using cache = ChangeUnifierCache.createSqliteBackedCache();