Fixes #32374: serve the reindex entity list from the index registry (#32375)

* Fixes #32374: serve the reindex entity list from the index registry

The Search Indexing app's entity picker read a hardcoded `items.enum` in the
UI-local schema JSON, so it was a per-build snapshot of a runtime fact and had
drifted 15 entity types behind the index mapping registry. Selecting everything
in the dropdown was therefore not the same as selecting `all`, which the app
already expands from the registry.

It also could never be right for a distribution that registers extra indexes:
IndexMappingLoader merges elasticsearch/indexMapping.json plus
elasticsearch/collate/indexMapping.json from the classpath at startup, so no
enum baked into the UI bundle can express the reindexable set.

Add SearchRepository.getIndexedEntityTypes() as the one definition and have both
ReindexingOrchestrator.getAll() and a new GET /v1/search/entityTypes read it, so
the picker and `all` cannot drift. The UI drops the enum and fills it in
ApplicationsClassBase.importSchema, the single funnel AppDetails and AppInstall
share; a failed fetch leaves only "All" selectable plus an error toast rather
than breaking the whole form.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Address review: align authz with the page, fix the IT and the picker spec

manerow, on #32375:

- The endpoint was admin-or-bot while the page that calls it is not. SettingsRouter
  renders AppDetails for `isAdminUser || hasViewPermissions(APPLICATION)` and
  AppDetails fetches the schema on every mount, so a viewer holding ViewBasic or
  ViewAll got a red 403 toast for a request they never triggered. Authorize on
  APPLICATION/VIEW_BASIC instead, which is exactly the page's gate.

- The 403 assertion in SearchResourceIT would have been a 404: authorizeAdminOrBot
  resolves the subject first, and SubjectCache.getUserContext falls back to
  Entity.getEntityByName, which throws EntityNotFoundException for a user this
  suite never creates. It could only pass on suite ordering. Pin the user in
  @BeforeAll, as AppOperationPermissionsIT already does, and flip the assertion to
  the case that now matters: an Application viewer gets 200.

- SearchIndexApplication.spec.ts would have broken. rc-tree-select filters on the
  node value (treeNodeFilterProp defaults to 'value'), so typing "Table" leaves
  `tableColumn` visible too, and rc-tree sets title="Table Column" on it;
  getByTitle is substring by default, so the locator resolved to 2 nodes and
  waitFor threw in strict mode. Made it exact.

  The Playwright run on the first commit was green because the impact map never
  selected this spec — the Settings/** entry picks no spec that opens an app config
  form. Added a mapping for the application config forms so a change to the
  schemas, the loader, or the endpoint that fills them selects it. That also puts
  this PR in a full run, so the locator fix gets exercised.

Copilot:

- 'all' is now in the injected enum. The backend sentinel is not an index, so the
  endpoint does not return it, but `default: ["all"]` has to validate against the
  enum; TreeSelectWidget already filters it out of the child nodes. Pre-existing
  inconsistency — the old hardcoded enum omitted it too — fixed while here. It
  also makes the fetch-failure fallback `['all']` rather than empty.

- Reused SEARCH_INDEXING_APPLICATION from constants/explore.constants.ts instead of
  a second copy. Its own `SortingField` import is type-only, so this pulls nothing
  new into the bundle graph.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Fix the payLoadSize description, which was a copy of batchSize's

payLoadSize is a byte size — ReindexingConfiguration.DEFAULT_PAYLOAD_SIZE is
SearchClusterMetrics.DEFAULT_BULK_PAYLOAD_SIZE_BYTES — but its description was a
verbatim copy of batchSize's "Maximum number of events entities in a batch
(Default 100)", so the rendered app docs showed the wrong units and the wrong
default for it.

Collate's forked schema happened to carry the correct string; resyncing that doc
against this one in openmetadata-collate#6326 would have propagated the error
downstream instead. Fixing it here keeps both sides on one description.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Stub getIndexedEntityTypes in ReindexingOrchestratorTest

setupEntitiesExpandsAllAndCountTotalEntitiesSkipsUnsupportedTypes stubbed
searchRepository.getEntityIndexMap(), which setupEntities() no longer calls after
getAll() moved to getIndexedEntityTypes(). searchRepository is a Mockito mock, so
the new call returned Mockito's default empty Set, setupEntities() expanded "all"
to nothing, and the entities assertion failed.

Stub the method the code now calls. The IndexMapping mocks the old stub built were
only there to shape the map, so the import goes with them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Lower the recorded no-positional-locator count to match the baseline

#32576 pruned one om-playwright/no-positional-locator entry out of
eslint-suppressions.json (1323 -> 1322) without lowering the number recorded in
corpus.test.mjs, so `the suppressions baseline matches its recorded state
exactly` fails on main and therefore on every branch merged with it:

  + actual   'om-playwright/no-positional-locator': 1322
  - expected 'om-playwright/no-positional-locator': 1323

Both files here are byte-identical to main, so this is not a violation this
branch introduced or fixed — only the bookkeeping the pruning PR left behind.
Lowering it is what the assertion message asks for; `yarn lint:playwright` is
already clean at 0 errors, so there is nothing to prune.

* Do not constrain the entity enum when the list cannot be fetched

ApplicationConfiguration validates with @rjsf/validator-ajv8 against the stored
appConfiguration, so injecting `enum: ["all"]` on a failed fetch made a saved
`entities: ["table", ...]` fail validation — during a transient outage an admin
could not save the form at all, including edits to unrelated fields. Return the
schema untouched instead: the picker degrades to a plain list, but the stored
selection stays visible and editable. The toast still fires.

Also add the endpoint's own files to the impact-map entry. It listed
resources/apps/** and the searchIndex bundle but not the two files that actually
produce the list, so editing /v1/search/entityTypes or getIndexedEntityTypes()
would not have selected the picker spec — the exact gap the entry exists to close.

Both found by Copilot on a111a47.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: mohityadav766

.github/playwright/impact-map.json

@@ -363,6 +363,26 @@
         "playwright/e2e/Pages/Roles.spec.ts"
       ]
     },
+    {
+      "_comment": "The application config forms are built from the UI-local application schemas by ApplicationsClassBase, and the Search Indexing form additionally fills its entity picker from GET /v1/search/entityTypes. The Settings/** entry above selects no spec that actually opens an app config form, so a change to the schemas, the loader, or the endpoint that fills them shipped unrun — the SearchIndexApplication entity-picker locator broke once the server-driven list gained tableColumn and nothing caught it.",
+      "sources": [
+        "openmetadata-ui/src/main/resources/ui/src/components/Settings/Applications/**",
+        "openmetadata-ui/src/main/resources/ui/src/pages/AppInstall/**",
+        "openmetadata-ui/src/main/resources/ui/src/utils/ApplicationSchemas/**",
+        "openmetadata-ui/src/main/resources/ui/src/jsons/applicationSchemas/**",
+        "openmetadata-service/src/main/java/org/openmetadata/service/apps/bundles/searchIndex/**",
+        "openmetadata-service/src/main/java/org/openmetadata/service/resources/apps/**",
+        "openmetadata-service/src/main/java/org/openmetadata/service/resources/search/SearchResource.java",
+        "openmetadata-service/src/main/java/org/openmetadata/service/search/SearchRepository.java"
+      ],
+      "projects": ["chromium", "Basic"],
+      "specs": [
+        "playwright/e2e/Pages/SearchIndexApplication.spec.ts",
+        "playwright/e2e/Pages/DataInsightReportApplication.spec.ts",
+        "playwright/e2e/Pages/AppStopRunModal.spec.ts",
+        "playwright/e2e/Pages/AppRunsHistoryLogs.spec.ts"
+      ]
+    },
     {
       "sources": [
         "openmetadata-service/src/main/java/org/openmetadata/service/security/**",

openmetadata-integration-tests/src/test/java/org/openmetadata/it/tests/SearchResourceIT.java

@@ -7,6 +7,7 @@
 import static org.junit.jupiter.api.Assertions.assertThrows;
 import static org.junit.jupiter.api.Assertions.assertTrue;
 
+import com.fasterxml.jackson.core.type.TypeReference;
 import com.fasterxml.jackson.databind.JsonNode;
 import com.fasterxml.jackson.databind.ObjectMapper;
 import java.net.URI;
@@ -19,10 +20,13 @@
 import java.util.Locale;
 import java.util.concurrent.TimeUnit;
 import org.awaitility.Awaitility;
+import org.junit.jupiter.api.BeforeAll;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.extension.ExtendWith;
 import org.junit.jupiter.api.parallel.Execution;
 import org.junit.jupiter.api.parallel.ExecutionMode;
+import org.openmetadata.it.auth.JwtAuthProvider;
+import org.openmetadata.it.factories.UserTestFactory;
 import org.openmetadata.it.util.SdkClients;
 import org.openmetadata.it.util.TestNamespace;
 import org.openmetadata.it.util.TestNamespaceExtension;
@@ -1985,4 +1989,74 @@ void testExportWithFromBeyondResults(TestNamespace ns) throws Exception {
     String[] lines = response.body().split("\n");
     assertEquals(1, lines.length, "Export beyond results should only contain header");
   }
+
+  // ===================================================================
+  // INDEXED ENTITY TYPES (backs the reindex entity picker)
+  // ===================================================================
+
+  @Test
+  void testEntityTypesComesFromTheIndexRegistry(TestNamespace ns) throws Exception {
+    HttpResponse<String> response = httpGetJson("/v1/search/entityTypes");
+
+    assertEquals(200, response.statusCode());
+
+    List<String> entityTypes =
+        OBJECT_MAPPER.readValue(response.body(), new TypeReference<List<String>>() {});
+
+    assertFalse(entityTypes.isEmpty(), "Entity types should not be empty");
+    assertTrue(entityTypes.contains("table"), "Entity types should contain table");
+    // tableColumn has an index mapping but was absent from the enum the UI used to hardcode.
+    // Its presence is what proves the list is read from the registry rather than copied.
+    assertTrue(entityTypes.contains("tableColumn"), "Entity types should contain tableColumn");
+    assertEquals(
+        entityTypes.stream().sorted().toList(), entityTypes, "Entity types should be sorted");
+  }
+
+  /**
+   * A dataConsumer JWT hits SubjectCache.getUserContext during authorization; if that user has not
+   * been created in this JVM session the lookup throws EntityNotFoundException (→404) and
+   * short-circuits the authorizer before it can reach the permission check. Pin the user up front so
+   * the result is deterministic regardless of suite ordering.
+   */
+  @BeforeAll
+  static void ensureDataConsumerUser() {
+    UserTestFactory.getDataConsumer(null);
+  }
+
+  @Test
+  void testEntityTypesIsReadableByApplicationViewer(TestNamespace ns) throws Exception {
+    String consumerToken =
+        JwtAuthProvider.tokenFor(
+            "data-consumer@open-metadata.org",
+            "data-consumer@open-metadata.org",
+            new String[] {"DataConsumer"},
+            3600);
+
+    HttpResponse<String> response = httpGetJson("/v1/search/entityTypes", consumerToken);
+
+    // The app details page renders for anyone with Application view permission and fetches the
+    // config schema on mount, so a non-admin viewer must not get a 403 here.
+    assertEquals(
+        200, response.statusCode(), "Application viewer should be able to list entity types");
+    assertFalse(
+        OBJECT_MAPPER.readValue(response.body(), new TypeReference<List<String>>() {}).isEmpty(),
+        "Entity types should not be empty for an Application viewer");
+  }
+
+  private HttpResponse<String> httpGetJson(String path) throws Exception {
+    return httpGetJson(path, SdkClients.getAdminToken());
+  }
+
+  private HttpResponse<String> httpGetJson(String path, String token) throws Exception {
+    HttpRequest request =
+        HttpRequest.newBuilder()
+            .uri(URI.create(SdkClients.getServerUrl() + path))
+            .header("Authorization", "Bearer " + token)
+            .header("Accept", "application/json")
+            .timeout(Duration.ofSeconds(30))
+            .GET()
+            .build();
+
+    return HTTP_CLIENT.send(request, HttpResponse.BodyHandlers.ofString());
+  }
 }

openmetadata-service/src/main/java/org/openmetadata/service/apps/bundles/searchIndex/ReindexingOrchestrator.java

@@ -509,7 +509,7 @@ private void setupEntities() {
   }
 
   private Set<String> getAll() {
-    return new HashSet<>(searchRepository.getEntityIndexMap().keySet());
+    return new HashSet<>(searchRepository.getIndexedEntityTypes());
   }
 
   private boolean hasSlackConfig() {

openmetadata-service/src/main/java/org/openmetadata/service/resources/search/SearchResource.java

@@ -20,6 +20,7 @@
 import es.co.elastic.clients.elasticsearch.core.SearchResponse;
 import io.swagger.v3.oas.annotations.Operation;
 import io.swagger.v3.oas.annotations.Parameter;
+import io.swagger.v3.oas.annotations.media.ArraySchema;
 import io.swagger.v3.oas.annotations.media.Content;
 import io.swagger.v3.oas.annotations.media.Schema;
 import io.swagger.v3.oas.annotations.parameters.RequestBody;
@@ -62,6 +63,7 @@
 import org.openmetadata.schema.search.SearchRequest;
 import org.openmetadata.schema.type.EntityReference;
 import org.openmetadata.schema.type.Include;
+import org.openmetadata.schema.type.MetadataOperation;
 import org.openmetadata.schema.utils.JsonUtils;
 import org.openmetadata.search.IndexMapping;
 import org.openmetadata.service.Entity;
@@ -82,6 +84,8 @@
 import org.openmetadata.service.search.SearchUtils;
 import org.openmetadata.service.search.indexes.SearchIndex;
 import org.openmetadata.service.security.Authorizer;
+import org.openmetadata.service.security.policyevaluator.OperationContext;
+import org.openmetadata.service.security.policyevaluator.ResourceContext;
 import org.openmetadata.service.security.policyevaluator.SubjectContext;
 import org.openmetadata.service.util.AsyncService;
 import org.openmetadata.service.util.AsyncService.DatabaseOperation;
@@ -834,6 +838,40 @@ public Response aggregateSearchRequest(
     return searchRepository.aggregate(aggregationRequest);
   }
 
+  @GET
+  @Path("/entityTypes")
+  @Operation(
+      operationId = "getIndexedEntityTypes",
+      summary = "List the entity types that have a search index",
+      description =
+          "Entity types registered in the index mapping for this deployment, sorted. Includes "
+              + "distribution-specific indexes (e.g. Collate-only entity types) because the "
+              + "registry is merged from the classpath at startup. This is the same set that "
+              + "reindexing expands \"all\" into, so clients can offer an entity picker without "
+              + "hardcoding a list that goes stale.",
+      responses = {
+        @ApiResponse(
+            responseCode = "200",
+            description = "Sorted list of entity types",
+            content =
+                @Content(
+                    mediaType = "application/json",
+                    array = @ArraySchema(schema = @Schema(type = "string")))),
+        @ApiResponse(responseCode = "403", description = "No view permission on Application")
+      })
+  public List<String> getIndexedEntityTypes(@Context SecurityContext securityContext) {
+    // Gate on Application view, not admin: SettingsRouter renders the app details page for
+    // `isAdminUser || hasViewPermissions(APPLICATION)` and that page fetches the config schema on
+    // every mount, so an admin-only check here would 403 a legitimate viewer on a request they
+    // never triggered.
+    OperationContext operationContext =
+        new OperationContext(Entity.APPLICATION, MetadataOperation.VIEW_BASIC);
+    authorizer.authorize(
+        securityContext, operationContext, new ResourceContext<>(Entity.APPLICATION));
+
+    return List.copyOf(searchRepository.getIndexedEntityTypes());
+  }
+
   @GET
   @Path("/entityTypeCounts")
   @Operation(

openmetadata-service/src/main/java/org/openmetadata/service/search/SearchRepository.java

@@ -984,6 +984,20 @@ public IndexMapping getIndexMapping(String entityType) {
     return entityIndexMap.get(entityType);
   }
 
+  /**
+   * Entity types that have a search index registered for this deployment, sorted. The registry is
+   * merged from the classpath at startup ({@code elasticsearch/indexMapping.json} plus
+   * {@code elasticsearch/collate/indexMapping.json} when present), so Collate-only indexes are
+   * included without the caller knowing which distribution it runs on.
+   *
+   * <p>This is the authoritative reindexing target list: {@code SearchIndexingApplication} expands
+   * {@code "all"} from it and {@code GET /v1/search/entityTypes} serves it to the entity picker, so
+   * the two cannot drift.
+   */
+  public Set<String> getIndexedEntityTypes() {
+    return Collections.unmodifiableSet(new TreeSet<>(entityIndexMap.keySet()));
+  }
+
   /**
    * Register a staged index as the live-write target for {@code entityType} while a reindex
    * populates it. Must be paired with {@link #unregisterStagedIndex(String, String)} once the

openmetadata-service/src/test/java/org/openmetadata/service/apps/bundles/searchIndex/ReindexingOrchestratorTest.java

@@ -43,7 +43,6 @@
 import org.openmetadata.schema.system.Stats;
 import org.openmetadata.schema.system.StepStats;
 import org.openmetadata.schema.utils.JsonUtils;
-import org.openmetadata.search.IndexMapping;
 import org.openmetadata.service.Entity;
 import org.openmetadata.service.apps.bundles.searchIndex.OrphanedIndexCleaner.CleanupResult;
 import org.openmetadata.service.apps.bundles.searchIndex.SearchIndexApp.ReindexingException;
@@ -316,9 +315,7 @@ void setupEntitiesExpandsAllAndCountTotalEntitiesSkipsUnsupportedTypes() throws
     String reportType =
         org.openmetadata.schema.analytics.ReportData.ReportDataType.ENTITY_REPORT_DATA.value();
 
-    when(searchRepository.getEntityIndexMap())
-        .thenReturn(
-            Map.of(Entity.TABLE, mock(IndexMapping.class), reportType, mock(IndexMapping.class)));
+    when(searchRepository.getIndexedEntityTypes()).thenReturn(Set.of(Entity.TABLE, reportType));
     when(entityRepository.getDao()).thenReturn(entityDao);
     when(entityDao.listCount(any())).thenReturn(7);
 

openmetadata-ui/src/main/resources/ui/playwright/e2e/Pages/SearchIndexApplication.spec.ts

@@ -318,7 +318,13 @@ test.describe('Search Index Application', PLAYWRIGHT_BASIC_TEST_TAG_OBJ, () => {
         .getByRole('combobox')
         .fill('Table');
 
-      const tableTitle = page.getByRole('tree').getByTitle('Table');
+      // Exact: the entity list is server-driven now, and rc-tree-select filters on the node value
+      // (treeNodeFilterProp defaults to 'value'), so typing "Table" also leaves `tableColumn` —
+      // rendered as "Table Column" — visible. A substring getByTitle would match both and break
+      // strict mode.
+      const tableTitle = page
+        .getByRole('tree')
+        .getByTitle('Table', { exact: true });
 
       // Wait for the filtered tree result to render
       await tableTitle.waitFor({ state: 'visible' });

openmetadata-ui/src/main/resources/ui/public/locales/en-US/Applications/SearchIndexingApplication.md

@@ -12,7 +12,7 @@ $$
 $$section
 ### Payload Size $(id="payLoadSize")
 
-Maximum number of events entities in a batch (Default 100).
+Payload size in bytes (Default 104857600).
 
 $$
 

openmetadata-ui/src/main/resources/ui/src/components/Settings/Applications/AppDetails/ApplicationsClassBase.test.ts

@@ -13,10 +13,78 @@
 
 import { AppType } from '../../../../generated/entity/applications/app';
 import rdfIndexAppSchema from '../../../../jsons/applicationSchemas/RdfIndexApp.json';
+import searchIndexingAppSchema from '../../../../jsons/applicationSchemas/SearchIndexingApplication.json';
+import { getSearchEntityTypes } from '../../../../rest/searchAPI';
+import { showErrorToast } from '../../../../utils/ToastUtils';
 import applicationsClassBase from './ApplicationsClassBase';
 
+jest.mock('../../../../rest/searchAPI', () => ({
+  getSearchEntityTypes: jest.fn().mockResolvedValue([]),
+}));
+
+jest.mock('../../../../utils/ToastUtils', () => ({
+  showErrorToast: jest.fn(),
+}));
+
+const mockGetSearchEntityTypes = getSearchEntityTypes as jest.Mock;
+
+// importSchema resolves to `{}` upstream, so name the one shape these tests read rather
+// than reaching through it untyped.
+type EntitiesEnumSchema = {
+  properties: { entities: { items: { enum: string[] } } };
+};
+
 describe('ApplicationsClassBase', () => {
+  beforeEach(() => {
+    jest.clearAllMocks();
+    mockGetSearchEntityTypes.mockResolvedValue([]);
+  });
+
   describe('importSchema', () => {
+    it('should fill the SearchIndexingApplication entity list from the server', async () => {
+      mockGetSearchEntityTypes.mockResolvedValue(['dynamicAgent', 'table']);
+
+      const schema = (await applicationsClassBase.importSchema(
+        'SearchIndexingApplication'
+      )) as EntitiesEnumSchema;
+
+      // 'all' is the backend sentinel for "every registered index"; it is not an index, so the
+      // endpoint does not return it, but the ["all"] default has to validate against the enum.
+      expect(schema.properties.entities.items.enum).toEqual([
+        'all',
+        'dynamicAgent',
+        'table',
+      ]);
+    });
+
+    it('should not hardcode the entity list in the schema json', () => {
+      const { items } = searchIndexingAppSchema.properties.entities;
+
+      // A shipped enum is the bug this endpoint replaced: one list for every deployment,
+      // stale every time an entity type was added and blind to Collate-only indexes.
+      expect(items).not.toHaveProperty('enum');
+    });
+
+    it('should toast and leave the schema unconstrained when the server call fails', async () => {
+      const error = new Error('boom');
+      mockGetSearchEntityTypes.mockRejectedValue(error);
+
+      const schema = (await applicationsClassBase.importSchema(
+        'SearchIndexingApplication'
+      )) as EntitiesEnumSchema;
+
+      // Narrowing the enum here would make a stored `entities: ["table", …]` fail the
+      // form's AJV validation and block saving until the endpoint recovers.
+      expect(schema.properties.entities.items).not.toHaveProperty('enum');
+      expect(showErrorToast).toHaveBeenCalledWith(error);
+    });
+
+    it('should not fetch entity types for other applications', async () => {
+      await applicationsClassBase.importSchema('RdfIndexApp');
+
+      expect(mockGetSearchEntityTypes).not.toHaveBeenCalled();
+    });
+
     it('should import pre-parsed schema', async () => {
       // Mock the dynamic import
       jest.doMock(