Fix elasticsearch2 schema load on ES7+/8 mapping metadata keys

[tostavio]

What type of PR is this?

• Bug Fix

Description

This pull request fixes the elasticsearch2 query runner, which fails to load the schema and shows "Schema refresh failed" against Elasticsearch 7.x and 8.x.

ElasticSearch2._parse_mappings iterates over the keys under mappings and assumes that each one is a mapping type whose value is a dictionary containing properties:

for m in index_mappings.get("mappings", {}):
    _parse_properties("", index_mappings["mappings"][m]["properties"])

This structure only held for Elasticsearch 6.x and earlier. Since mapping types were removed in Elasticsearch 7, the _mappings response places the metadata keys directly under mappings. For example, dynamic, which is a boolean, and dynamic_templates, which is a list, now appear alongside properties:

{
  "my-index": {
    "mappings": {
      "dynamic": false,
      "dynamic_templates": [ { "strings": { "mapping": { "type": "keyword" } } } ],
      "properties": { "sku": { "type": "keyword" } }
    }
  }
}

When the loop reaches m == "dynamic", it evaluates mappings["dynamic"]["properties"], which is False["properties"], and this raises TypeError: 'bool' object is not subscriptable. The dynamic_templates key, which is a list, fails in the same way. The surrounding except KeyError does not catch TypeError, so get_schema aborts for the entire data source whenever any visible index carries those keys. This is common with APM and Kibana system indices, such as .internal.alerts-*, .kibana-*, .ds-traces-apm-*, and .ds-metrics-apm-*, and with user indices that set dynamic or dynamic_templates.

Fix

Catch TypeError alongside KeyError, so that the loop falls back to reading mappings["properties"] directly, which is the correct shape for Elasticsearch 7 and 8:

-        except KeyError:
+        except (KeyError, TypeError):
             _parse_properties("", index_mappings["mappings"]["properties"])

Queries are not affected, because they go through /{index}/_search; only the schema parsing was broken. The change is compatible with Elasticsearch 7.x and 8.x.

How is this tested?

• Unit tests (pytest, jest)

I added TestElasticSearch.test_parse_mappings_with_dynamic_keys, which parses a mapping that contains dynamic and dynamic_templates next to properties and asserts that the field types are extracted. Without the fix this test raises TypeError; with the fix, the schema is parsed correctly.

Related Tickets & Documents

Closes #7761.

This pull request is also related to #5545, which reports the same user-facing symptom ("Schema refresh failed") against the older elasticsearch runner. This change addresses the elasticsearch2 runner specifically.

Mobile & Desktop Screenshots/Recordings (if there are UI changes)

N/A. This is a backend-only change, with no UI.

[cubic-dev-ai]

No issues found across 2 files

_{Re-trigger cubic}

[greptile-apps]

Greptile Summary

This PR restores schema loading for Elasticsearch 7 and 8 mappings whose metadata values are not dictionaries.

• Extends the mapping-layout fallback to handle TypeError from metadata keys such as dynamic and dynamic_templates.
• Adds a regression test covering boolean and list metadata alongside top-level properties.

Confidence Score: 5/5

The PR appears safe to merge, with the schema parser now handling the documented Elasticsearch 7 and 8 mapping shape.

The added exception handling reaches the existing top-level properties fallback when mapping metadata is boolean or list-valued, and the regression test covers both representative metadata forms without changing query execution.

Important Files Changed

Filename	Overview
redash/query_runner/elasticsearch2.py	Broadens the existing ES7 mapping fallback to catch the concrete exception raised by non-dictionary mapping metadata values.
tests/query_runner/test_elasticsearch2.py	Adds focused regression coverage confirming fields remain correctly extracted when dynamic and dynamic_templates metadata are present.

_{Reviews (1): Last reviewed commit: "Merge branch 'master' into fix/es2-mappi..." | Re-trigger Greptile}