Fix InfluxDB dropping columns when a query returns several series - #7805
Open
MaxFreedomPollard wants to merge 2 commits into
Open
MaxFreedomPollard wants to merge 2 commits into
MaxFreedomPollard wants to merge 2 commits into
Conversation
_transform_result in redash/query_runner/influx_db.py collects the union of column names and tag keys across every series, then throws that union away and builds the result columns from the keys of the first row only. When a query returns several series with different column sets, such as "SELECT * FROM /^(cpu|mem)$/" or two statements separated by a semicolon, any column missing from the first row is left out of "columns" even though its values are present in "rows". The table visualization renders from "columns", so those values never reach the user. Record each column's type the first time a row supplies a value for it and build the result columns from the union of column names, falling back to TYPE_STRING for a column no row supplies. Results with a single series, including the empty one, are unchanged.
Greptile SummaryThe PR fixes InfluxDB multi-series result transformation so columns appearing after the first row remain visible.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| redash/query_runner/influx_db.py | Preserves the complete multi-series column union and derives metadata from available non-null values. |
| tests/query_runner/test_influx_db.py | Adds regression coverage for columns distributed across series and type inference after null values. |
Reviews (2): Last reviewed commit: "InfluxDB: take column types from non-nul..." | Re-trigger Greptile
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 2 files
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
_transform_result in redash/query_runner/influx_db.py records a column's type at the "if column not in column_types" test on line 62, from whichever row first supplies a value for that column. When that first value is None, _get_type returns TYPE_STRING because type(None) is not in TYPES_MAP, and the entry is never revisited, so a numeric column whose first value is null is labeled a string column. InfluxDB produces null values for empty buckets, for example a GROUP BY time query over a range with gaps, and it returns several series whose column sets differ, so the row that first mentions a column often carries no value for it. Record the type only from a non-null value. A column whose values are all null keeps the TYPE_STRING fallback that column_types.get already supplies. test_influxdb_column_types_from_first_non_null_value in tests/query_runner/test_influx_db.py covers a column whose first value is null and whose second is a float, a column supplied only by a later series, and a column that is null in every row.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Description
_transform_resultinredash/query_runner/influx_db.pywalks every series first and collectscolumn_names, the union of the column names and tag keys across all of them. It then discards that union and builds the result columns fromresult_rows[0].keys()whenever there is at least one row.Any InfluxDB query that returns several series with different column sets hits this.
SELECT * FROM /^(cpu|mem)$/returns one series per measurement, and two statements separated by a semicolon return one ResultSet each;run_queryalready wraps both into the list this function iterates. A column that the first row happens not to have is then left out ofcolumnseven though its values are sitting inrows. The table visualization builds its columns fromdata.columns(getColumnsinclient/app/services/query-result.js, thengetOptionsinviz-lib/src/visualizations/table/getOptions.ts), so those values never appear in the UI.The fix records a column's type from the first non-null value a row supplies for it, and builds the result columns from
column_names, the union the function was already computing. A column that no row supplies falls back toTYPE_STRING, which is what the old empty-result branch did. Single series results, including the empty one, come out exactly as before, so the two existing tests needed no changes.How is this tested?
test_influxdb_columns_from_all_seriesintests/query_runner/test_influx_db.pyfeeds one series with columnstime, v1and one with columnstime, v2plus the tagk1. Without the change it fails withcolumnsholding onlytimeandv1; with it,columnsholdstime,v1,v2andk1and the types are read from whichever row first supplies each value.test_influxdb_column_types_from_first_non_null_valuein the same file feeds a series whose first row is null in every value column, and checks thatv1takesTYPE_FLOATfrom the next row of the same series,v2takesTYPE_INTEGERfrom a later series, andv3, which is null in every row, keepsTYPE_STRING; without the changev1andv2are labeled strings.ruff check .at 0.0.287 andblack --check .at 23.1.0, the versions the backend-lint job installs, are both clean, and so isblack --checkat the 24.4.2 that Restyled runs.Related Tickets & Documents
None.