Fix silent loss of CF-conformant NetCDF grids - #130
Open
mikebartles wants to merge 1 commit into
Open
Conversation
A NetCDF variable that declares its statistic the way CF requires, as "time: mean", resolved to UNDEFINED and fell through to type inference. Inference then typed the record INSTANTANEOUS despite a non-zero interval, and the instantaneous index holds only records whose start and end times are equal, so every record was skipped. The reader reported no time range and every read came back empty. Callers received a null grid with nothing to explain it. NetcdfDataReader now extracts the method applied to the time coordinate before mapping it to a data type. VortexDataType.fromString stays a plain token mapper because it is shared with the DSS path. The entry may be keyed on either the literal "time" or the file's own time axis name, both of which CF permits. Bare tokens still resolve as they did, so files vortex wrote itself are unaffected. Inference no longer returns a point type for a record that spans a period. A record with a non-zero interval is a period type by definition, and calling it instantaneous produces a record the index cannot hold. This alone fixes the reported file, independently of any cell_methods parsing. A record dropped for disagreeing with its own declared type is now logged and reported through DataReader.isValid(). Previously isValid() returned true with an empty message list for a file that could not be read at all, so every validation gate passed and the defect surfaced only at compute time. The NetCDF writer now emits "time: <method>" so vortex stops producing the non-conformant form it was the sole reader of. Two fixtures that differ only in that attribute, cf_style.nc and bare_mean.nc, pin both spellings against regression. Resolves: HMS-5051
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A NetCDF variable that declares its statistic the way CF requires, as "time: mean", resolved to UNDEFINED and fell through to type inference. Inference then typed the record INSTANTANEOUS despite a non-zero interval, and the instantaneous index holds only records whose start and end times are equal, so every record was skipped. The reader reported no time range and every read came back empty. Callers received a null grid with nothing to explain it.
NetcdfDataReader now extracts the method applied to the time coordinate before mapping it to a data type. VortexDataType.fromString stays a plain token mapper because it is shared with the DSS path. The entry may be keyed on either the literal "time" or the file's own time axis name, both of which CF permits. Bare tokens still resolve as they did, so files vortex wrote itself are unaffected.
Inference no longer returns a point type for a record that spans a period. A record with a non-zero interval is a period type by definition, and calling it instantaneous produces a record the index cannot hold. This alone fixes the reported file, independently of any cell_methods parsing.
A record dropped for disagreeing with its own declared type is now logged and reported through DataReader.isValid(). Previously isValid() returned true with an empty message list for a file that could not be read at all, so every validation gate passed and the defect surfaced only at compute time.
The NetCDF writer now emits "time: " so vortex stops producing the non-conformant form it was the sole reader of. Two fixtures that differ only in that attribute, cf_style.nc and bare_mean.nc, pin both spellings against regression.
Resolves: HMS-5051