Raster Filename Parsing

Scientific data is commonly serialized and exchanged in common raster data formats such as GeoTIFF and ASC. While these file types are ubiquitous and relatively easy to explore, they lack the self-describing quality of other scientific data formats such as NetCDF and HDF. As such, metadata such as variable type, units, and dates are often added to the filename, following no particular convention, as a way to describe the contents of the file.

It is a challenge for a tool like Vortex to reasonably handle all of the possibilities when tasked with parsing unconventional file names. There are a few common data sources that use non-self-describing raster formats, for which specialized file name parsing logic has been added to Vortex. These sources include:

• PRISM (ASC and BIL files)
• ATLAS 14 (ASC files)
• SNODAS (DAT files)

If there are other common data sources that you believe would provide benefit to the hydrologic modeling community given specialized parsing logic, feel free to suggest them.

For importing uncommon or unconventional data sources, you may consider using the HEC-DSS C, Java, Jython, or Python APIs directly. Alternatively you can use Vortex, keeping a few things in mind. The rest of this page describes what Vortex does with a file it has no specialized logic for.

Dates

Vortex will attempt to parse dates in the filename. The dates must use the ISO 8601 standard with minimal variation.

If two compliant dates are provided, the first will be applied to the DSS pathname D-Part and the second will be applied to the DSS pathname E-Part. If one compliant date is provided, it will apply to both.

Filename	D-Part	E-Part
precip_2021-12-07T1200_to_2021-12-07T1300.tif	07DEC2021:1200	07DEC2021:1300
2021-12-07T1200_TEMPERATURE.tif	07DEC2021:1200	07DEC2021:1200

The accepted variations from the ISO 8601 standard are that the separator between year and month, and between month and day, can optionally be an underscore rather than a hyphen, and that the separator between date and time can optionally be an underscore or a hyphen rather than a T. Case does not matter, so t works as well as T.

Two limits are worth knowing, since neither is obvious from the examples above:

• A time is required. The date must be followed by a four digit HHmm time. A filename carrying only 2021-12-07 is not recognised as a date at all, and the record is written without one.
• Seconds are not parsed. The time is exactly four digits, so 120000 will not match.

Times are interpreted as UTC.

Variable type

Vortex will attempt to parse the filename for variable type. You can also specify the C-Part as a DSS write option using the partC argument. For more, see Batch Import Options.

Units

Vortex does not parse the filename for units. You can specify the units as a DSS write option using the units argument. For more, see Batch Import Options.