birdnet prediction outputs are not serialized consistently across save methods.
Problem
Two friction points:
-
Parquet
start_time and end_time are written as Arrow halffloat
- Python reads them correctly
- R Arrow reads raw float16 bit patterns instead of seconds
-
CSV
start_time and end_time are serialized as "HH:MM:SS.xx" strings
- in-memory outputs use numeric seconds
Expected
Saved outputs should be predictable across formats and languages.
- Parquet should store time columns in an interoperable numeric type
- CSV should either preserve numeric seconds or be explicitly treated as a human-readable export
Observed
In memory:
arr = prediction.to_structured_array()
arr["start_time"][:3]
# array([0., 3., 6.], dtype=float16)
Parquet schema
prediction.to_arrow_table().schema
# start_time: halffloat
# end_time: halffloat
R read-back
arrow::read_parquet("test.parquet", as_data_frame = TRUE)
# start_time
# 0
# 16896
# 17920
Expected values were 0, 3, 6.
CSV output:
00:00:00.00
00:00:03.00
00:00:06.00
Likely cause
start_time / end_time inherit float16 and are passed to Arrow as pa.float16(). That produces Parquet halffloat, which is not read correctly by R Arrow.
Proposed fix
Upcast start_time and end_time to at least float32 before Arrow table creation.
Possibly also upcast confidence when it is float16, for consistency.
Why this matters
The same prediction currently becomes:
- numeric seconds in memory
- formatted strings in CSV
- correct numeric values in Python Parquet
- corrupted-looking values in R Parquet
That makes cross-language round-tripping unreliable.
birdnetprediction outputs are not serialized consistently across save methods.Problem
Two friction points:
Parquet
start_timeandend_timeare written as ArrowhalffloatCSV
start_timeandend_timeare serialized as"HH:MM:SS.xx"stringsExpected
Saved outputs should be predictable across formats and languages.
Observed
In memory:
Parquet schema
R read-back
Expected values were 0, 3, 6.
CSV output:
Likely cause
start_time/end_timeinheritfloat16and are passed to Arrow aspa.float16(). That produces Parquethalffloat, which is not read correctly by R Arrow.Proposed fix
Upcast
start_timeandend_timeto at leastfloat32before Arrow table creation.Possibly also upcast
confidencewhen it isfloat16, for consistency.Why this matters
The same prediction currently becomes:
That makes cross-language round-tripping unreliable.