Skip to content

farallon_bird-mammal: cruise_key is NULL on all 66,272 obs rows #74

Description

@bbest

In release v2026.08.04, every obs row for farallon_bird-mammal has a
NULL cruise_key:

SELECT dataset_key, count(*) AS obs, count(cruise_key) AS with_cruise,
       count(DISTINCT cruise_key) AS n_cruises
FROM read_parquet('.../v2026.08.04/parquet/obs.parquet')
WHERE dataset_key IN ('farallon_bird-mammal','cce-lter_zoodb','swfsc_ichthyo')
GROUP BY 1;
dataset_key obs with_cruise n_cruises
cce-lter_zoodb 18,276 17,022 35
farallon_bird-mammal 66,272 0 0
swfsc_ichthyo 459,286 459,286 671

Why it matters

cruise_key is denormalized onto obs specifically so consumers can roll up by
survey without a join. Any consumer doing count(DISTINCT cruise_key) reports
zero for this dataset — not "unknown", but a confident zero.

It's already visible to users: the station portal's coverage card reads
"0 surveys · 760 obs" for Seabirds & Marine Mammals at station 090.0 055.0,
and does so for all 101 of its stations. Other datasets have only 1–3
stragglers each. Screenshot in CalCOFI/db-viz-station#10.

The observations themselves look fine — 760 obs across 457 samples at that
station, 1987-05-05 → 2018-04-10 — so this is a missing FK stamp, not missing
data.

Suggested fix

In the bird/mammal ingest's "Emit Core Tables" section, resolve each transect to
its cruise and pass cruise_key through to append_obs() the way the other
ingests do. If these transects genuinely have no CalCOFI cruise association,
that's worth stating explicitly in the dataset metadata so consumers can
distinguish "not applicable" from "not populated" — a NULL FK currently reads as
the latter.

Worth checking whether sample.cruise_key has the same gap.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions