Skip to content

Generate accurate schema.org provenance for current data releases #325

Description

@cboettig

inst/prov/fb.prov and inst/prov/slb.prov ship ~1 MB of JSON-LD that is now stale: the records describe releases up to v23.05 hosted on the rfishbase_board GitHub repo, name import_db.R as the generating script (removed in #324), and were produced by the prov / contentid workflow that has not run since hosting moved to HuggingFace and then to Source Cooperative. Nothing in the package reads these files.

The data itself is now published under cboettig/fishbase/{fb,slb}/v<YY.MM>/parquet/ on Source Cooperative, with data-raw/ holding the scripts that generate it (see data-raw/README.md). So the pieces for real provenance exist, they're just not wired to anything.

Worth deciding:

  • Regenerate or drop. Either emit fresh schema.org Dataset records covering the current releases (through fb v26.07 / slb v26.04) or delete inst/prov/ outright. Shipping records that point at dead URLs and a script that no longer exists is worse than shipping none.
  • Where it lives. A per-release JSON-LD sidecar next to the parquet on Source Cooperative is probably more useful than a file inside the R package — it describes the data, not the package, and it would not bloat the installed package or need a CRAN release to update.
  • How it gets made. If we want this maintained rather than bit-rotting again, it should be a step in data-raw/import_dumps.sh that runs on every import, not a script invoked by hand.
  • Content hashing. The old workflow registered content identifiers with contentid against hash-archive. Decide whether that is still wanted; if so it needs a home that is expected to outlive the current one.

Not urgent — the data and the package are correct without it. Filing so the decision is recorded rather than rediscovered at the next import.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions