Conversion to and from PACKAGES file

[dgkf]

Developer day outcome:
Allow package data to be able to be round-trip'ed to a PACKAGES file

In support of a conventional repository of package qualities, we decided to make the PACKAGES file (dcf format, similar to DESCRIPTION) the common denominator for sharing package metadata. Writing user-facing data (data which may be used to form decisions) out to a PACKAGES file and reading it back in to R should produce the same object that we may use within riskmetric itself for decisions.

This decision has a few ramifications:

• Scalars are preferred because it's the least ambiguous data to write and read
• Vectors are likely needed as well. Some investigation needed to standardize a style, but we initially took the dependencies fields as inspiration (eg, a, b, c comma separated list)
• We would like error values to be able to pass through this format. Perhaps as error codes or as R code (similar to Authors@R)

Even before we have a full implementation of this, I think it would allow other teams to move forward if we could produce a minimal example of what we're aiming for.

[dgkf]

Just in terms of syntax, I think metrics should have a prefix of some sort to limit conflict with other fields.

Something like MetricHasNews or RiskmetricHasNews

[dgkf]

Coming from the reporting team, it would be nice to store the download url of the tarball

MetricSource: https://site.org/package.tar.gz

pharmaR/riskreports#31 (comment)

[llrs-roche]

Not to derail from this issue/objective, but I found a related file on Bioconductor. It is a JSON file with some of metrics riskmetrics calculates too:

https://bioconductor.org/packages/json/3.20/bioc/packages.json

    "hasREADME": false,
    "hasNEWS": true,
    "hasINSTALL": false,
    "hasLICENSE": false,
    ....
    "dependencyCount": "10",

Maybe we can find a common ground so that we work with them and the repository could provide some of these metrics (and CRAN in the future).

I think there is a VIEWS file I cannot locate where they provide the same or similar information in a DCF format, but I haven't found it again.

For future references here is a DCF file for Bioconductor packages with the same metadata as the json file:
https://bioconductor.org/packages/3.20/bioc/VIEWS

[rabiibouhestine]

I have been trying to find a good way to write the object that we currently get from pkg_score() to a file that can be round-trip'ed (get the same object back when reading it). I have been experimenting with both dcf and json formats.

dcf seems to provide very little support for complexe structures (lists, attributes, or S3 class metadata).

json seems more promising but I haven't been able yet to serialise the pkg_score() output properly, I've been working on something like this:

serialize_with_attributes <- function(x) {
  if (is.list(x) && !is.null(names(x))) {
    result <- list()
    for (n in names(x)) {
      result[[n]] <- serialize_with_attributes(x[[n]])
    }
    return(result)
  }

  # For atomic or S3 elements with attributes
  val <- unclass(x)
  attrs <- attributes(x)

  # Sanitize attributes (no S3 objects inside)
  if (!is.null(attrs)) {
    for (a in names(attrs)) {
      if (inherits(attrs[[a]], "AsIs") || is.factor(attrs[[a]])) {
        attrs[[a]] <- as.character(attrs[[a]])
      } else if (!is.atomic(attrs[[a]])) {
        attrs[[a]] <- unclass(attrs[[a]])
      }
    }
  }

  list(
    value = val,
    attributes = attrs
  )
}

metrics_serialized <- serialize_with_attributes(metrics)
write_json(metrics_serialized, "PACKAGE.json", pretty = TRUE, auto_unbox = FALSE)

Do we have to keep the current structure of the output of pkg_score() (for backwards compatibility) ?

[llrs-roche]

Hi Rabii, thanks for looking into this.
Yes, the DCF file format is mostly a plain text file with key: value structure, this is the challenge of this issue.
I think the JSON format has the same problem with attributes and S3 class metadata (but it handles better the lists).

Regarding the pkg_score output, I am not sure if we decided to break back compatibility or not. But the object format will change (Not sure if the attributes will be kept), as the decision on the developer meeting was to "rewrite object structure".