How to enable binaries per repository

[jeroen]

Usually not all repositories support (some type of) binaries. For example we may use R_PLATFORM_PKGTYPE=linux.binary.resolute-x86_64 and have a repo like: https://ropensci.r-universe.dev/bin/linux/resolute-x86_64/contrib/4.6/PACKAGES

But then if we do:

options(repos = c(ropensci = "https://ropensci.r-universe.dev", CRAN = "https://cran.r-project.org"))
install.packages("magick")

We get a warning every time that CRAN does not support the binary pkgtype. Can we set a flag on which repos have binaries?

The etc/repositories file in R has columns that specify source, win.binary, mac.binary. Does this work at all? Is it expected that mac.binary also flips on/off the bin repo for custom binaries?

@s-u Is there a way we can set this mac.binary flag when setting options('repos') from R code?

[s-u]

Which type= setting is this for? "binary" or "both"? For "binary" this shouldn't be an issue as you are supposed to only list the ones that have those binaries. As usual, "both" is the main problem. This likely will needs some thought - one of the ideas was to allow a list in repos to have separate URLs for binaries and sources.

[jeroen]

It is indeed for type = "both" which is the default getOption("pkgType") when we start R with any R_PLATFORM_PKGTYPE.

[s-u]

Got it, that's as expected. I think supporting something like repos=list(source=c(ropensci = "https://ropensci.r-universe.dev", CRAN="https://cran.r-project.org"), binary="https://ropensci.r-universe.dev") is probably a way to go (or possibly something like both= or all= so we don't have to repeat URLs).

The other part you raised as repositories - and related setRepositories() which actually ignores "both" (i.e. just replaces it with "sources" as most of the tools did). If we support the above, then we can change the loading of default repos (and setRepositories()) to generate the list according to etc/repositories including the binaries.

[jeroen]

Yes that would work. An alternative more backward compatible API (and easier to implement) is to remain options('repos') be a named vector, and support an optional attribute pkgtype which says "both" or "source" or "binary".

repos <- c(ropensci = "https://ropensci.r-universe.dev", CRAN = "https://cran.r-project.org")
attributes(repos)$pkgType <- c("both", "source")
options(repos = repos)

Notably [<- preserves attributes, so repos["CRAN"] <- url in chooseCRANmirror() keeps the attribute intact; the ~38 direct consumers of getOption("repos") in base packages all keep working since it's still a character vector.

The benefit would be that this also works for older versions of R, where the new attribute is ignored.

[jeroen]

Maybe it is not too complicated after all, for example: https://github.com/r-devel/r-svn/pull/279/changes

[s-u]

Fair enough, but I would use list instead of structure (as attributes are unwieldy) and just take types= element. I don't think we need to be backward-compatible with repos as it is R-version specific anyway.

I think at this point it's a matter of defining the structure - what you implemented was repos=list(repos=, types=) where their lengths must match. The proposal above was repos=list(source=, binary=, both=). I'm slightly worried about the types and repos getting out of sync. @kurthornik do you have a preference? My notes from Vienna last year show:

   pkgType = list(CRAN=c("binary", "source"), rforge="source")
   ->
   c("https://cran.r-project.org/bin/macosx/big-sur-arm64/contrib/4.5",
     "https://cran.r-project.org/src/contrib", "https://forge.net/src/contrib")

which was putting the burden on pkgType - I think what we have now is better and equially functional (unless I'm missing something), but as a concept looks closer to what @jeroen implemented so the above would be

   repos=list(c(CRAN="https://cran.r-project.org/", rforge="https://forge.net/src/contrib"),
              types=c("both", "source"))

whereas the alternative was

   repos=list(both=c(CRAN="https://cran.r-project.org/")
              source=c(rforge="https://forge.net/src/contrib")))

I like the latter better as being more readable, but have no strong opinion.

[s-u]

I should add that in theory the first version is more versatile as it could specify the order in which things are resolved between sources and binaries, but, honestly, I don't want to go there since our current implementation treats it as a set so we go with binaries first regardless of the order so at the moment the latter matches that better. I'm not sure we want to open that can of worms, though, since the dependency resolution is already bad enough.

[jeroen]

New proposal discussed in Birmingham:

Use an optional ".binary" or ".source" postfix on the names property of the repos to limit (filter) repos to only binary or source:

# We have Linux binaries only for r-universe
options( repos =  c(
   CRAN.source = "https://cran.r-project.org",
   ropensci = "https://ropensci.r-universe.dev"
))

Benefits for this format:

• Existing/legacy scripts that work with options("repos") assuming a character vector keeps working if we keep using a character()
• The names attribute naturally supports subsetting, whereas a custom attribute would break