`toJSON()` enters infinite recursion for `labelled` vectors

[gadenbuie]

Reported by a user in rstudio/htmltools#398

A very minimal reprex:

labelled::labelled(1:3) |> jsonlite::toJSON()
#> Error: C stack usage  7955752 is too close to the limit

[mculbert]

I ran into this issue, too. I think this is related to the fact that the haven_labelled class inherits from vctrs_vctr.

> x <- haven::labelled(1:10)
> class(x)
[1] "haven_labelled" "vctrs_vctr"     "integer"
> toJSON(x)
Error: C stack usage  7971200 is too close to the limit

As far as I can tell, the asJSON() method for vctrs_vctr removes the vctrs_vctr entry from the class list and then tries to re-dispatch on the amended class list, but when R tries to dispatch a method for c('haven_labelled', 'integer'), it selects the same vctrs_vctr method, causing the infinite loop:

> selectMethod('asJSON', c('haven_labelled', 'integer'))
Method Definition:

function (x, ...)
{
    class(x) <- setdiff(class(x), "vctrs_vctr")
    asJSON(x, ...)
}
<bytecode: 0x7fc74c54eb38>
<environment: namespace:jsonlite>

Signatures:
        x               
target  "haven_labelled"
defined "vctrs_vctr"  

I don't know enough about S3 classes to follow exactly what's going on here, since jsonlite doesn't explicitly register a method for haven_labelled. haven calls setOldClass(c("haven_labelled", "vctrs_vctr")), so maybe that tells R about the inheritance structure and results in the redispatch to asJSON.vctrs_vctr?

In any case, perhaps the fix is in how to pop vctrs_vctr from the class list. Instead of setdiff(class(x), 'vctrs_vctr'), which leaves the subclass (haven_labelled, in this case) in the class list, perhaps something like:

class(x) <- class(x)[-(1:which(class(x) == "vctrs_vctr"))]

which, I believe, would remove vctrs_vctr and all subclasses.

Once I discovered that it was a haven_labelled variable that was causing the infinite loop, it was easy enough for me to convert it to a regular numeric vector before conversion to JSON, but it took a while to figure out that was the source of the issue (I wasn't calling toJSON() directly, and the haven_labelled variable was in a nested data structure—I didn't even initially realize that I wasn't using regular vectors...), and others might not realize the problem when they get an unexpected and unexplained infinite recursion.

Thanks!

[jeroen]

Hmm we put that workaround in place to address #408