CBOR `UByteArraySerializer` encodes as signed bytes

[twyatt]

Describe the bug
The CBOR UByteArraySerializer encodes as signed bytes rather than unsigned. Only tested with UByteArraySerializer but may affect other unsigned encoding.

To Reproduce

Cbor.encodeToHexString(UByteArraySerializer(), ubyteArrayOf(208u))
    .also(::println)

Expected behavior

9f18d0ff

Which cbor.me shows as:

9F       # array(*)
   18 D0 # unsigned(208)
   FF    # primitive(*)

Actual behavior

9f382fff

Which cbor.me shows as:

9F       # array(*)
   38 2F # negative(47)
   FF    # primitive(*)

Environment

• Kotlin version: 2.3.20
• Library version: 1.11.0
• Kotlin platforms tested on: JVM, JS, MacOS ARM64, WASM
• Gradle version: 9.4.0

[sandwwraith]

Was it also the case in previous versions (e.g. 1.10.0)?

[twyatt]

Was it also the case in previous versions (e.g. 1.10.0)?

Yes

Spinning up a quick project to check, looks like this has been the behavior ever since 1.4.1 (when kotlinx.serialization.builtins.UByteArraySerializer became available).

[twyatt]

As a workaround, we've been using the following in our projects:

/**
 * kotlinx.serialization provides a builtin serializer for `UByteArray` but as of 1.10.0 it expects entries
 * to have a minimum value of -128 and a maximum value of 127 (inclusive).
 *
 * See https://github.com/Kotlin/kotlinx.serialization/issues/3143 for more details.
 * Once the issue is resolved, this serializer should be removed.
 */
public object UByteArraySerializer : KSerializer<UByteArray> {

    private val listSerializer = ListSerializer(UByteSerializer)

    override val descriptor: SerialDescriptor
        get() = listSerializer.descriptor

    override fun serialize(encoder: Encoder, value: UByteArray) {
        listSerializer.serialize(encoder, value.toList())
    }

    override fun deserialize(decoder: Decoder): UByteArray {
        val list = listSerializer.deserialize(decoder)
        return UByteArray(list.size) { list[it] }
    }
}

private object UByteSerializer : KSerializer<UByte> {

    override val descriptor: SerialDescriptor
        get() = UByte.serializer().descriptor

    override fun deserialize(decoder: Decoder): UByte {
        // We can decode as a `Short` (to read a `UByte`) because CBOR decoding will properly read the next **number**
        // (regardless of size):
        // https://github.com/Kotlin/kotlinx.serialization/blob/v1.10.0/formats/cbor/commonMain/src/kotlinx/serialization/cbor/internal/Decoder.kt#L132-L137
        //
        // It isn't until after reading a **number** that it validates that it falls within expected range:
        // https://github.com/Kotlin/kotlinx.serialization/blob/v1.10.0/formats/cbor/commonMain/src/kotlinx/serialization/cbor/internal/Decoder.kt#L356-L362
        val number = decoder.decodeShort()

        val from = UByte.MIN_VALUE.toShort()
        val to = UByte.MAX_VALUE.toShort()
        if (number !in from..to) {
            throw SerializationException("Decoded number $number is not within the range for type UByte ([$from..$to])")
        }
        return number.toUByte()
    }

    override fun serialize(encoder: Encoder, value: UByte) {
        // We can encode as a `Short` (to write a `UByte`) because encoding will properly write the next **number**
        // (regardless of size):
        // https://github.com/Kotlin/kotlinx.serialization/blob/v1.10.0/formats/cbor/commonMain/src/kotlinx/serialization/cbor/internal/Encoder.kt#L81-L97
        // https://github.com/Kotlin/kotlinx.serialization/blob/v1.10.0/formats/json/commonMain/src/kotlinx/serialization/json/internal/StreamingJsonEncoder.kt#L190-L204
        encoder.encodeShort(value.toShort())
    }
}

[fzhinkin]

#3194 fixed the problem for both scalar unsigned numbers and corresponding array types.