Skip to content

utf-8 is incorrect when used in browsers or React Native #369

@ChALkeR

Description

@ChALkeR

In Node.js, UTF-8 decoding is fine

The issue is that in browsers:

  1. Buffer is polyfilled to http://npmjs.com/package/buffer, which is old and invalid
  2. string_decoder is polyfilled to https://npmjs.com/package/string_decoder, which is also old and invalid

And iconv-lite just uses those everywhere without checking

Example:

import iconv from 'iconv-lite'

console.log(escape(iconv.decode(Buffer.of(0xf0, 0x90, 0x80), 'utf8'))) // should be %uFFFD
console.log(escape(iconv.decode(Buffer.of(0xf0, 0x80, 0x80), 'utf8'))) // should be %uFFFD%uFFFD%uFFFD

Correct:

$ node iconv-test.js
%uFFFD
%uFFFD%uFFFD%uFFFD

But when run in browsers or React Native, it outputs:

%uFFFD
%uFFFD

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions