Skip to content

Flag badly encoded documents #716

Description

@bernhardreiter

Do we detect badly encoded documents and treat them accordingly?

ISDuBA/ISDuBA#1470 has the example document opensuse-su-2025_14737-1.json which contains an invalid characters, so it is not UTF-8 encoded.

Go' json parser changes those characters within strings to the unicode replacement character, but does not raise an error.

  • check if our downloader, checker and validator will accept those documents without warning or error.
  • decide what would be a good behaviour? E.g. download acceptance by default, checker reject by default?
  • estimate and then potentially implement an improvement.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions