Skip to content

JpegEncoderTests are not reliable #1550

@antonfirsov

Description

@antonfirsov

Prerequisites

  • I have written a descriptive issue title
  • I have verified that I am running the latest version of ImageSharp
  • I have verified if the problem exist in both DEBUG and RELEASE mode
  • I have searched open and closed issues to ensure it has not already been reported

Description

Our current validation logic in JpegEncoderTests is too tolerant, which caused #1549 to go through the quality gate completely unnoticed. The method for calculating the tolerance percentage is result of trial-and error experiments (dealing with platform differences if memory serves well):

/// <summary>
/// Anton's SUPER-SCIENTIFIC tolerance threshold calculation
/// </summary>
private static ImageComparer GetComparer(int quality, JpegSubsample subsample)
{
float tolerance = 0.015f; // ~1.5%
if (quality < 50)
{
tolerance *= 10f;
}
else if (quality < 75 || subsample == JpegSubsample.Ratio420)
{
tolerance *= 5f;
if (subsample == JpegSubsample.Ratio420)
{
tolerance *= 2f;
}
}
return ImageComparer.Tolerant(tolerance);
}

The method above returns a comparer of 15% tolerance, for subsample=420, quality=100, which doesn't really make sense. The image in #1549 (comment) has ~4% difference compared to the image before encoding.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions