[Proposal] Extend KnownFPClass with unit-Interval refinement flags

[MaxGraey]

Motivation

KnownFPClass provides a lightweight classification of floating-point values (NaN, Inf, normal, subnormal, zero), but it lacks the ability to express common, high-value numeric constraints such as:

exact constants: x == ±1.0
bounded ranges: x ∈ (0,1) or x ∈ (-1,0)

Note: (0, 1 + machine_eps) -> (0, 1]

At the same time for example ConstantRange doesn't support floating-point types.

Proposal

Introduce the following refinement flags for KnownFPClass and FPClassTest:

fcPosOne     // x == +1.0
fcNegOne     // x == -1.0
fcPosSubOne  // x ∈ (0, +1)
fcNegSubOne  // x ∈ (-1, 0)

Or Unit instead One suffix.

Possible additional flags:

fcPosEps
fcNegEps
fcPosSubEps
fcNegSubEps
fcPosMaximum
fcNegMaximum

And composed flags:

fcOne = fcNegOne | fcPosOne,                       // -1.0 or 1.0
fcSubOne = fcNegSubOne | fcPosSubOne,              // x ∈ (-1, 0) U (0 1)
fcSubOneFull = fcNegSubOne | fcPosSubOne | fcZero, // x ∈ (-1, 1)
fcSubOneClosed = fcSubOneFull | fcOne,             // x ∈ [-1, 1]

Use Cases

Fractional part patterns

x - trunc(x)
x - floor(x)
x - ceil(x)

These are guaranteed to be in:

[0, 1)  -->  fcPosZero | fcPosSubOne

And unlock enables better simplification of comparisons around ±1.0, clamps, min/max, math functions often has bound checks in such intervals etc.

Here is a real case:

%i = tail call float @llvm.floor.f32(float %x)
%sub.i.i = fsub nsz float %x, %i ;; sub.i.i ∈ [0, 1.0)
%i1 = tail call nsz float @llvm.minnum.f32(float %sub.i.i, float 0x3FEFFFFFE0000000)

which could be simplified with this proposal to:

%i = tail call float @llvm.floor.f32(float %x)
%i1 = fsub nsz float %x, %i
;; get rid of @llvm.minnum.f32 due to min(x, ~ 1.0) -> x, when we know x ∈ [0, 1.0)

WDYT? Does it make sense?

[RKSimon]

CC @arsenm @dtcxzyw

[dtcxzyw]

We have limited support for fp ranges with ConstantFPRange. It should be good enough to eliminate redundant fmin/fmax expressions. Unfortunately the SCCP/CVP support was blocked by a huge compile-time impact: #111544.

+/-1.0 are special values that allow us to refine the analysis result further when processing fmul/fdiv instructions. I am ok with adding new flags to knownfpclass, but not sure if it brings a significant maintenance burden. Modifying FPClassTest might not be feasible, as it breaks semantics of llvm.is.fpclass (and clang intrinsic __builtin_isfpclass).

[MaxGraey]

Unfortunately the SCCP/CVP support was blocked by a huge compile-time impact:

Yes, the main idea is to simplify the analysis as much as possible, making it similar to fcZero and fcSubnormal. At the same time, dividing into the subdomains [0, 1] and (1, +∞) is quite practical in many aspects.

Modifying FPClassTest might not be feasible, as it breaks semantics of llvm.is.fpclass (and clang intrinsic __builtin_isfpclass).

I've been thinking about this. There might be a few options here.

We could add these flags to FPClassTest, but create a placeholder/divider flag (for fast range flag detects) and mark these extra flags as pseudo/internal and does not expose to the llvm.is.fpclass / __builtin_isfpclass bit flags at all.

Pros: Minimal refactoring
Cons: I'm not sure; it probably isn't very semantically clear and might cause confusion to someone.

Or create a new superset enum that aggregate FPClassTest as is and the additional flags. And replace all FPClassTest usages to this new enum

Cons: much more refactoring

[arsenm]

The current mask value exactly matches llvm.is.fpclass's mask format, which we don't want to change. This should be some kind of extended, separate thing which doesn't break the existing mask. The special values could maybe go into the KnownFPClass struct

[MaxGraey]

Something like this?

enum class FPClassRefinedTest : unsigned {
  fcPosOne    = 0x0400, // continue from last of FPClassTest or reset?
  fcPosSubOne = 0x0800,
  fcNegOne    = 0x1000,
  fcNegSubOne = 0x2000,
};

class KnownFPClass {
  ...
  LLVM_DECLARE_ENUM_AS_BITMASK(FPClassTest, /* LargestValue */ fcPosInf);
  LLVM_DECLARE_ENUM_AS_BITMASK(FPClassRefinedTest, /* LargestValue */ fcNegSubOne);
};

[MaxGraey]

The thing is, it would be useful to bit union the existing fcZero with the newfcSubOne, for example. And to do that, it seems like it would make sense to do something like this:

enum class FPClassTest : unsigned {
  fcNone,
  ...
}

enum class FPClassRefinedTest : unsigned {
  None = fcNone,

  SNan = fcSNan,
  QNan = fcQNan,
  NegInf = fcNegInf,
  NegNormal = fcNegNormal,
  NegSubnormal = fcNegSubnormal,
  NegZero = fcNegZero,
  PosZero = fcPosZero,
  PosSubnormal = fcPosSubnormal,
  PosNormal = fcPosNormal,
  PosInf = fcPosInf,

  PosOne    = 0x0400,
  PosSubOne = 0x0800,
  NegOne    = 0x1000,
  NegSubOne = 0x2000,
};

class KnownFPClass {
  ...
  LLVM_DECLARE_ENUM_AS_BITMASK(FPClassTest, /* LargestValue */ fcPosInf);
  LLVM_DECLARE_ENUM_AS_BITMASK(FPClassRefinedTest, /* LargestValue */ fcNegSubOne);
};

and later use FPClassRefinedTest for all methods related to KnownFPClass and static helpers