When the interquartile range (IQR) of the observed data is zero, the current RobustScaler falls back to minimum_scale (default is 1e-10). This produces extreme scaled values when the IQR is 0 but there is still some variance. For example, with the data [10, 10, 10, 10, 12] (and median of 10, IQR of 0), the scaler subtracts 10 and divides by 1e-10, resulting in [0, 0, 0, 0, 2e10]. This in turn can cause the model to output near-zero values.
Proposed Fix
Instead of always falling back to minimum_scale when IQR is zero, compute the full range (max - min) and use that as a scale if the IQR is zero. This approach gives a more reasonable scaling. For the same example, that would yield a scale of 2 and final outputs of [0, 0, 0, 0, 1].
Steps to reproduce
- Call
RobustScaler with data [10, 10, 10, 10, 12] and weights of [1, 1, 1, 1, 1].
- Observe the output scaled values:
[0, 0, 0, 0, 2e10].
Expected behaviour
The scaler should produce outputs [0, 0, 0, 0, 1] by relying on a sensible fallback (the full range) when the IQR is zero.
When the interquartile range (IQR) of the observed data is zero, the current
RobustScalerfalls back tominimum_scale(default is1e-10). This produces extreme scaled values when the IQR is 0 but there is still some variance. For example, with the data[10, 10, 10, 10, 12](and median of 10, IQR of 0), the scaler subtracts 10 and divides by1e-10, resulting in[0, 0, 0, 0, 2e10]. This in turn can cause the model to output near-zero values.Proposed Fix
Instead of always falling back to
minimum_scalewhen IQR is zero, compute the full range (max - min) and use that as a scale if the IQR is zero. This approach gives a more reasonable scaling. For the same example, that would yield a scale of 2 and final outputs of[0, 0, 0, 0, 1].Steps to reproduce
RobustScalerwith data[10, 10, 10, 10, 12]and weights of[1, 1, 1, 1, 1].[0, 0, 0, 0, 2e10].Expected behaviour
The scaler should produce outputs
[0, 0, 0, 0, 1]by relying on a sensible fallback (the full range) when the IQR is zero.