Skip to content

Commit cb434d5

Browse files
david-mollitor-dbIsaac
andcommitted
[SPARK-59507][SQL] Parse interval fractional seconds without building a padded string
`IntervalUtils.parseNanos` right-padded the fractional-second digits to 9 characters with a string concatenation and a substring before parsing: (nanos + "000000000").substring(0, maxNanosLen) That allocated two throwaway strings per call purely to zero-pad ahead of an integer parse. Since the fractional part is guaranteed to be 1-9 ASCII digits by the interval grammar, parse it directly and scale by the corresponding power of ten instead. The result is identical -- the parsed value stays in [0, 999999999], so the range check and error behavior are unchanged -- and no intermediate string is allocated. Co-authored-by: Isaac <no-reply@databricks.com>
1 parent 93d4016 commit cb434d5

1 file changed

Lines changed: 10 additions & 4 deletions

File tree

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/IntervalUtils.scala

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -540,14 +540,20 @@ object IntervalUtils extends SparkIntervalUtils {
540540
}
541541
}
542542

543+
// 10^0 .. 10^8: scales a 1..9 digit fraction up to nanosecond precision without
544+
// building a zero-padded temporary string.
545+
private val nanosMultiplier: Array[Long] =
546+
Array(1L, 10L, 100L, 1000L, 10000L, 100000L, 1000000L, 10000000L, 100000000L)
547+
543548
// Parses a string with nanoseconds, truncates the result and returns microseconds
544549
private def parseNanos(nanos: String, isNegative: Boolean): Long = {
545550
if (nanos != null) {
546551
val maxNanosLen = 9
547-
val alignedStr = if (nanos.length < maxNanosLen) {
548-
(nanos + "000000000").substring(0, maxNanosLen)
549-
} else nanos
550-
val nanoSecond = toLongWithRange(nanosStr, alignedStr, 0L, 999999999L)
552+
// `nanos` is 1..9 digits (per the interval grammar), so parse it directly and scale
553+
// by the missing trailing zeros instead of building a padded string. `raw` is in
554+
// [0, 999999999], and scaling keeps it within that range.
555+
val raw = toLongWithRange(nanosStr, nanos, 0L, 999999999L)
556+
val nanoSecond = raw * nanosMultiplier(maxNanosLen - nanos.length)
551557
val microSecond = nanoSecond / NANOS_PER_MICROS
552558
if (isNegative) -microSecond else microSecond
553559
} else {

0 commit comments

Comments
 (0)