Skip to content

SedonaFairSpillPool can reject sort memory even when spill memory is available #918

Description

@camden-lowrance

We're seeing external sort failures where Sedona reports spillable memory is still available, but the allocation is rejected anyway.

The root cause appears to be that SedonaFairSpillPool splits the spill budget across all registered spill-capable consumers, including consumers that are idle and have reserved 0 bytes.

Minimal test case:

#[test]
fn test_idle_spillers_do_not_split_budget() {
    let pool: Arc<dyn MemoryPool> = Arc::new(SedonaFairSpillPool::new(100, 0.0));

    let active_consumer = MemoryConsumer::new(active).with_can_spill(true);
    let mut active = active_consumer.register(&pool);

    let _idle_reservations: Vec<_> = (0..128)
        .map(|i| {
            MemoryConsumer::new(format!(idle-{i}))
                .with_can_spill(true)
                .register(&pool)
        })
        .collect();

    active.try_grow(100).unwrap();
}

On current main, this fails with:

Failed to allocate additional 100 bytes for active with 0 bytes already allocated - maximum available is 0 bytes.
Current unspillable memory usage: 0 bytes, spillable memory available: 100 bytes

Expected behavior: idle spill-capable consumers should not reduce the memory available to the active consumer.

Observed behavior: the active consumer effectively gets a 0-byte limit because the 100-byte pool is divided across 129 registered consumers, even though only one consumer is using memory.

This seems related to sort-heavy queries with many ExternalSorter consumers, where some sorters are idle but still reduce the budget for active sorters.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions