Queues

Queues

Senna supports multiple queues with weighted priority distribution, allowing you to prioritize certain jobs over others.

Queue Configuration

Configure queues when creating a worker:

w, err := worker.New(&worker.Config{
    Redis:     redisConfig,
    Namespace: "myapp",
    Settings: senna.WorkerSettings{
        Queues: []senna.QueueConfig{
            {Name: "critical", Priority: 10},
            {Name: "default", Priority: 5},
            {Name: "low", Priority: 1},
        },
    },
})

Priority Weights

The Priority value determines how often a queue is checked relative to others. Higher priority queues are checked more frequently using weighted round-robin:

Queue	Priority	Check Frequency
critical	10	10x more than low
default	5	5x more than low
low	1	Baseline

With the above configuration:

• critical is checked 10 times for every 1 time low is checked
• default is checked 5 times for every 1 time low is checked

Sending Jobs to Queues

Use WithQueue to send jobs to a specific queue:

// High priority job
c.Enqueue(ctx, "process_payment", args, client.WithQueue("critical"))

// Normal priority (default)
c.Enqueue(ctx, "send_email", args) // Uses default queue

// Low priority
c.Enqueue(ctx, "generate_report", args, client.WithQueue("low"))

Default Queue

If no queue is specified, jobs go to the default queue:

// Client configuration
c, err := client.New(&client.Config{
    Redis:     redisConfig,
    Namespace: "myapp",
    Settings: client.Settings{
        DefaultQueue: "default", // Jobs without WithQueue go here
    },
})

// These are equivalent:
c.Enqueue(ctx, "job_type", args)
c.Enqueue(ctx, "job_type", args, client.WithQueue("default"))

Queue Design Patterns

By Priority

The most common pattern - separate queues for different priority levels:

Queues: []senna.QueueConfig{
    {Name: "critical", Priority: 10},  // Payments, alerts
    {Name: "default", Priority: 5},    // Normal operations
    {Name: "low", Priority: 1},        // Reports, cleanup
}

By Job Type

Separate queues for different job categories:

Queues: []senna.QueueConfig{
    {Name: "emails", Priority: 5},
    {Name: "webhooks", Priority: 5},
    {Name: "reports", Priority: 3},
}

By Customer Tier

Different queues for different customer tiers:

Queues: []senna.QueueConfig{
    {Name: "enterprise", Priority: 10},
    {Name: "pro", Priority: 5},
    {Name: "free", Priority: 1},
}

Dedicated Workers

You can run specialized workers that only process specific queues:

// Worker 1: Only critical jobs
w1, _ := worker.New(&worker.Config{
    Settings: senna.WorkerSettings{
        Concurrency: 20,
        Queues: []senna.QueueConfig{
            {Name: "critical", Priority: 1},
        },
    },
})

// Worker 2: Everything else
w2, _ := worker.New(&worker.Config{
    Settings: senna.WorkerSettings{
        Concurrency: 10,
        Queues: []senna.QueueConfig{
            {Name: "default", Priority: 5},
            {Name: "low", Priority: 1},
        },
    },
})

This is useful for:

• Ensuring critical jobs always have dedicated capacity
• Scaling different job types independently
• Isolating slow jobs from fast jobs

Single Queue Setup

For simple applications, a single queue works fine:

Queues: []senna.QueueConfig{
    {Name: "default", Priority: 1},
}

Queue Storage

Queues are stored in Redis as lists:

namespace:queue:critical   (Redis list)
namespace:queue:default    (Redis list)
namespace:queue:low        (Redis list)
namespace:queues           (Set of active queue names)

Best Practices

1. Start simple: Begin with a single queue and add more as needed

2. Use meaningful names: Queue names should describe their purpose

3. Don't over-segment: Too many queues can be hard to manage

4. Monitor queue depth: High queue depth indicates workers can't keep up

5. Consider dedicated workers: For critical jobs that need guaranteed capacity

Next Steps

• Learn about Batches for grouped job processing
• Add Rate Limiters to control throughput
• Configure Middleware for logging