[rust/scheduler] Distributed Scheduler (#2104)

This PR introduces a new module called "scheduler." This module is
responsible for the booking aspect of Cuebot and is designed to offload
this feature from the central module.

Rationale: Cuebot's booking logic depends on responding to each
HostReport with a new task that searches for layers to dispatch to the
reporting host. Consequently, each request generates a
[BookingQuery](https://github.com/AcademySoftwareFoundation/OpenCue/blob/master/cuebot/src/main/java/com/imageworks/spcue/dao/postgres/DispatchQuery.java),
which significantly impacts the database. As a result, scaling Cuebot is
limited by the need to optimize database capacity to handle complex
queries. This new module alleviates the booking workload from Cuebot.

Booking on the Scheduler is not triggered by host reports; instead, it
operates through an internal loop that searches for pending jobs and
seeks suitable matches from a cached view of the hosts in the database.
The scheduler organizes layers and hosts into clusters, with each
cluster representing a group of show and allocation combinations. This
structure allows multiple instances of the scheduler to share the load
without competing for work, which is a significant issue in Cuebot.

To enable Cuebot and the Scheduler to run concurrently without competing
for work, a new feature was added to Cuebot, as detailed in
#2087. This
feature allows for the addition of an exclusion list containing show and
allocations that should not be booked, or it can halt booking for all
shows altogether.

---------

Signed-off-by: Diego Tavares <dtavares@imageworks.com>

Author: DiegoTavares

.gitignore

@@ -27,10 +27,17 @@ cuebot/.project
 /pycue/opencue/compiled_proto/
 /rqd/rqd/compiled_proto/
 docker-compose-local.yml
+/sandbox/kafka*
+/sandbox/zookeeper*
 docs/_site/
 docs/bin/
 sandbox/kafka-data
 sandbox/zookeeper-data
 sandbox/zookeeper-logs
 sandbox/rqd/shots/
 docs/_data/version.yml
+target/*
+
+# Agent files
+AGENTS.md
+CLAUDE.md

rust/.gitignore

@@ -3,7 +3,16 @@
 # TODO: Remove once these crates are stable and ready for public use
 /crates/cuebot-config
 /crates/dist-lock
-/crates/scheduler
 
 .DS_Store
 config/rqd.local_docker.yaml
+/sandbox/kafka*
+/reference
+Cargo.lock
+
+# Localized files only meant for building docker images locally
+proto
+
+# Agent files
+AGENTS.md
+CLAUDE.md