fix(validator): widen Trainer controller readiness timeout

On cold start, the Kubeflow Trainer controller-manager's cert-controller
sidecar provisions its webhook cert via a get-or-create against the API
server; racing that against a not-yet-synced informer cache produces a
resourceVersion conflict on the update. This is expected behavior under
cert-controller's optimistic-concurrency retry, not a defect in Trainer
or in this validator. The sidecar's own reconcile loop retries and
self-heals unassisted.

Each retry adds latency, though, and on a slow cold start the cumulative
delay can push first-ready past the old 2-minute budget, failing the
validator's readiness wait for a controller that was already recovering
on its own. Widen to 3 minutes so the wait accommodates the expected
retry latency instead of racing it.

Signed-off-by: Mike Cook <micook@nvidia.com>

Author: mikecook

pkg/defaults/timeouts.go

@@ -667,7 +667,7 @@ const (
 
 	// TrainerControllerReadyTimeout is the time to wait for the Kubeflow Trainer
 	// controller-manager Deployment to have at least one ready replica after installation.
-	TrainerControllerReadyTimeout = 2 * time.Minute
+	TrainerControllerReadyTimeout = 3 * time.Minute
 
 	// TrainerInstallPollInterval is the sleep between checks that a
 	// recipe-declared Kubeflow Trainer installation has become complete. The