feat: add Prometheus metrics for operator reconciliation

[leecalcote]

fDescription

This PR fixes #779.

Adds reconciliation observability to the operator and continues the work started in #780 by @Chaithanya5gif, rebased onto the post-Kubebuilder-go/v4 (WS-1) layout per the request on that PR.

A new pkg/metrics package registers three metrics with the controller-runtime registry, so they are exposed automatically on the manager's existing metrics endpoint (default :8080/metrics) with no extra wiring:

• meshery_operator_reconcile_total{controller}
• meshery_operator_reconcile_errors_total{controller}
• meshery_operator_reconcile_duration_seconds{controller}

The BrokerReconciler and MeshSyncReconciler are instrumented via a deferred closure that records the count and duration on every reconcile and increments the error counter only when a non-nil error is returned.

Notes for Reviewers

Carried over from #780 (metric naming and the defer-based timing pattern), with the following review changes applied:

• Dropped the Makefile and CONTRIBUTING.md changes from feat: add Prometheus metrics for operator reconciliation #780 — WS-1 already provides the envtest-aware make test target and contributor docs (docs/testing.md), so those edits are now obsolete and would conflict.
• Named return values (result, reconcileErr) keep the instrumentation in a single defer at the top of Reconcile instead of rewriting every return site, minimizing churn in the control flow.
• Lint-clean: extracted controller-name and label constants to satisfy goconst, grouped imports for gci/goimports, added the Apache license headers, and renamed an inner result to avoid shadowing under govet's shadow analyzer.
• Tests assert all three metrics are registered with the controller-runtime registry (the acceptance criterion) and that the counters and histogram record values.

Verified locally: gofmt, go vet, go build ./..., golangci-lint v2.12.2 run ./... (0 issues), and the full unit/envtest suite (make test) all pass.

Signed commits

• Yes, I signed my commits.

[pontusringblom]

/gemini review

[Copilot]

Pull request overview

This PR adds first-class Prometheus reconciliation observability to Meshery Operator by defining and registering controller-runtime metrics and instrumenting the Broker and MeshSync controllers to record reconcile counts, errors, and latency.

Changes:

• Introduces a new pkg/metrics package that registers reconcile metrics with the controller-runtime metrics registry (auto-exposed on the manager metrics endpoint).
• Instruments BrokerReconciler and MeshSyncReconciler reconcile loops using a single top-level defer to consistently record total reconciles, duration, and error counts.
• Adds unit tests validating metric registration and basic counter/histogram behavior; updates go.mod to add a direct Prometheus client dependency.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
pkg/metrics/metrics.go	Defines and registers reconcile counters/histogram with controller-runtime’s metrics registry.
pkg/metrics/metrics_test.go	Verifies metrics are registered and that counters/histogram accept observations.
controllers/broker_controller.go	Adds deferred reconcile instrumentation (total/errors/duration) to Broker controller.
controllers/meshsync_controller.go	Adds deferred reconcile instrumentation (total/errors/duration) to MeshSync controller.
go.mod	Adds prometheus/client_golang as a direct dependency and updates indirect deps accordingly.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.