Skip to content

Commit ab9536a

Browse files
committed
Merge remote-tracking branch 'origin/main' into registration-sequences-preview
2 parents 5741f4e + a4c61e8 commit ab9536a

1,021 files changed

Lines changed: 28705 additions & 7731 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
---
2+
name: admin-data-api
3+
description: >-
4+
Pull live production Sidekiq and PgHero status from Bike Index's
5+
OAuth-authenticated AdminData API (`GET /api/admin_data/sidekiq`,
6+
`GET /api/admin_data/pghero` on bikeindex.org) — the same data as the
7+
cookie-gated `/sidekiq` and `/pghero` dashboards, but agent-friendly JSON.
8+
Trigger when the user asks about production queue depth, job backlog,
9+
retries/dead jobs, running Sidekiq processes, or Postgres health
10+
(slow/blocked queries, index usage, unused/invalid indexes, connection
11+
counts, table/db sizes, vacuum/transaction-id danger) — and wants the
12+
*live* answer from production rather than logs or Honeybadger. Also trigger
13+
when a request returns 401/expired and the AdminData token needs
14+
refreshing/re-authorizing. Not for reading log files (use
15+
production-log-inspection) or aggregated exception triage (Honeybadger MCP).
16+
---
17+
18+
# AdminData production API
19+
20+
Two OAuth-authenticated JSON endpoints on production, added in `b323f97`:
21+
22+
- `GET https://bikeindex.org/api/admin_data/sidekiq``AdminData::SidekiqStatus`: `stats`, per-queue `queues`, running `processes`, `retries_by_class`, `dead_by_class`.
23+
- `GET https://bikeindex.org/api/admin_data/pghero``AdminData::PgheroStatus`: `query_stats`, `database_size`, connection/query health, index usage, unused/invalid/duplicate indexes, sequence/txid/autovacuum danger, `settings`, etc. Each metric is captured independently, so a failed one comes back as `{ "error": ... }` in its slot instead of blanking the payload.
24+
25+
Auth is a Bearer token gated on an `admin_data` superuser ability **and** the admin Doorkeeper app. Controller: `app/controllers/api/admin_data_controller.rb`; auth concern: `app/controllers/concerns/api/token_authenticatable.rb`.
26+
27+
All operations go through the helper — run it from the repo root:
28+
29+
```
30+
.claude/skills/admin-data-api/scripts/admin_data.rb <command>
31+
```
32+
33+
## Fetch data
34+
35+
```
36+
.claude/skills/admin-data-api/scripts/admin_data.rb get sidekiq
37+
.claude/skills/admin-data-api/scripts/admin_data.rb get pghero
38+
```
39+
40+
It reads `ADMIN_DATA_TOKEN` from `.env.development`, calls production, and prints `HTTP <status>` then the JSON body. Pipe the body to `jq` for specific fields. Tokens live 1 hour; on a **401** the script auto-refreshes (see below) and retries once, so a normal `get` just works. A **403** means the token's user lacks the `admin_data` superuser ability or the token is from the wrong app. Other non-200s exit non-zero.
41+
42+
Ignore the sidekiq dead set (`dead_size`, `dead_by_class`) — it's a large lifetime accumulation the endpoint caps at `{"too_large": …}`, not actionable here. Don't report it.
43+
44+
### Health-check flow
45+
46+
For a general "how's production" check, use one command:
47+
48+
```
49+
.claude/skills/admin-data-api/scripts/admin_data.rb check
50+
```
51+
52+
It fetches sidekiq then pghero and prints a `summary:` line and an `OK`/`ABNORMAL` verdict for each. Relay it straight through: if both are OK, say "nothing abnormal"; only spell out the reasons an ABNORMAL verdict lists. The verdict logic lives in the script — what it counts as abnormal:
53+
54+
- **Sidekiq**: a queue with `latency > 30` (a real backlog, not transient depth), `size > 400`, or `paused`; `retry_size > 0`; no worker processes; or all workers quiet.
55+
- **PgHero**: a real metric `error` (the disabled-feature `"System stats not enabled"` doesn't count), non-empty `long_running_queries`/`blocked_queries`, a danger metric (`sequence_danger`, `transaction_id_danger`, `autovacuum_danger`), `invalid_indexes`, or `index_hit_rate < 0.90`. Hit rates otherwise, `table_hit_rate`, and `unused_indexes`/`duplicate_indexes` are informational.
56+
57+
## Refreshing the token
58+
59+
`.env.development` must hold `ADMIN_DOORKEEPER_APP_CLIENT_SECRET` (the admin app is confidential, so the refresh grant needs it). With it, `get` refreshes automatically on a 401 — you rarely call this directly. To force a refresh:
60+
61+
```
62+
.claude/skills/admin-data-api/scripts/admin_data.rb refresh
63+
```
64+
65+
It POSTs the `refresh_token` grant and writes the new `ADMIN_DATA_TOKEN` + `ADMIN_DATA_REFRESH` into `.env.development` (values never printed).
66+
67+
### First-time setup / dead refresh token (browser flow)
68+
69+
Needed only when there's no token yet, or the refresh token itself was revoked (refresh reports a failure):
70+
71+
1. Print the authorize URL (fills in `ADMIN_DOORKEEPER_APP_CLIENT_ID`):
72+
```
73+
.claude/skills/admin-data-api/scripts/admin_data.rb authorize-url
74+
```
75+
2. Ask the user to open it and approve. Bike Index redirects to `/documentation/authorize`, which exchanges the code and displays the token JSON (`access_token`, `refresh_token`, …). Ask the user to paste that response back.
76+
3. Store both values:
77+
```
78+
.claude/skills/admin-data-api/scripts/admin_data.rb set-tokens <access_token> <refresh_token>
79+
```
80+
81+
The authorization code expires 10 minutes after the page loads — if it shows an error, have the user reload the authorize URL.
82+
83+
## Notes
84+
85+
- These hit **production** with a superuser token — read-only status only; never a substitute for the log or Honeybadger workflows for their jobs.
86+
- `.env.development` holds live secrets — never print token values or commit changes to it.
Lines changed: 196 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,196 @@
1+
#!/usr/bin/env ruby
2+
# frozen_string_literal: true
3+
4+
#
5+
# Helper for the AdminData production API (Sidekiq / PgHero status).
6+
# Reads/writes token values in .env.development (located relative to this script,
7+
# so it works from any cwd). Run it directly, e.g.
8+
# .claude/skills/admin-data-api/scripts/admin_data.rb check
9+
#
10+
# Requires in .env.development: ADMIN_DOORKEEPER_APP_CLIENT_ID,
11+
# ADMIN_DOORKEEPER_APP_CLIENT_SECRET, and (after first authorize) ADMIN_DATA_TOKEN
12+
# + ADMIN_DATA_REFRESH. Tokens auto-refresh on a 401 using the secret.
13+
14+
require "net/http"
15+
require "uri"
16+
require "json"
17+
require "dotenv"
18+
19+
BASE = "https://bikeindex.org"
20+
REPO_ROOT = File.expand_path("../../../..", __dir__)
21+
ENV_FILE = File.join(REPO_ROOT, ".env.development")
22+
23+
def env_get(key)
24+
return nil unless File.exist?(ENV_FILE)
25+
26+
Dotenv.parse(ENV_FILE)[key]
27+
end
28+
29+
# Upsert KEY=VALUE in .env.development, preserving the rest of the file. dotenv is
30+
# read-only (no file writer), so the token write-back stays an in-place line edit.
31+
def env_set(key, value)
32+
content = File.exist?(ENV_FILE) ? File.read(ENV_FILE) : ""
33+
if content.match?(/^#{Regexp.escape(key)}=.*$/)
34+
content = content.sub(/^#{Regexp.escape(key)}=.*$/) { "#{key}=#{value}" }
35+
else
36+
content += "\n" unless content.empty? || content.end_with?("\n")
37+
content += "#{key}=#{value}\n"
38+
end
39+
File.write(ENV_FILE, content)
40+
end
41+
42+
def request(method, url, headers: {}, form: nil)
43+
uri = URI(url)
44+
http = Net::HTTP.new(uri.host, uri.port)
45+
http.use_ssl = uri.scheme == "https"
46+
req = (method == :post) ? Net::HTTP::Post.new(uri) : Net::HTTP::Get.new(uri)
47+
headers.each { |k, v| req[k] = v }
48+
req.set_form_data(form) if form
49+
http.request(req)
50+
end
51+
52+
# Exchange ADMIN_DATA_REFRESH for a new token pair and store it. Returns true on success.
53+
def refresh_token!
54+
client_id = env_get("ADMIN_DOORKEEPER_APP_CLIENT_ID")
55+
secret = env_get("ADMIN_DOORKEEPER_APP_CLIENT_SECRET")
56+
refresh = env_get("ADMIN_DATA_REFRESH")
57+
return warn_false("ADMIN_DOORKEEPER_APP_CLIENT_SECRET missing from #{ENV_FILE}") if secret.to_s.empty?
58+
return warn_false("ADMIN_DATA_REFRESH missing — run the browser authorize flow (see SKILL.md)") if refresh.to_s.empty?
59+
60+
res = request(:post, "#{BASE}/oauth/token", form: {
61+
"grant_type" => "refresh_token", "refresh_token" => refresh,
62+
"client_id" => client_id, "client_secret" => secret
63+
})
64+
data = begin
65+
JSON.parse(res.body)
66+
rescue
67+
{}
68+
end
69+
access = data["access_token"].to_s
70+
new_refresh = data["refresh_token"].to_s
71+
if access.empty? || new_refresh.empty?
72+
warn "Refresh failed: #{data["error_description"] || data["error"] || res.body}"
73+
return warn_false("The refresh token may be revoked — run the browser authorize flow (see SKILL.md).")
74+
end
75+
env_set("ADMIN_DATA_TOKEN", access)
76+
env_set("ADMIN_DATA_REFRESH", new_refresh)
77+
warn "Refreshed ADMIN_DATA_TOKEN and ADMIN_DATA_REFRESH in #{ENV_FILE}"
78+
true
79+
end
80+
81+
def warn_false(message)
82+
warn message
83+
false
84+
end
85+
86+
# GET the endpoint with the current token. Returns [status(Integer or nil), body].
87+
def get_status(endpoint)
88+
token = env_get("ADMIN_DATA_TOKEN")
89+
if token.to_s.empty?
90+
warn "ADMIN_DATA_TOKEN missing from #{ENV_FILE} — run the authorize flow (see SKILL.md)"
91+
return [nil, nil]
92+
end
93+
res = request(:get, "#{BASE}/api/admin_data/#{endpoint}", headers: {"Authorization" => "Bearer #{token}"})
94+
warn "HTTP #{res.code}"
95+
[res.code.to_i, res.body]
96+
end
97+
98+
# get_status with a one-shot refresh + retry on failure (typically a 401). Returns body or nil.
99+
def get_or_refresh(endpoint)
100+
status, body = get_status(endpoint)
101+
return body if status == 200
102+
103+
warn "Token rejected — refreshing and retrying…" if status
104+
return nil unless refresh_token!
105+
106+
status, body = get_status(endpoint)
107+
(status == 200) ? body : nil
108+
end
109+
110+
def array_length(value)
111+
value.is_a?(Array) ? value.length : 0
112+
end
113+
114+
# "nothing abnormal" verdicts. A queue backlogs at latency > 30s or size > 400 (not
115+
# transient depth). PgHero hit rates and unused/duplicate indexes are informational,
116+
# and "System stats not enabled" is a disabled feature, not a fault.
117+
def sidekiq_verdict(data)
118+
stats = data["stats"] || {}
119+
processes = data["processes"] || []
120+
reasons = []
121+
(data["queues"] || []).each do |q|
122+
next unless q["latency"].to_f > 30 || q["size"].to_i > 400 || q["paused"]
123+
124+
reasons << "queue #{q["name"]} size=#{q["size"]} latency=#{q["latency"]}#{" PAUSED" if q["paused"]}"
125+
end
126+
reasons << "retry_size=#{stats["retry_size"]}" if stats["retry_size"].to_i > 0
127+
if processes.empty?
128+
reasons << "no worker processes"
129+
elsif processes.all? { |p| p["quiet"] }
130+
reasons << "all workers quiet"
131+
end
132+
<<~OUT.chomp
133+
summary: enqueued=#{stats["enqueued"]} retry=#{stats["retry_size"]} scheduled=#{stats["scheduled_size"]} processes=#{processes.length}
134+
#{verdict_line(reasons)}
135+
OUT
136+
end
137+
138+
def pghero_verdict(data)
139+
reasons = []
140+
data.each do |key, value|
141+
next unless value.is_a?(Hash) && value.key?("error") && value["error"] != "System stats not enabled"
142+
143+
reasons << "#{key}: #{value["error"]}"
144+
end
145+
reasons << "long_running_queries=#{array_length(data["long_running_queries"])}" if array_length(data["long_running_queries"]) > 0
146+
reasons << "blocked_queries=#{array_length(data["blocked_queries"])}" if array_length(data["blocked_queries"]) > 0
147+
reasons << "invalid_indexes=#{array_length(data["invalid_indexes"])}" if array_length(data["invalid_indexes"]) > 0
148+
reasons << "sequence_danger=#{array_length(data["sequence_danger"])}" if array_length(data["sequence_danger"]) > 0
149+
reasons << "transaction_id_danger" if array_length(data["transaction_id_danger"]) > 0
150+
reasons << "autovacuum_danger=#{array_length(data["autovacuum_danger"])}" if array_length(data["autovacuum_danger"]) > 0
151+
reasons << "index_hit_rate=#{data["index_hit_rate"]}" if data["index_hit_rate"] && data["index_hit_rate"].to_f < 0.90
152+
max_conn = (data["settings"] || {})["max_connections"] || "?"
153+
<<~OUT.chomp
154+
summary: connections=#{data["total_connections"]}/#{max_conn} db=#{data["database_size"]} running=#{array_length(data["running_queries"])} index_hit=#{data["index_hit_rate"].to_s[0, 5]} table_hit(info)=#{data["table_hit_rate"].to_s[0, 5]} unused_indexes=#{array_length(data["unused_indexes"])}
155+
#{verdict_line(reasons)}
156+
OUT
157+
end
158+
159+
def verdict_line(reasons)
160+
reasons.empty? ? "verdict: OK — nothing abnormal" : "verdict: ABNORMAL — #{reasons.join("; ")}"
161+
end
162+
163+
case ARGV[0]
164+
when "authorize-url"
165+
client_id = env_get("ADMIN_DOORKEEPER_APP_CLIENT_ID")
166+
abort "ADMIN_DOORKEEPER_APP_CLIENT_ID missing from #{ENV_FILE}" if client_id.to_s.empty?
167+
redirect = "https%3A%2F%2Fbikeindex.org%2Fdocumentation%2Fauthorize"
168+
puts "#{BASE}/oauth/authorize?client_id=#{client_id}&redirect_uri=#{redirect}&response_type=code&scope=public"
169+
170+
when "get" # get <sidekiq|pghero> — auto-refreshes and retries once on a 401
171+
endpoint = ARGV[1] or abort("usage: get <sidekiq|pghero>")
172+
body = get_or_refresh(endpoint) or exit(22)
173+
puts body
174+
175+
when "check" # full health check: sidekiq, then pghero — summary + OK/ABNORMAL verdict each
176+
puts "== SIDEKIQ =="
177+
body = get_or_refresh("sidekiq") or exit(22)
178+
puts sidekiq_verdict(JSON.parse(body))
179+
puts "\n== PGHERO =="
180+
body = get_or_refresh("pghero") or exit(22)
181+
puts pghero_verdict(JSON.parse(body))
182+
183+
when "set-tokens" # set-tokens <access_token> <refresh_token> — for the browser authorize flow
184+
access = ARGV[1] or abort("access_token required")
185+
refresh = ARGV[2] or abort("refresh_token required")
186+
env_set("ADMIN_DATA_TOKEN", access)
187+
env_set("ADMIN_DATA_REFRESH", refresh)
188+
puts "Updated ADMIN_DATA_TOKEN and ADMIN_DATA_REFRESH in #{ENV_FILE}"
189+
190+
when "refresh" # refresh the token pair now (needs ADMIN_DOORKEEPER_APP_CLIENT_SECRET)
191+
exit(refresh_token! ? 0 : 1)
192+
193+
else
194+
warn "usage: admin_data.rb {check | get <sidekiq|pghero> | authorize-url | set-tokens <access> <refresh> | refresh}"
195+
exit 64
196+
end

0 commit comments

Comments
 (0)