Skip to content

feat: server-side pagination for job list APIs (#193)#436

Draft
Jinghao-coding wants to merge 3 commits into
raids-lab:mainfrom
Jinghao-coding:feat/list-pagination
Draft

feat: server-side pagination for job list APIs (#193)#436
Jinghao-coding wants to merge 3 commits into
raids-lab:mainfrom
Jinghao-coding:feat/list-pagination

Conversation

@Jinghao-coding

@Jinghao-coding Jinghao-coding commented Jun 19, 2026

Copy link
Copy Markdown
Member

Summary / 摘要

Closes #193 — 为作业列表接口引入服务端分页,并把 admin 作业总览页(/admin/jobs)切换为服务端驱动的 DataTable。其余列表页保留客户端模式。

后端完全向后兼容:调用方不传 page / page_size 时,仍然返回老的 JSON 数组形态;任意一个传了,就返回 { items, total, facets? } 的新信封。


中文说明

背景

/v1/vcjobs/v1/vcjobs/all/v1/admin/vcjobs/v1/vcjobs/user/{username} 等作业列表接口此前一次性返回全量数据,作业量大的环境(admin 作业管理页)会出现明显延迟。本 PR 把分页下沉到后端,同时通过 errgroup 并发返回 status / job_type / queue 三个维度的 facet 计数,前端筛选下拉里 Running (12) 这种数字徽标的视觉表现保持不变。

后端改动

  • backend/internal/payload/list.go 新增:
    • ListPageQuery{Page,PageSize,SortBy,Order}Normalize() (offset,limit,order)
    • IsPagingRequested() 判断调用方是否显式开启分页
    • 单页上限 MaxListPageSize = 200
  • backend/internal/resputil/response.goList[T] 结构增加可选 Facets map[string]map[string]int64
  • backend/internal/handler/vcjob/pagination.go(新文件):
    • parseJobFilters 把请求拆成 scope 谓词 + 维度谓词
    • fetchJobsPagederrgroup 并发跑「分页 rows / 总数 / 三维 facet GROUP BY」
    • facet 计数遵循标准 faceted search 语义:算某维度时排除该维度自身的过滤
  • backend/internal/handler/vcjob/vcjob.go:6 个 list / billing handler 共用上面的工具,不传 page / page_size 时退回原有数组返回,避免破坏现有调用方
  • backend/internal/handler/aijob/emias.gobackend/internal/handler/spjob/spjob.go:新增 page / page_size 入参,未传时走老逻辑
  • backend/pkg/aitaskctl/task.go:补 ListByQueuePaged / ListAllPaged
  • 新增 backend/internal/payload/list_test.go 单元测试覆盖 Normalize

前端改动

  • frontend/src/services/types.ts:新增 IListResponse<T>IListQuery
  • frontend/src/services/api/{vcjob,billing}.ts:新增 *Paged 调用方,老函数保留
  • frontend/src/components/query-table/index.tsx:新增可选的 serverDriven 模式
    • 启用 manualPagination / manualSorting / manualFiltering
    • 注入 pageCount / rowCount
    • 用自定义 getFacetedUniqueValues provider 接住后端 facet map,让筛选下拉的 Running (12) 计数继续显示
    • 父组件传入并控制 pagination / sorting / columnFilters / globalFilter 状态
  • frontend/src/components/query-table/pagination.tsx:底部计数改用 table.getRowCount(),server 模式取后端 total,客户端模式仍取过滤后行数
  • frontend/src/components/job/overview/admin-jobs.tsx:迁移到 server-driven 模式;page / sort / filter / search / days 全部下沉到后端请求;billing 子查询用同一份 params;两个查询都开 keepPreviousData 让翻页不闪烁

视觉与体验

  • 表格本体、卡片容器、列定义、分页栏布局、toolbar 完全不变
  • 筛选下拉的 Running (12) 计数徽标保留(由后端 facet 提供)
  • 翻页 / 切换筛选 / 改排序时会触发新请求,配合 keepPreviousData 旧数据先撑住,不闪屏
  • 总条数从「过滤后客户端行数」切换为后端 total

向后兼容性

调用方行为 响应形态
未传 page / page_size data: IJobInfo[](老数组)
传了 page=1&page_size=20 data: { items, total, facets }(新信封)

crater CLI 当前没有 list-jobs 命令,因此不受影响。

范围控制

  • 本 PR 只迁移 admin-jobs(issue list查询分页 #193 直接吐槽的「作业管理」页)
  • custom-jobs.tsx / emias-jobs.tsx / user-jobs.tsx / portal/overview / portal/jobs/inter 仍走客户端模式 + 老接口,依赖 isInteracitveJob 客户端二次过滤;待后端补 interactive boolean 过滤后再迁移

验证步骤

后端:

cd backend
make docs                         # swagger 同步生成
go build ./...                    # 全量编译
./bin/golangci-lint run -c .golangci-full.yml --timeout 5m
go test ./internal/payload/...

前端:

cd frontend
pnpm install --frozen-lockfile
pnpm tsc --noEmit -p tsconfig.json
pnpm eslint <changed files>
pnpm vite build

Test plan

  • 后端起本地服务,访问 /v1/admin/vcjobs?page=1&page_size=20 应返回 { items, total, facets }
  • 同一接口不带 query 应返回老的 IJobInfo[]
  • 前端打开 /admin/jobs:翻页、按 status / jobType / queue 过滤、按 jobName 搜索、按 createdAt 排序,每步在 Network 中应看到带正确 query 的新请求;过滤下拉的计数徽标继续显示
  • 检查未迁移的页面(/portal/overview/portal/jobs/inter、custom-jobs、user-jobs)渲染不变
  • 验证 crater CLI 不受影响

后续

  • 等后端补一个 interactive 过滤后,迁移 custom-jobs.tsx / emias-jobs.tsx
  • 视实际 QPS 评估为 (account_id, created_at)(user_id, created_at)(status)(job_type)(queue) 增加索引

English summary

This PR introduces server-side pagination for job list APIs and migrates the admin job overview page to a server-driven DataTable. The backend stays fully backwards-compatible: requests without page / page_size keep the legacy array response; either field flips the response into { items, total, facets? }. Faceted counts are computed concurrently via errgroup so the toolbar's "Running (12)" badges remain accurate without extra round trips.

Other list pages (custom-jobs, emias-jobs, user-jobs, portal/overview, portal/jobs/inter) intentionally stay on the client-side path until the backend grows an interactive filter; they currently rely on a client-side isInteracitveJob filter that does not have a clean server-side equivalent.

Jinghao-coding added 3 commits June 19, 2026 22:19
Backend (backwards compatible):
- payload: add ListPageQuery + Normalize/IsPagingRequested helpers
- resputil: add optional Facets field to List[T]
- vcjob: rewrite 6 list/billing handlers with errgroup-based concurrent
  rows / total / status+jobType+queue facet queries; legacy callers that
  don't pass page/page_size still get the full array response
- aijob/spjob: accept page/page_size, fall back to legacy ListAll/ListByQueue
- aitaskctl: add ListByQueuePaged / ListAllPaged

Frontend:
- query-table: add serverDriven mode (manual pagination/sorting/filtering,
  custom facetedUniqueValues provider so filter dropdowns keep "Running (12)"
  badges, rowCount injection for the pagination summary)
- vcjob/billing services: add *Paged variants returning IListResponse<T>
- admin-jobs page: opt into serverDriven mode; page/sort/filter/search/days
  are pushed into the backend request, billing query keyed by the same params
- other list pages stay on the legacy client-side path
@Jinghao-coding Jinghao-coding requested a review from Cx330-502 June 19, 2026 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

list查询分页

1 participant