Skip to content

Commit 5be2114

Browse files
chore(treewide): add agents context files
This commit adds 2 context files used by ai agents: - AGENTS.md - general context file containing project overview - integration-test-review/SKILL.md - context file containing instructions for reviewing changes made to integration tests AGENTS.md file placed at repo root is recognized by both opencode and copilot tools. By default, SKILL.md files live under different paths in opencode and copilot tools, but they both support claude SKILL.md paths, so that's why it was placed there. This is the first iteration of trying to use context files in SM repo and their contents will be adjusted after gathering more info on how they do. Fixes https://scylladb.atlassian.net/browse/CLOUD-1174 Refs https://scylladb.atlassian.net/browse/CLOUD-1175 Refs https://scylladb.atlassian.net/browse/CLOUD-1242
1 parent 148c49b commit 5be2114

2 files changed

Lines changed: 165 additions & 0 deletions

File tree

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
---
2+
name: integration-test-review
3+
description: Review changes to the new or already existing integration tests from `*_integration_test.go` files. Do this before making the changes yourself or when asked to review them.
4+
---
5+
6+
## Review integration tests changes
7+
8+
Changes made to integration tests should be reviewed while working on them or when asked to review them.
9+
When working on integration tests, make sure to follow the review comments.
10+
11+
## Review scope
12+
13+
Analyze all connected implementation and integration tests, but focus and generate comments for the ones changed in the review scope.
14+
The review scope can be the current PR scope or a specific set of last commits.
15+
16+
## Integration test requirements
17+
18+
Integration tests should meet the following criteria:
19+
- Test name should start with `Test` followed by service name (if applicable) and end with `Integration` (e.g., `TestBackupSmokeIntegration`)
20+
- They should start with a comment explaining which features are tested
21+
- They should follow the table-driven approach whenever multiple scenarios are tested. Table entry should contain at least the test name and description
22+
- Each tested feature should be covered by a separate test or subtest, unless it's too small to justify such separation
23+
- Tested feature should be tested in a single test or subtest, unless it's not orthogonal to other features. In such cases, both single feature test and combined features tests can coexist
24+
- Test code should be clearly separated to set up and validation stages with the help of helper functions
25+
- Test code should cover the changes made in the review scope
26+
- Test code should follow established best practices
27+
- Test code should be comprehensive, and it should be clear why it succeeds or fails
28+
29+
## Review summary
30+
31+
Review should end with a summary containing the following points:
32+
- whether the integration test requirements are met
33+
- whether the added integration tests cover the changes made in reviewed scope
34+
- whether user facing changes made in reviewed scope are reflected in the documentation
35+
- whether there is a better way of implementing or organizing changed integration tests. If so, propose it
36+
- whether generated review comments must be addressed before merging, or they are optional follow-ups or nitpicks
37+
- when changing existing integration tests, whether the changes didn't decrease test coverage

AGENTS.md

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
# AGENTS.md — Scylla Manager
2+
3+
Scylla Manager (SM) is a management tool for Scylla clusters.
4+
Scylla cluster is a distributed database serving traffic via either CQL or Alternator frontend.
5+
The most important role of SM is to orchestrate repair, backup, restore tasks on managed Scylla cluster.
6+
SM consists of three binaries: `scylla-manager` (SM server), `scylla-manager-agent` (per-node SM agent), and `sctool` (CLI for interacting with SM server).
7+
8+
## Project structure
9+
10+
It consists of 5 go modules:
11+
- the main module (.) containing SM server and SM agent
12+
- backupspec module (./backupspec) containing backup directory specification
13+
- managerclient module (./v3/pkg/managerclient) containing client for SM server
14+
- swagger module (./v3/swagger) containing definitions of used Scylla, SM server, SM agent swagger endpoints
15+
- util module (./v3/pkg/util) containing generic legacy helpers (new helpers are added to ./pkg/util2)
16+
17+
In case changes need to be made across multiple modules, the need to be merged in separate per module PRs.
18+
Modules in go.mod can only reference commits from master branch.
19+
All dependencies for main module (also other go modules from this repo) are vendored in /vendor dir.
20+
Documentation is stored in ./docs/source dir.
21+
22+
### SM server architecture
23+
24+
SM server is built around its scheduler responsible for scheduling and triggering tasks and services responsible
25+
for executing those tasks.
26+
Scheduler is implemented on two levels:
27+
- ./pkg/scheduler - implementation responsible for scheduling tasks at the right time
28+
- ./pkg/service/scheduler - implementation responsible for triggering tasks with all required information
29+
30+
Services are implemented in ./pkg/service and the most important ones are:
31+
- ./pkg/service/backup - responsible for backup task
32+
- ./pkg/service/restore - responsible for restore task
33+
- ./pkg/service/repair - responsible for repair task
34+
35+
The `sctool` CLI is implemented in ./pkg/command.
36+
37+
A single SM server can manage multiple Scylla clusters.
38+
SM server stores cluster info, task definition and progress in a separate Scylla cluster (SM DB).
39+
It's usually local, single node cluster. SM server communicates with SM DB over CQL.
40+
SM server uses those information to start, pause, resume tasks and display their progress.
41+
42+
User interacts with SM server via HTTP/S REST API defined in swagger module.
43+
To do so, user can use the `sctool` CLI or `managerclient` SDK.
44+
45+
SM server communicates with managed Scylla clusters mainly via SM agent proxy. This communication is always required.
46+
In many other, but not all cases, SM server communicates with Scylla cluster directly over CQL.
47+
It requires adding cluster with specified CQL credentials. The same goes for Scylla clusters using Alternator frontend.
48+
49+
### SM agent architecture
50+
51+
SM agent is a small server running on every Scylla node. It serves as a proxy for SM server to Scylla HTTP REST API,
52+
which is exposed only on localhost. Additionally, SM agent also encapsulates rclone server. It is used for managing
53+
backup files on Scylla nodes and in backup locations (e.g., S3, GS, Azure, Minio, etc.).
54+
Communication with SM agent requires special auth token to be present in each request header.
55+
56+
SM agent is implemented in ./pkg/cmd/agent and its rclone component is implemented in ./pkg/rclone.
57+
58+
## General code guidelines
59+
60+
When executing tasks, SM server does not perform any CPU, disk, network intensive operations.
61+
It serves as an orchestrator while Scylla node is doing all the heavy work. In case of backup
62+
and restore, SM agent is also responsible for moving files from backup location, which is network
63+
intensive operation.
64+
The only resource that might be limited to SM server is its memory.
65+
The rule of thumb is that SM server can keep all single node or table scope context needed for
66+
running given task (e.g., entire SM backup manifest with all files listed in it).
67+
Extra caution should be taken when storing entire cluster scope context in memory.
68+
In such cases, it's preferable to change the implementation to a per node or table context,
69+
unless this would result in less optimized usage of intensive Scylla node or SM agent API calls.
70+
71+
SM project is in maintenance state - meaning that no new SM features are added,
72+
but only the changes needed to support new Scylla features are implemented.
73+
SM is also supposed to support all currently supported Scylla versions.
74+
75+
Because of that, when working on SM server codebase, it's important to remember that:
76+
- changes to existing codebase should be minimal and backward compatible
77+
- new code should live mostly in new files and new packages
78+
- new code should optimize the usage of intensive Scylla node and SM agent API calls
79+
- new code should prioritize simplicity and readability over optimizing the usage of lightweight Scylla node and SM agent API calls
80+
- new code shouldn't optimize SM codebase itself
81+
- new code should take into consideration SM server memory consumption
82+
83+
If new code introduces user facing changes, it should also adjust the documentation.
84+
85+
## Build Commands
86+
87+
```bash
88+
make build # Build all three binaries
89+
```
90+
91+
## Test Commands
92+
93+
```bash
94+
# Unit tests (all packages or specific test)
95+
make unit-test
96+
make unit-test RUN=TestName
97+
98+
# Integration tests (per pkg or specific test, require already set up test dev env)
99+
make pkg-integration-test PKG=./pkg/service/backup
100+
make pkg-integration-test PKG=./pkg/service/repair RUN=TestName
101+
```
102+
103+
## Lint and Format
104+
105+
```bash
106+
make check # Full static analysis suite (should pass for every commit)
107+
```
108+
109+
## Development Environment
110+
111+
```bash
112+
make start-dev-env SCYLLA_VERSION=<version> TABLETS=<enabled|disabled> SSL_ENABLED=<true|false> # Start test dev env (6-node Scylla cluster, another 2-node cluster MinIO, etc.)
113+
make run-server SSL_ENABLED=<true|false> # Build and run SM server in test dev env
114+
```
115+
116+
## Testing
117+
118+
Code should be written with both unit tests and integration tests in mind.
119+
Integration tests should follow table-driven approach whenever possible,
120+
have separate setup and test execution stages and focus on comprehensiveness.
121+
For the changes to be approved, the following checks must pass:
122+
- make check
123+
- make unit-test
124+
- make pkg-integration-test PKG=<service path> RUN=<new test name>
125+
126+
## Commit Messages
127+
128+
Changes should be split into commits following Conventional Commits specification.

0 commit comments

Comments
 (0)