|
| 1 | +# Contributing to SOC Dashboard |
| 2 | + |
| 3 | +SOC Dashboard is an open-source Flask + PostgreSQL SOC analytics dashboard, and contributions are welcome. Whether you are fixing a bug, adding a new KPI, improving the frontend, or tightening up the docs — this guide has everything you need to get started. |
| 4 | + |
| 5 | +## Ways to contribute |
| 6 | + |
| 7 | +- **Bug reports** — something behaves incorrectly or throws an unexpected error |
| 8 | +- **New KPI widgets** — additional SOC metrics beyond MTTR, SLA breach rate, and escalation rate |
| 9 | +- **New chart types** — additional Chart.js visualisations driven by `/api/stats` |
| 10 | +- **New API endpoints** — extending the REST surface (e.g. alert assignment, bulk actions) |
| 11 | +- **Frontend improvements** — UX, filtering, accessibility, responsiveness |
| 12 | +- **Database schema changes** — new tables or columns that extend the data model |
| 13 | +- **Security hardening** — see [SECURITY.md](SECURITY.md) for vulnerability reports |
| 14 | +- **Documentation** — clearer setup steps, architecture notes, API examples |
| 15 | + |
| 16 | +## Getting started |
| 17 | + |
| 18 | +**Prerequisites:** Python 3.12+, PostgreSQL 14+. |
| 19 | + |
| 20 | +```bash |
| 21 | +# 1. Clone and create a virtual environment |
| 22 | +git clone https://github.com/Romil2112/SOC-Dashboard.git |
| 23 | +cd SOC-Dashboard |
| 24 | +python3 -m venv .venv |
| 25 | +source .venv/bin/activate # Windows: .venv\Scripts\activate |
| 26 | + |
| 27 | +# 2. Install runtime + dev dependencies |
| 28 | +pip install -r requirements-dev.txt |
| 29 | + |
| 30 | +# 3. Create and initialise the database |
| 31 | +createdb soc_dashboard |
| 32 | +psql soc_dashboard -f schema.sql |
| 33 | + |
| 34 | +# 4. Seed 50 demo alerts |
| 35 | +python seed.py |
| 36 | + |
| 37 | +# 5. Configure secrets |
| 38 | +cp .env.example .env |
| 39 | +# Open .env and fill in: |
| 40 | +# FLASK_SECRET_KEY — generate with: python -c "import secrets; print(secrets.token_hex(32))" |
| 41 | +# ALERTS_API_KEY — same command; used by POST /api/alerts |
| 42 | + |
| 43 | +# 6. Create a local analyst account (no self-registration in the app) |
| 44 | +python manage.py create-user alice 'your-passphrase' --role analyst |
| 45 | + |
| 46 | +# 7. Start the dev server |
| 47 | +python app.py |
| 48 | +# → http://localhost:8000 |
| 49 | +``` |
| 50 | + |
| 51 | +### Docker alternative |
| 52 | + |
| 53 | +```bash |
| 54 | +cp .env.example .env # set FLASK_SECRET_KEY and ALERTS_API_KEY |
| 55 | +docker compose up |
| 56 | +``` |
| 57 | + |
| 58 | +`docker-compose.yml` starts PostgreSQL, applies `schema.sql`, and starts Flask — no separate database setup needed. |
| 59 | + |
| 60 | +### Test database |
| 61 | + |
| 62 | +Tests need their own throwaway database. Spin one up with Docker: |
| 63 | + |
| 64 | +```bash |
| 65 | +docker run -d --name soc-pg \ |
| 66 | + -e POSTGRES_PASSWORD=postgres \ |
| 67 | + -e POSTGRES_DB=soc_test \ |
| 68 | + -p 5433:5432 postgres:16 |
| 69 | + |
| 70 | +export DATABASE_URL=postgresql://postgres:postgres@localhost:5433/soc_test |
| 71 | +``` |
| 72 | + |
| 73 | +`tests/conftest.py` drops and recreates the schema before each test run, so this database can be reused safely. |
| 74 | + |
| 75 | +## Project structure |
| 76 | + |
| 77 | +``` |
| 78 | +app.py — Flask application: all routes, auth, CSRF, DB queries |
| 79 | +schema.sql — PostgreSQL schema (alerts, analyst_actions, users tables) |
| 80 | +manage.py — CLI: create-user subcommand (bcrypt-hashed, no self-registration) |
| 81 | +seed.py — loads 50 demo alerts across 5 categories, 4 severities, 5 sources |
| 82 | +crypto.py — optional Fernet field-level encryption for title/source_ip/description |
| 83 | +templates/ |
| 84 | + base.html — shared layout, Chart.js import, nav |
| 85 | + dashboard.html — main dashboard: KPI cards, charts, alert queue |
| 86 | + analyst.html — analyst performance page: MTTR trend and per-analyst breakdown |
| 87 | + login.html — login form |
| 88 | +static/ |
| 89 | + dashboard.js — Chart.js wiring: fetches /api/stats, /api/alerts, renders charts + queue |
| 90 | +tests/ |
| 91 | + conftest.py — fixtures: clean schema + deterministic alerts + test analyst per run |
| 92 | + test_app.py — ingest API, classify/escalate flow, filter query params, stats math |
| 93 | + test_crypto.py — Fernet encrypt/decrypt round-trips |
| 94 | + test_manage.py — create-user CLI happy path and error cases |
| 95 | + test_security.py — CSRF enforcement, auth boundaries, constant-time key check |
| 96 | + test_seed.py — seed.py idempotency and row counts |
| 97 | +``` |
| 98 | + |
| 99 | +## How to add a new API endpoint and chart |
| 100 | + |
| 101 | +This is the most common contribution type. The pattern used throughout the codebase: |
| 102 | + |
| 103 | +**1. Add a SQL query in `app.py`** |
| 104 | + |
| 105 | +All database work is raw psycopg2 — no ORM. Open a connection with `get_db()`, run a parameterised query, and return JSON: |
| 106 | + |
| 107 | +```python |
| 108 | +@app.get("/api/alerts/by_analyst") |
| 109 | +@login_required |
| 110 | +def alerts_by_analyst(): |
| 111 | + with get_db() as conn, conn.cursor(cursor_factory=psycopg2.extras.RealDictCursor) as cur: |
| 112 | + cur.execute(""" |
| 113 | + SELECT assigned_to, count(*) AS total |
| 114 | + FROM alerts |
| 115 | + WHERE assigned_to IS NOT NULL |
| 116 | + GROUP BY assigned_to |
| 117 | + ORDER BY total DESC |
| 118 | + """) |
| 119 | + return jsonify(cur.fetchall()) |
| 120 | +``` |
| 121 | + |
| 122 | +**2. Expose the data from `/api/stats` if it belongs there**, or add a new dedicated endpoint like the example above. Stats that feed dashboard KPIs belong in `/api/stats`; per-entity breakdowns can be separate endpoints. |
| 123 | + |
| 124 | +**3. Wire a Chart.js chart in `static/dashboard.js`** |
| 125 | + |
| 126 | +```js |
| 127 | +fetch('/api/alerts/by_analyst') |
| 128 | + .then(r => r.json()) |
| 129 | + .then(data => { |
| 130 | + new Chart(document.getElementById('byAnalystChart'), { |
| 131 | + type: 'bar', |
| 132 | + data: { |
| 133 | + labels: data.map(d => d.assigned_to), |
| 134 | + datasets: [{ label: 'Alerts', data: data.map(d => d.total) }] |
| 135 | + } |
| 136 | + }); |
| 137 | + }); |
| 138 | +``` |
| 139 | + |
| 140 | +**4. Add a `<canvas id="byAnalystChart">` element** in the relevant template (`dashboard.html` or `analyst.html`). |
| 141 | + |
| 142 | +**5. Write tests** — add a test in `tests/test_app.py` that calls the endpoint with the `client` fixture and asserts on the response shape and values against the deterministic fixture data defined in `conftest.py`. |
| 143 | + |
| 144 | +## Database schema changes |
| 145 | + |
| 146 | +The project uses plain SQL migrations, not an ORM migration framework. For any schema change: |
| 147 | + |
| 148 | +1. Update `schema.sql` — this is the source of truth and is re-applied on every test run |
| 149 | +2. Write a standalone migration SQL file if you need to document the upgrade path for existing deployments (e.g. `migrations/0002_add_analyst_notes.sql`) |
| 150 | +3. Update `seed.py` if the new columns need demo data |
| 151 | +4. Update `tests/conftest.py` fixture data if the change affects the deterministic test rows |
| 152 | + |
| 153 | +Do not add nullable columns with no default without discussing it in the issue first — the test fixtures use explicit column lists and will break if the schema diverges. |
| 154 | + |
| 155 | +## Code style |
| 156 | + |
| 157 | +The project uses **ruff** (100-character line length) and **mypy**: |
| 158 | + |
| 159 | +```bash |
| 160 | +# Format / lint |
| 161 | +ruff check . |
| 162 | +ruff format . |
| 163 | + |
| 164 | +# Type check |
| 165 | +mypy app.py |
| 166 | +``` |
| 167 | + |
| 168 | +Configuration is in `pyproject.toml`. The selected ruff rules are E, W, F (pyflakes), I (isort), N (pep8-naming), UP (pyupgrade), B (bugbear). `tests/*` is exempt from N802/N806. |
| 169 | + |
| 170 | +No `black` is used in this project — use `ruff format` instead. |
| 171 | + |
| 172 | +## Running tests |
| 173 | + |
| 174 | +Tests run against a real PostgreSQL database (not a mock). Set `DATABASE_URL` first (see "Test database" above), then: |
| 175 | + |
| 176 | +```bash |
| 177 | +# Run all tests |
| 178 | +python -m pytest tests/ -v |
| 179 | + |
| 180 | +# Run with coverage report |
| 181 | +python -m pytest tests/ -v --cov=. --cov-report=term-missing |
| 182 | + |
| 183 | +# Run a single test file |
| 184 | +python -m pytest tests/test_app.py -v |
| 185 | +``` |
| 186 | + |
| 187 | +The test suite covers the ingest API, auth, CSRF, classify/escalate flow, KPI math, filter query params, encryption, and the `manage.py` / `seed.py` CLIs — 48 tests at 95% line / 92% branch coverage. New contributions should maintain or improve that coverage. |
| 188 | + |
| 189 | +## PR guidelines |
| 190 | + |
| 191 | +- **One concern per PR.** A new chart and a schema change should be two separate PRs. |
| 192 | +- **Tests are required.** Every new endpoint or behaviour change needs a test in the matching `test_*.py` file using the existing fixtures. |
| 193 | +- **No breaking API changes without discussion.** `/api/stats`, `/api/alerts`, and `/api/alerts/<id>/classify` are consumed by log-analyzer. Changing their shape requires an issue discussion first. |
| 194 | +- **Run the full suite locally before opening a PR.** CI runs the same `pytest` command against a PostgreSQL service container. |
| 195 | +- **Security-sensitive changes** (auth, CSRF, API key handling, encryption) need extra scrutiny — flag them clearly in the PR description. |
| 196 | + |
| 197 | +## Reporting bugs |
| 198 | + |
| 199 | +Open an issue and include: |
| 200 | + |
| 201 | +- What you did, what you expected, and what actually happened |
| 202 | +- Browser and version (for frontend bugs) |
| 203 | +- Python version (`python --version`) |
| 204 | +- Flask version (`pip show flask | grep Version`) |
| 205 | +- PostgreSQL version (`psql --version`) |
| 206 | +- Relevant log output or error message (redact any secrets or IP addresses) |
| 207 | +- Whether you are running locally or via Docker Compose |
| 208 | + |
| 209 | +See [SECURITY.md](SECURITY.md) for vulnerabilities — do not open a public issue for security bugs. |
0 commit comments