Skip to content

Commit a36434c

Browse files
Sleepwalker86Sleepwalker86
committed
Refactor image handling and enhance directory structure for uploads
- Replaced the `image` directory with a more structured `data` directory to organize uploads, including `data/paperless` for images and `data/invoices` for receipts. - Updated Docker and Docker Compose configurations to reflect the new directory structure, ensuring proper volume mounts. - Modified the `import_data.py` and `upload.py` scripts to save uploaded files in the new `data` directories, improving clarity and maintainability. - Introduced a `.flake8` configuration file for consistent code style checks across the project. - Added a `pyproject.toml` file to manage code formatting and linting tools, enhancing development workflow.
1 parent d334701 commit a36434c

20 files changed

Lines changed: 671 additions & 43 deletions

‎.dockerignore‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ config.json
4444
.env.local
4545
.env.example
4646
import/
47-
image/
47+
data/
4848

4949
# Installations-Skripte (nicht im Container benötigt)
5050
install.sh

‎.flake8‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
[flake8]
2+
# Maximale Zeilenlänge
3+
max-line-length = 127
4+
5+
# Maximale Komplexität
6+
max-complexity = 10
7+
8+
# Zu ignorierende Fehler
9+
ignore =
10+
E501, # Zeilenlänge (wird von Black gehandhabt)
11+
W503, # Zeilenumbruch vor Operator
12+
E203, # Whitespace vor ':' (wird von Black gehandhabt)
13+
14+
# Zu prüfende Dateien
15+
exclude =
16+
.git,
17+
__pycache__,
18+
.venv,
19+
venv,
20+
env,
21+
ENV,
22+
.eggs,
23+
*.egg,
24+
build,
25+
dist,
26+
migrations,
27+
data,
28+
logs,
29+
import
30+
31+
# Dateien die immer geprüft werden sollen
32+
per-file-ignores =
33+
__init__.py:F401
34+
migrate.py:F401,E501
35+
import_data.py:F401,E501

‎.github/workflows/.gitkeep‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
# Dieser Ordner enthält GitHub Actions Workflows

‎.github/workflows/README.md‎

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# GitHub Actions Workflows
2+
3+
Dieses Verzeichnis enthält die CI/CD-Pipeline für die Finanzapp.
4+
5+
## Workflows
6+
7+
### 1. `ci.yml` - Haupt-CI-Pipeline
8+
Wird bei jedem Push und Pull Request ausgeführt:
9+
- **Code Quality**: Linting mit flake8, pylint
10+
- **Docker Build**: Testet ob Docker Image erfolgreich gebaut werden kann
11+
- **Migrations**: Testet Datenbank-Migrationen mit MySQL
12+
- **Imports**: Prüft ob alle Python-Imports funktionieren
13+
- **Security**: Sicherheitsprüfungen mit Bandit
14+
- **Documentation**: Prüft README und Dokumentation
15+
16+
### 2. `docker-build.yml` - Docker Build & Push
17+
Wird ausgeführt bei:
18+
- Push zu `main` Branch
19+
- Erstellung eines Version-Tags (z.B. `v1.0.3`)
20+
- Manueller Auslösung via `workflow_dispatch`
21+
22+
Baut und pusht das Docker Image zu Docker Hub:
23+
- Multi-Architecture (AMD64 + ARM64)
24+
- Taggt mit Versionsnummer und `latest`
25+
26+
**Benötigte Secrets:**
27+
- `DOCKER_USERNAME`: Docker Hub Benutzername
28+
- `DOCKER_PASSWORD`: Docker Hub Token/Passwort
29+
30+
### 3. `code-quality.yml` - Erweiterte Code-Qualität
31+
Fokus auf Code-Qualität:
32+
- Black Code-Formatierung
33+
- isort Import-Sortierung
34+
- flake8 Linting
35+
- mypy Type-Checking
36+
37+
### 4. `release.yml` - GitHub Release erstellen
38+
Wird bei Erstellung eines Version-Tags ausgeführt:
39+
- Erstellt automatisch ein GitHub Release
40+
- Verwendet `RELEASE_NOTES_X.X.X.md` als Release-Notizen
41+
- Verlinkt zum Docker Image auf Docker Hub
42+
43+
## Secrets einrichten
44+
45+
In GitHub Repository Settings → Secrets and variables → Actions:
46+
47+
1. **DOCKER_USERNAME**: Dein Docker Hub Benutzername
48+
2. **DOCKER_PASSWORD**: Docker Hub Access Token (nicht Passwort!)
49+
50+
Docker Hub Token erstellen:
51+
1. Docker Hub → Account Settings → Security
52+
2. "New Access Token" erstellen
53+
3. Token kopieren und als Secret speichern
54+
55+
## Workflow manuell auslösen
56+
57+
```bash
58+
# Via GitHub Web-Interface:
59+
# Actions → Workflow auswählen → "Run workflow"
60+
61+
# Via GitHub CLI:
62+
gh workflow run docker-build.yml -f version=1.0.3
63+
```
64+
65+
## Lokale Tests
66+
67+
Vor dem Commit kannst du einige Checks lokal ausführen:
68+
69+
```bash
70+
# Code-Formatierung prüfen
71+
black --check .
72+
73+
# Code formatieren
74+
black .
75+
76+
# Linting
77+
flake8 .
78+
79+
# Syntax prüfen
80+
python -m py_compile app.py
81+
find . -name "*.py" -exec python -m py_compile {} \;
82+
```
83+
84+
## Workflow-Status
85+
86+
Die Workflows zeigen ihren Status direkt im GitHub Repository:
87+
- ✅ Grün: Alle Checks erfolgreich
88+
- ❌ Rot: Mindestens ein Check fehlgeschlagen
89+
- 🟡 Gelb: Workflow läuft noch
90+
91+
Bei fehlgeschlagenen Checks:
92+
1. Klicke auf den fehlgeschlagenen Check
93+
2. Prüfe die Logs
94+
3. Behebe die Fehler
95+
4. Committe und pushe erneut

‎.github/workflows/ci.yml‎

Lines changed: 241 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,241 @@
1+
name: CI/CD Pipeline
2+
3+
on:
4+
push:
5+
branches: [ main, develop ]
6+
pull_request:
7+
branches: [ main, develop ]
8+
workflow_dispatch:
9+
10+
jobs:
11+
# Job 1: Python Code Quality Checks
12+
lint-and-format:
13+
name: Code Quality & Linting
14+
runs-on: ubuntu-latest
15+
16+
steps:
17+
- name: Checkout code
18+
uses: actions/checkout@v4
19+
20+
- name: Set up Python
21+
uses: actions/setup-python@v5
22+
with:
23+
python-version: '3.11'
24+
cache: 'pip'
25+
26+
- name: Install dependencies
27+
run: |
28+
python -m pip install --upgrade pip
29+
pip install flake8 black pylint
30+
pip install -r requirements.txt
31+
32+
- name: Check code formatting with Black
33+
run: |
34+
black --check --diff .
35+
continue-on-error: true
36+
37+
- name: Lint with flake8
38+
run: |
39+
# Stop the build if there are Python syntax errors or undefined names
40+
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
41+
# Exit-zero treats all errors as warnings
42+
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
43+
continue-on-error: true
44+
45+
- name: Lint with pylint
46+
run: |
47+
pylint --disable=all --enable=E,F --fail-under=5 routes/ utils/ services/ || true
48+
continue-on-error: true
49+
50+
- name: Check Python syntax
51+
run: |
52+
python -m py_compile app.py
53+
find . -name "*.py" -not -path "./.venv/*" -not -path "./venv/*" -exec python -m py_compile {} \;
54+
55+
- name: Check for common issues
56+
run: |
57+
# Prüfe auf TODO/FIXME ohne Issue-Nummer
58+
if grep -r "TODO\|FIXME" --include="*.py" . | grep -v "# TODO:" | grep -v "# FIXME:"; then
59+
echo "⚠️ Gefundene TODO/FIXME Kommentare"
60+
fi
61+
62+
# Job 2: Test Docker Build
63+
docker-build:
64+
name: Docker Build Test
65+
runs-on: ubuntu-latest
66+
67+
steps:
68+
- name: Checkout code
69+
uses: actions/checkout@v4
70+
71+
- name: Set up Docker Buildx
72+
uses: docker/setup-buildx-action@v3
73+
74+
- name: Build Docker image
75+
uses: docker/build-push-action@v5
76+
with:
77+
context: .
78+
file: ./Dockerfile
79+
push: false
80+
tags: finanzapp:test
81+
cache-from: type=gha
82+
cache-to: type=gha,mode=max
83+
84+
- name: Test Docker image
85+
run: |
86+
docker run --rm finanzapp:test python -c "import flask; import mysql.connector; print('✅ Imports erfolgreich')"
87+
docker run --rm finanzapp:test python -m py_compile app.py
88+
echo "✅ Docker Image Build erfolgreich"
89+
90+
# Job 3: Test Migrations
91+
test-migrations:
92+
name: Test Database Migrations
93+
runs-on: ubuntu-latest
94+
95+
services:
96+
mysql:
97+
image: mysql:8.0
98+
env:
99+
MYSQL_ROOT_PASSWORD: test_password
100+
MYSQL_DATABASE: test_db
101+
ports:
102+
- 3306:3306
103+
options: >-
104+
--health-cmd="mysqladmin ping -h localhost"
105+
--health-interval=10s
106+
--health-timeout=5s
107+
--health-retries=3
108+
109+
steps:
110+
- name: Checkout code
111+
uses: actions/checkout@v4
112+
113+
- name: Set up Python
114+
uses: actions/setup-python@v5
115+
with:
116+
python-version: '3.11'
117+
cache: 'pip'
118+
119+
- name: Install dependencies
120+
run: |
121+
python -m pip install --upgrade pip
122+
pip install -r requirements.txt
123+
124+
- name: Wait for MySQL
125+
run: |
126+
while ! mysqladmin ping -h 127.0.0.1 -u root -ptest_password --silent; do
127+
echo "Warte auf MySQL..."
128+
sleep 2
129+
done
130+
echo "✅ MySQL ist bereit"
131+
132+
- name: Test migrations
133+
env:
134+
DB_HOST: 127.0.0.1
135+
DB_USER: root
136+
DB_PASSWORD: test_password
137+
DB_NAME: test_db
138+
run: |
139+
python migrate.py
140+
echo "✅ Migrationen erfolgreich ausgeführt"
141+
142+
- name: Verify schema
143+
env:
144+
DB_HOST: 127.0.0.1
145+
DB_USER: root
146+
DB_PASSWORD: test_password
147+
DB_NAME: test_db
148+
run: |
149+
python -c "
150+
import mysql.connector
151+
conn = mysql.connector.connect(
152+
host='127.0.0.1',
153+
user='root',
154+
password='test_password',
155+
database='test_db'
156+
)
157+
cur = conn.cursor()
158+
cur.execute('SHOW TABLES')
159+
tables = [row[0] for row in cur.fetchall()]
160+
print(f'✅ Gefundene Tabellen: {tables}')
161+
assert 'buchungen' in tables, 'Tabelle buchungen fehlt'
162+
assert 'schema_migrations' in tables, 'Tabelle schema_migrations fehlt'
163+
cur.close()
164+
conn.close()
165+
print('✅ Schema-Verifikation erfolgreich')
166+
"
167+
168+
# Job 4: Test Import Scripts
169+
test-imports:
170+
name: Test Import Scripts
171+
runs-on: ubuntu-latest
172+
173+
steps:
174+
- name: Checkout code
175+
uses: actions/checkout@v4
176+
177+
- name: Set up Python
178+
uses: actions/setup-python@v5
179+
with:
180+
python-version: '3.11'
181+
cache: 'pip'
182+
183+
- name: Install dependencies
184+
run: |
185+
python -m pip install --upgrade pip
186+
pip install -r requirements.txt
187+
188+
- name: Test import syntax
189+
run: |
190+
python -c "import app; import db; import migrate; import init_db"
191+
python -c "from routes import dashboard, actions, settings, upload"
192+
python -c "from utils import beleg_upload, csrf, csv_parser, helpers, logging_config, version"
193+
python -c "from services import data_service"
194+
echo "✅ Alle Imports erfolgreich"
195+
196+
# Job 5: Security Checks
197+
security:
198+
name: Security Checks
199+
runs-on: ubuntu-latest
200+
201+
steps:
202+
- name: Checkout code
203+
uses: actions/checkout@v4
204+
205+
- name: Run Bandit security linter
206+
uses: securecodewarrior/github-action-bandit@v1
207+
with:
208+
path: .
209+
exit_zero: true
210+
continue-on-error: true
211+
212+
- name: Check for secrets
213+
run: |
214+
# Prüfe auf mögliche hardcodierte Secrets
215+
if grep -r "password.*=.*['\"].*[A-Za-z0-9]" --include="*.py" . | grep -v "test_password" | grep -v "# "; then
216+
echo "⚠️ Mögliche hardcodierte Passwörter gefunden"
217+
fi
218+
if grep -r "SECRET_KEY.*=.*['\"].*[A-Za-z0-9]" --include="*.py" . | grep -v "change-me" | grep -v "# "; then
219+
echo "⚠️ Mögliche hardcodierte Secret Keys gefunden"
220+
fi
221+
222+
# Job 6: Documentation Check
223+
docs:
224+
name: Documentation Check
225+
runs-on: ubuntu-latest
226+
227+
steps:
228+
- name: Checkout code
229+
uses: actions/checkout@v4
230+
231+
- name: Check README exists
232+
run: |
233+
test -f README.md && echo "✅ README.md vorhanden" || exit 1
234+
235+
- name: Check for broken links in README
236+
run: |
237+
# Einfache Prüfung auf Markdown-Links
238+
if grep -r "\[.*\](" README.md | grep -v "http" | grep -v "mailto"; then
239+
echo "⚠️ Lokale Links gefunden - bitte prüfen"
240+
fi
241+
continue-on-error: true

0 commit comments

Comments
 (0)