Skip to content

Commit 0895634

Browse files
Copilotsubmarcos
Copilot
and
submarcos
authored
Merge master and resolve parser conflicts
Co-authored-by: submarcos <7448208+submarcos@users.noreply.github.com>
2 parents b1f2c47 + 4e7eff2 commit 0895634

1,318 files changed

Lines changed: 101548 additions & 55020 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.githooks/pre-commit‎

Lines changed: 4 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,17 @@
11
#!/bin/bash
22

3-
function docker_compose {
4-
# detect docker-compose or docker compose command
5-
if which docker-compose -eq 0
6-
then
7-
docker-compose "$@"
8-
else
9-
docker compose "$@"
10-
fi
11-
}
12-
133
echo "Check dependency graph and fix requirements"
144

15-
docker_compose run -T --rm web bash -c "pip-compile -q && pip-compile dev-requirements.in"
16-
17-
5+
make deps
186

19-
echo "run Flake8..."
7+
echo "run Ruff..."
208

21-
docker_compose run -T --rm web flake8 geotrek
9+
make quality
2210
status=$?
2311

2412
if test $status -eq 1
2513
then
2614
exit $status
2715
else
28-
echo "Flake8 check is ok..."
16+
echo "Ruff check is ok..."
2917
fi

‎.github/actions/python-cache-requirements/action.yml‎

Lines changed: 0 additions & 24 deletions
This file was deleted.

‎.github/copilot-instructions.md‎

Lines changed: 307 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,307 @@
1+
# Geotrek-admin Development Instructions
2+
3+
Geotrek-admin is a Django-based geospatial web application for managing trekking, outdoor tourism data and geographic information. It uses PostGIS for spatial data, Redis for caching, and provides both a web admin interface and comprehensive APIs.
4+
5+
**Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here.**
6+
7+
## Working Effectively
8+
9+
### Network Limitations - CRITICAL
10+
- **Docker build from source FAILS** due to firewall limitations preventing Python download from GitHub
11+
- Timeout occurs when downloading `cpython-3.10.18+20250828-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz`
12+
- Build fails after exactly ~2 minutes with "operation timed out" error
13+
- **WORKAROUND**: Use pre-built official images: `docker pull geotrekce/admin:latest`
14+
- **Do not attempt** `docker compose build` - it will fail after ~2 minutes with network timeout
15+
- **Alternative**: Use Ubuntu package installation method for full functionality
16+
17+
### Development Setup
18+
19+
#### Recommended: Development Target (Full Environment)
20+
The `docker-compose.yml` at root is configured for development with `target: dev`. Use this for full development functionality:
21+
22+
```bash
23+
# 1. Setup environment and permissions
24+
cp .env.dist .env
25+
mkdir -p var/{log,cache,media,static,tmp,conf}
26+
chmod -R 777 var
27+
echo "127.0.0.1 geotrek.local" | sudo tee -a /etc/hosts
28+
29+
# 2. Build and start the development environment
30+
docker compose build # NEVER CANCEL: Takes 5-10+ minutes, set timeout to 15+ minutes
31+
docker compose up -d # NEVER CANCEL: Takes 2-3 minutes, set timeout to 10+ minutes
32+
33+
# 3. Initialize the development environment
34+
docker compose run --rm web update.sh # NEVER CANCEL: Takes 2-5 minutes, set timeout to 15+ minutes
35+
docker compose run --rm web load_data.sh # NEVER CANCEL: Takes 5-15 minutes, set timeout to 30+ minutes
36+
```
37+
38+
**TIMING**: Complete setup takes 15-30+ minutes. NEVER CANCEL during any step.
39+
40+
#### Alternative: Network-Limited Environment
41+
If Docker build fails due to network restrictions, use the production image workaround:
42+
43+
```bash
44+
# 1. Setup environment and permissions (same as above)
45+
cp .env.dist .env
46+
mkdir -p var/{log,cache,media,static,tmp,conf}
47+
chmod -R 777 var
48+
echo "127.0.0.1 geotrek.local" | sudo tee -a /etc/hosts
49+
50+
# 2. Pull pre-built production image and tag for local use
51+
docker pull geotrekce/admin:latest
52+
docker tag geotrekce/admin:latest geotrek:latest
53+
54+
# 3. Start supporting services only
55+
docker compose up postgres redis convertit screamshotter -d
56+
# Wait for postgres to be healthy (about 30 seconds)
57+
58+
# Note: Web container may not start properly with production image due to missing dev dependencies
59+
```
60+
61+
**TIMING**: Service startup takes ~30-45 seconds. NEVER CANCEL during startup phase.
62+
63+
#### Alternative Installation Methods
64+
For full functionality when Docker is not suitable:
65+
66+
1. **Ubuntu Package Installation** (Recommended for production-like setup):
67+
```bash
68+
# Follow instructions from docs/installation-and-configuration/installation.rst
69+
bash -c "$(curl -fsSL https://raw.githubusercontent.com/GeotrekCE/Geotrek-admin/master/tools/install.sh)"
70+
```
71+
72+
## Testing
73+
74+
### Django Tests
75+
**CRITICAL TIMING**: Django tests take 15-30+ minutes. **NEVER CANCEL.** Set timeout to 45+ minutes minimum.
76+
77+
```bash
78+
# Full test suite with coverage (LONG RUNNING - 20-30 minutes)
79+
make coverage # NEVER CANCEL: Takes 20-30 minutes. Set timeout to 45+ minutes
80+
81+
# Specific test environments
82+
make test # NEVER CANCEL: Standard tests, ~15 minutes, set timeout to 30+ minutes
83+
make test_nds # NEVER CANCEL: Non-dynamic segmentation tests, ~15 minutes, set timeout to 30+ minutes
84+
make tests # NEVER CANCEL: Both test and test_nds, ~30 minutes, set timeout to 60+ minutes
85+
```
86+
87+
**Note**: Tests work with the development target (`target: dev` in docker-compose.yml). They may not work with the production image workaround due to missing development dependencies.
88+
89+
### Code Quality
90+
```bash
91+
make quality # ✅ Works with development target (dev), ❌ Fails with production image
92+
make lint # ✅ Works with development target (dev), ❌ Fails with production image
93+
make format # ✅ Works with development target (dev), ❌ Fails with production image
94+
```
95+
96+
**Note**: Quality commands work with the development target (`target: dev` in docker-compose.yml) which includes ruff and other dev tools. They fail with the production image workaround due to missing development dependencies.
97+
98+
### End-to-End Tests (Cypress)
99+
**CRITICAL**: Requires complete setup with loaded data. Takes 10-15 minutes. **NEVER CANCEL.**
100+
101+
```bash
102+
# Setup test data first (NEVER CANCEL each step)
103+
make load_data # NEVER CANCEL: ~5-10 minutes, set timeout to 20+ minutes
104+
make load_test_integration # NEVER CANCEL: ~2-5 minutes, set timeout to 15+ minutes
105+
106+
# Run Cypress tests (NEVER CANCEL)
107+
cd cypress
108+
npm ci # NEVER CANCEL: ~2-3 minutes first time, set timeout to 10+ minutes
109+
./node_modules/.bin/cypress run # NEVER CANCEL: ~5-10 minutes, set timeout to 20+ minutes
110+
```
111+
112+
**Warning**: These commands require working development environment setup.
113+
114+
## Key Development Commands
115+
116+
### Management Commands (After successful setup)
117+
```bash
118+
# Database operations
119+
docker compose run --rm web ./manage.py migrate
120+
docker compose run --rm web ./manage.py createsuperuser
121+
docker compose run --rm web ./manage.py collectstatic --clear --noinput
122+
123+
# Translations
124+
make messages # Extract translatable strings
125+
make compilemessages # Compile translations
126+
127+
# Dependencies (requires network access)
128+
make deps # Update all requirements files, ~2-3 minutes
129+
```
130+
131+
### Running the Application
132+
```bash
133+
# Development server (if working environment)
134+
docker compose up # Access at http://geotrek.local:8000
135+
# or
136+
make serve
137+
```
138+
139+
## Validation Scenarios
140+
141+
**Always test these workflows after making changes:**
142+
143+
### 1. Basic Django Functionality
144+
```bash
145+
# Verify Django starts without errors
146+
docker compose run --rm web ./manage.py check
147+
148+
# Test basic management commands
149+
docker compose run --rm web ./manage.py help
150+
```
151+
152+
### 2. Database Operations
153+
```bash
154+
# Test database connectivity
155+
docker compose run --rm web ./manage.py dbshell -c "SELECT version();"
156+
157+
# Verify migrations
158+
docker compose run --rm web ./manage.py showmigrations
159+
```
160+
161+
### 3. API Endpoint Testing
162+
```bash
163+
# Test API availability (if server running)
164+
curl -f http://geotrek.local:8000/api/settings.json
165+
```
166+
167+
## What Actually Works in Network-Limited Environments
168+
169+
### ✅ Working Commands (Development Target)
170+
```bash
171+
# Full development environment (if network allows)
172+
docker compose build # Works when network allows GitHub downloads
173+
docker compose up -d # Full development environment
174+
docker compose run --rm web update.sh # Works with dev target
175+
make test # Works with dev target
176+
make lint # Works with dev target (includes ruff)
177+
make quality # Works with dev target (includes ruff)
178+
```
179+
180+
### ✅ Working Commands (Network-Limited Environments)
181+
```bash
182+
# Service management
183+
docker compose up postgres redis convertit screamshotter -d
184+
docker compose down
185+
docker compose ps
186+
187+
# Production image workaround
188+
docker pull geotrekce/admin:latest
189+
docker tag geotrekce/admin:latest geotrek:latest
190+
191+
# File system operations
192+
cp .env.dist .env
193+
mkdir -p var/{log,cache,media,static,tmp,conf}
194+
chmod -R 777 var
195+
196+
# Basic validation
197+
curl -I https://raw.githubusercontent.com/GeotrekCE/Geotrek-admin/master/tools/install.sh
198+
```
199+
200+
### ❌ Known Failing Commands (Network-Limited)
201+
```bash
202+
# Development environment setup when network is restricted
203+
docker compose build # Network timeout during Python download
204+
docker compose run --rm web update.sh # Only works with dev target, not production image
205+
make test # Only works with dev target
206+
make lint # Only works with dev target (missing ruff in production image)
207+
```
208+
209+
### 🔄 Recommended Workflow for Coding Agents
210+
211+
1. **Try development target first**: Use `docker compose build` and `docker compose up` for full development environment with all tools (linting, testing, dev dependencies)
212+
2. **Fall back to production image workaround** if network restrictions prevent Docker builds from source
213+
3. **Use Ubuntu package installation** for production-like setup when Docker is not suitable
214+
4. **Focus on file-based changes** that don't require running the full application when development environment is not available
215+
5. **Test with full development environment** when possible for comprehensive validation
216+
217+
## Verified Command Timings
218+
219+
**Based on actual testing - use these timeout values:**
220+
221+
| Command | Duration | Recommended Timeout | With Dev Target | With Prod Image |
222+
|---------|----------|-------------------|-----------------|-----------------|
223+
| `docker compose build` | ~5-10 minutes | 900+ seconds | ✅ Works (if network allows) | ❌ Not applicable |
224+
| `docker compose up postgres` | ~30 seconds | 120 seconds | ✅ Works | ✅ Works |
225+
| `docker pull geotrekce/admin:latest` | ~2-3 minutes | 600 seconds | Not needed | ✅ Works |
226+
| `make quality` | ~30 seconds | 120 seconds | ✅ Works | ❌ Missing ruff |
227+
| `make lint` | ~15 seconds | 60 seconds | ✅ Works | ❌ Missing ruff |
228+
| `make format` | ~10 seconds | 60 seconds | ✅ Works | ❌ Missing ruff |
229+
| `make test` | ~15+ minutes | 1800+ seconds | ✅ Works | ❌ Needs dev env |
230+
| `make coverage` | ~20-30 minutes | 2700+ seconds | ✅ Works | ❌ Needs dev env |
231+
| Cypress tests | ~10-15 minutes | 1200+ seconds | ✅ Works | ❌ Needs dev env |
232+
233+
**Key**: ✅ = Verified working, ❌ = Known to fail or missing dependencies
234+
235+
## Common Tasks
236+
237+
### Repository Structure
238+
```
239+
/home/runner/work/Geotrek-admin/Geotrek-admin/
240+
├── geotrek/ # Main Django application
241+
│ ├── settings/ # Django settings
242+
│ ├── api/ # REST API
243+
│ ├── trekking/ # Core trekking models
244+
│ ├── tourism/ # Tourism models
245+
│ └── ... # Other Django apps
246+
├── docker/ # Docker configuration
247+
│ └── Dockerfile # Multi-stage build with dev/prod targets
248+
├── docs/ # Sphinx documentation
249+
├── cypress/ # E2E tests
250+
├── requirements.txt # Python dependencies
251+
├── requirements-dev.txt # Development dependencies (ruff, testing tools)
252+
├── Makefile # Build shortcuts
253+
└── docker-compose.yml # ⭐ Configured for development (target: dev)
254+
```
255+
256+
### Key Files to Monitor
257+
- `geotrek/settings/` - Django configuration
258+
- `requirements.txt` / `requirements-dev.txt` - Dependencies
259+
- `Makefile` - Build and test commands
260+
- `.env` - Environment configuration
261+
- `docker-compose.yml` - Service definitions
262+
263+
### Configuration Files
264+
- `ruff.toml` - Code quality configuration
265+
- `setup.py` - Package definition and dependencies
266+
- `.env.dist` - Environment template
267+
268+
## Troubleshooting
269+
270+
### Known Issues
271+
1. **Docker build fails**: Use pre-built images as documented above
272+
2. **Permission errors**: Ensure `var/` directory has proper permissions (777)
273+
3. **Network timeouts**: All downloads from GitHub may fail in restricted environments
274+
4. **Database connection**: Ensure PostgreSQL container is healthy before running commands
275+
276+
### Log Locations
277+
- Application logs: `var/log/`
278+
- Docker logs: `docker compose logs [service]`
279+
- Test results: Console output, Cypress videos in `cypress/videos/`
280+
281+
### Performance Notes
282+
- Database queries can be slow due to complex geometric operations
283+
- PDF generation requires proper domain setup (geotrek.local)
284+
- Map tiles and static files need proper permissions
285+
286+
## CI/CD Integration
287+
288+
The repository includes GitHub Actions workflows:
289+
- `.github/workflows/test.yml` - Comprehensive test matrix
290+
- `.github/workflows/lint.yml` - Code quality checks
291+
- `.github/workflows/doc.yml` - Documentation builds
292+
293+
**Always run** `make quality` before committing to match CI requirements.
294+
295+
## Required System Resources
296+
- 4+ CPU cores (for complex geometric operations)
297+
- 8+ GB RAM (PostGIS operations are memory-intensive)
298+
- 50+ GB disk space (including media files)
299+
- Docker with at least 4GB memory allocation
300+
301+
## Important Notes
302+
- **NEVER CANCEL** long-running operations (builds, tests) - they may take 30+ minutes
303+
- Always use timeout values of 45+ minutes for build operations
304+
- Always use timeout values of 30+ minutes for test operations
305+
- Always validate changes with both Django and Cypress tests when possible
306+
- Geographic operations are CPU-bound and take significant time
307+
- The application serves as the backend for Geotrek-rando (public website) and Geotrek-mobile apps

0 commit comments

Comments
 (0)