Baskerville Solo - Self-Hosted Prototype

Simplified self-hosted version of Baskerville for local deployment without Kafka and Logstash.

Architecture

┌─────────────────┐
│   Web Server    │
│  (nginx/etc)    │
└────────┬────────┘
         │ HTTP POST
         ▼
┌─────────────────┐
│  Predictor API  │  ← REST API for receiving logs
│   (FastAPI)     │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│   PostgreSQL    │  ← Store logs and predictions
└─────────────────┘

┌─────────────────┐
│     Trainer     │  ← Model training (future phase)
└─────────────────┘

Current Phase: Simulation

What's working now:

• ✅ REST API for receiving logs
• ✅ Storing logs in PostgreSQL
• ✅ Simulation of predictions with random scores
• ✅ Writing predictions to PostgreSQL
• ✅ API for viewing predictions and statistics

What will be added:

• ⏳ Grouping logs into sessions
• ⏳ Loading real ML models
• ⏳ Real predictions with Isolation Forest and AutoEncoder
• ⏳ Model training process

Quick Start

1. Start services

cd baskerville_solo
docker compose up --build

# Or in detached mode:
docker compose up -d --build

Three services will start:

• postgres on port 5432
• predictor on port 8000
• trainer (placeholder only for now)

2. Check health

curl http://localhost:8000/health

Response:

{
  "status": "healthy",
  "database": "connected",
  "timestamp": "2025-10-29T12:00:00"
}

3. Submit a test log

curl -X POST http://localhost:8000/api/v1/logs \
  -H "Content-Type: application/json" \
  -d '{
    "timestamp": "2025-10-29 12:00:00",
    "ip": "192.168.1.100",
    "host": "example.com",
    "path": "/index.html",
    "method": "GET",
    "status": 200,
    "user_agent": "Mozilla/5.0",
    "country": "US",
    "session_id": "test_session_123"
  }'

Response:

{
  "status": "success",
  "message": "Log received and processed",
  "prediction": {
    "result": "anomaly",
    "score_if": -0.234,
    "score_ae": 1.456,
    "is_anomaly": true
  }
}

4. Submit batch of logs

curl -X POST http://localhost:8000/api/v1/logs/batch \
  -H "Content-Type: application/json" \
  -d '{
    "logs": [
      {
        "timestamp": "2025-10-29 12:00:00",
        "ip": "192.168.1.100",
        "host": "example.com",
        "path": "/page1.html",
        "method": "GET",
        "status": 200,
        "user_agent": "Mozilla/5.0",
        "country": "US"
      },
      {
        "timestamp": "2025-10-29 12:00:01",
        "ip": "192.168.1.100",
        "host": "example.com",
        "path": "/page2.html",
        "method": "GET",
        "status": 200,
        "user_agent": "Mozilla/5.0",
        "country": "US"
      }
    ]
  }'

5. View predictions

curl http://localhost:8000/api/v1/predictions?limit=10

6. View statistics

curl http://localhost:8000/api/v1/stats

Response:

{
  "status": "success",
  "stats": {
    "total_logs": 150,
    "total_predictions": 75,
    "total_anomalies": 23,
    "anomaly_rate": 30.67
  }
}

API Endpoints

Health & Status

• GET / - Root endpoint
• GET /health - Health check
• GET /api/v1/stats - Statistics

Log Processing

• POST /api/v1/logs - Submit single log
• POST /api/v1/logs/batch - Submit batch of logs

Predictions

• GET /api/v1/predictions?limit=N - Get recent predictions

Database

Tables

raw_logs - Store incoming logs

CREATE TABLE raw_logs (
    id SERIAL PRIMARY KEY,
    timestamp TIMESTAMP NOT NULL,
    ip VARCHAR(45) NOT NULL,
    host VARCHAR(255) NOT NULL,
    path TEXT NOT NULL,
    method VARCHAR(10),
    status INTEGER,
    user_agent TEXT,
    country VARCHAR(10),
    session_id VARCHAR(255),
    received_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

predictions - Store predictions

CREATE TABLE predictions (
    id SERIAL PRIMARY KEY,
    session_id VARCHAR(255) NOT NULL,
    ip VARCHAR(45) NOT NULL,
    host VARCHAR(255) NOT NULL,
    prediction VARCHAR(50) NOT NULL,
    score_if FLOAT,
    score_ae FLOAT,
    is_anomaly BOOLEAN,
    metadata JSONB,
    predicted_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

Connecting to PostgreSQL

docker exec -it baskerville-postgres psql -U baskerville -d baskerville

Example queries:

-- Count logs
SELECT COUNT(*) FROM raw_logs;

-- Recent predictions
SELECT * FROM predictions ORDER BY predicted_at DESC LIMIT 10;

-- Statistics by IP
SELECT ip, COUNT(*) as log_count FROM raw_logs GROUP BY ip;

-- Anomaly percentage
SELECT
  COUNT(CASE WHEN is_anomaly THEN 1 END)::FLOAT / COUNT(*) * 100 as anomaly_percent
FROM predictions;

Stopping services

docker compose down

# With data removal
docker compose down -v

Next Steps

1. Sessions: Add grouping of logs into sessions (by IP, host, time window)
2. Feature extraction: Port feature_extractor.py from main project
3. Model loading: Add loading of Isolation Forest and AutoEncoder from files
4. Real predictions: Replace simulation with real ML predictions
5. Trainer: Implement model training

File Structure

baskerville_solo/
├── docker-compose.yaml       # Service orchestration
├── Dockerfile.predictor       # Docker image for predictor API
├── Dockerfile.trainer         # Docker image for trainer
├── predictor_api.py          # REST API for receiving logs
├── trainer.py                # Placeholder for model training
├── requirements.txt          # Python dependencies
└── README.md                 # This documentation

Environment Variables

Can be configured via .env file:

# PostgreSQL
POSTGRES_HOST=postgres
POSTGRES_PORT=5432
POSTGRES_DB=baskerville
POSTGRES_USER=baskerville
POSTGRES_PASSWORD=baskerville123

# API
API_PORT=8000

Logs

View logs:

# All services
docker compose logs -f

# Predictor only
docker compose logs -f predictor

# Trainer only
docker compose logs -f trainer

Database Utilities

Check database script

./check_db.sh

This script displays:

• Total logs count
• Total predictions count
• Recent predictions (last 10)
• Anomaly statistics
• Top IPs with predictions

Test API script

./test_api.sh

This script:

• Checks API health
• Sends single test log
• Sends batch of logs
• Retrieves predictions
• Shows statistics