Real-Time Face Detection Streaming API

A containerized full-stack application for real-time webcam streaming and face detection. The browser captures frames, sends JPEG bytes to a FastAPI backend over WebSockets, the backend detects faces without OpenCV, draws a bounding box, and streams the processed frames back to the client.

The project intentionally avoids cv2 and implements the computer vision path with MediaPipe, Pillow, and NumPy.

Features

• Real-time webcam frame capture from a React/Vite frontend.
• Producer and consumer WebSocket flow for low-latency streaming.
• Face detection with MediaPipe and image decoding/drawing with Pillow.
• No OpenCV dependency anywhere in the processing pipeline.
• In-memory bounded frame queue to keep the output close to live video.
• Asynchronous PostgreSQL writes for Region of Interest (ROI) history.
• Docker Compose setup for frontend, backend, and database services.

Architecture

Browser Webcam
    |
    | JPEG frames over WebSocket
    v
Frontend Producer
    |
    | ws://localhost:8000/api/v1/stream/ingest
    v
FastAPI Backend
    |
    | Pillow decode -> NumPy array -> MediaPipe detection -> Pillow drawing
    |
    +--> PostgreSQL ROI history
    |
    | processed JPEG frames
    v
Frontend Consumer
    |
    | ws://localhost:8000/api/v1/stream/out
    v
Annotated Live Feed

Frontend

The React frontend has two main responsibilities:

• CameraIngest asks for webcam access, paints frames to a hidden canvas, converts each frame to JPEG, and sends the frame bytes to the ingest WebSocket.
• ProcessedFeed listens for processed JPEG bytes and displays them through temporary Blob URLs, revoking each URL after load to avoid browser memory leaks.

Backend

The FastAPI backend exposes both WebSocket and REST routes:

• /api/v1/stream/ingest receives raw JPEG frames from the browser.
• /api/v1/stream/out sends processed JPEG frames back to the browser.
• /api/v1/roi/ returns recent face bounding-box records from PostgreSQL.
• /health reports backend availability.

The backend uses an asyncio.Queue(maxsize=5) between ingest and output. When the queue is full, the oldest frame is dropped so the stream favors live responsiveness over processing stale frames.

Database

PostgreSQL stores detected ROI records with:

• UUID primary key
• UTC timestamp
• x_min
• y_min
• width
• height

No OpenCV Design

The application avoids OpenCV by replacing the usual cv2 steps with native Python imaging tools:

• Decode JPEG bytes with PIL.Image.open(io.BytesIO(...)).
• Convert the image to a NumPy array for MediaPipe inference.
• Convert MediaPipe normalized bounding boxes to pixel coordinates.
• Add 30% padding to improve full-face coverage.
• Draw the final rectangle with PIL.ImageDraw.
• Encode the result back to JPEG bytes with Pillow.

Getting Started

Prerequisites

• Docker
• Docker Compose
• A browser with webcam access
• Node.js only if you want to run the frontend outside Docker

Run with Docker Compose

git clone https://github.com/SiddharthPatel/face-detection-streaming.git
cd face-detection-streaming
docker-compose up -d --build

Open the app:

• Frontend: http://localhost:3000
• Backend API docs: http://localhost:8000/docs
• Backend health check: http://localhost:8000/health

When the frontend loads, allow browser webcam permission and click Start Camera.

Stop the Stack

docker-compose down

To remove the PostgreSQL volume as well:

docker-compose down -v

API Reference

WebSocket Endpoints

Endpoint	Role	Payload
ws://localhost:8000/api/v1/stream/ingest	Producer input	Raw JPEG frame bytes
ws://localhost:8000/api/v1/stream/out	Consumer output	Processed JPEG frame bytes

REST Endpoints

Method	Endpoint	Description
GET	/health	Returns backend status
GET	/api/v1/roi/?limit=50	Returns recent ROI records ordered by newest first

Example ROI response:

[
  {
    "id": "9b7816d5-1f9a-4b0f-9bb3-93fbc0e3d91a",
    "timestamp": "2026-05-07T08:30:00.000000+00:00",
    "x_min": 82,
    "y_min": 41,
    "width": 155,
    "height": 170
  }
]

Project Structure

face-detection-streaming/
|-- backend/
|   |-- app/
|   |   |-- api/
|   |   |   |-- rest.py
|   |   |   `-- websockets.py
|   |   |-- models/
|   |   |   `-- roi.py
|   |   |-- services/
|   |   |   |-- database.py
|   |   |   `-- vision.py
|   |   `-- main.py
|   |-- Dockerfile
|   `-- requirements.txt
|-- frontend/
|   |-- src/
|   |   |-- components/
|   |   |   |-- CameraIngest.jsx
|   |   |   `-- ProcessedFeed.jsx
|   |   |-- hooks/
|   |   |   `-- useWebSocket.js
|   |   |-- App.jsx
|   |   `-- main.jsx
|   |-- Dockerfile
|   |-- package.json
|   `-- vite.config.js
|-- docker-compose.yml
|-- README.md
`-- REPORT.md

Key Technologies

Layer	Technology
Frontend	React 19, Vite
Backend	FastAPI, Uvicorn, asyncio
Vision	MediaPipe, Pillow, NumPy
Database	PostgreSQL 15, SQLAlchemy async, asyncpg
Deployment	Docker, Docker Compose

Development Notes

• The backend runs with Uvicorn reload enabled inside Docker.
• The frontend Vite server is exposed on 0.0.0.0:3000 for Docker access.
• The frontend currently uses localhost WebSocket URLs directly in the React components.
• PostgreSQL data persists in the pgdata Docker volume.
• The application assumes a single visible face for each frame.

Troubleshooting

Webcam does not start

Make sure the browser has camera permission for localhost. Some browsers require a secure context, and localhost is treated as secure for development.

Output feed is blank

Confirm that:

• The backend is running at http://localhost:8000/health.
• Both WebSocket status indicators show Connected.
• The camera has started and the browser has permission to use it.

Port already in use

Ports 3000, 8000, and 5432 are used by the frontend, backend, and database. Stop the conflicting service or update the mappings in docker-compose.yml.