Medical MPR Viewer with Enhanced AI Detection

A comprehensive Multi-Planar Reconstruction (MPR) medical imaging viewer with AI-powered organ and orientation detection. Features real-time interactive crosshair navigation, oblique plane cutting, ROI tools, and advanced visualization capabilities for DICOM, NIfTI, and MHD medical image formats.

📋 Table of Contents

• Overview
• Features
• Demo
• Architecture
• Requirements
• Installation
• Usage
• Interactive Controls
• Supported Formats
• Color Coding
• License
• Contributing
• Contact

🔍 Overview

This project implements an advanced Multi-Planar Reconstruction (MPR) viewer that enables radiologists, researchers, and medical professionals to visualize and analyze medical imaging data across multiple anatomical planes. The system integrates AI-powered detection capabilities to automatically identify organs and anatomical orientations.

🎯 Features

Core Viewing Capabilities

• Multi-Planar Reconstruction (MPR): Simultaneous visualization of Axial, Coronal, and Sagittal views
• Interactive Crosshairs: Click and drag reference lines to navigate through slices
• Synchronized Navigation: All views update together for coordinated examination
• Zoom & Pan: Ctrl+Scroll to zoom, Right-click to pan across images
• Slice Scrolling: Mouse wheel to navigate through slices

Advanced Visualization

• Fourth View Modes:
  • Contour View: Real-time edge detection and contour extraction from any plane
  • Oblique Plane: Draw a cutting line to create custom oblique slices through the volume

AI-Powered Detection

• Organ Detection: Automatic identification of anatomical regions (Brain, Chest/Heart, Abdomen/Liver, Pelvis, Spine, Extremities)
• Orientation Detection: Per-plane orientation analysis with confidence scores
• Dual AI Approach:
  • ResNet18-based deep learning features (when PyTorch available)
  • Enhanced heuristic algorithms with statistical analysis

Region of Interest (ROI) Tools

• Visual ROI Drawing: Click and drag to define regions of interest
• Manual ROI Input: Precise numerical specification of ROI bounds
• ROI Cropping: Apply ROI to zoom and focus on specific areas
• ROI Persistence: Save and load ROI definitions as JSON files

File Format Support

• DICOM Series: Load entire folders of DICOM slices
• NIfTI: Support for .nii and .nii.gz formats
• MetaImage: Support for .mhd/.raw format

User Interface

• Modern dark theme with color-coded reference lines
• Real-time voxel spacing and position information
• Responsive controls with visual feedback
• Comprehensive keyboard and mouse shortcuts

🎬 Demo

Contour View

Real-time edge detection and contour extraction from any plane

CONTOUR.mp4

Oblique Plane Cutting

Custom oblique plane visualization with drawn cutting line

OBLIQUE.mp4

ROI Selection

Interactive ROI selection with visual overlay

ROI.mp4

🏗️ Architecture

┌──────────────────────────────────────────────────┐
│          Medical Image Input (DICOM/NIfTI/MHD)   │
└──────────────────┬───────────────────────────────┘
                   │
                   ▼
         ┌─────────────────────┐
         │   Image Loading     │
         │  - DICOM Series     │
         │  - NIfTI Files      │
         │  - MHD/RAW Files    │
         └──────────┬──────────┘
                    │
                    ▼
         ┌──────────────────────┐
         │  AI Detection Engine │
         │  - ResNet18          │
         │  - Heuristic Analysis│
         │  - Orientation Check │
         └──────────┬───────────┘
                    │
        ┌───────────┴────────────┐
        │                        │
        ▼                        ▼
┌───────────────────┐    ┌──────────────────────┐
│  MPR Rendering    │    │  Interactive Tools   │
│  - Axial View     │    │  - Crosshair Control │
│  - Coronal View   │    │  - ROI Drawing       │
│  - Sagittal View  │    │  - Oblique Cutting   │
│  - Special View   │    │  - Zoom/Pan          │
└───────────────────┘    └──────────────────────┘
        │                        │
        └───────────┬────────────┘
                    │
                    ▼
         ┌────────────────────┐
         │  Output & Export   │
         │  - Visualizations  │
         │  - ROI Data (JSON) │
         │  - Measurements    │
         └────────────────────┘

📋 Requirements

Hardware

• Mouse or trackpad for navigation and interaction
• 4GB+ RAM (8GB+ recommended)
• CUDA-compatible GPU (optional, for AI detection)
• 2GB+ free disk space

Software

• Python 3.8 or higher
• Operating System: Windows, macOS, or Linux
• Required libraries:
  • PyQt5
  • NumPy
  • pydicom
  • nibabel (for NIfTI support)
  • scipy & scikit-image (for contour/oblique views)
  • torch & torchvision (optional, for AI detection)

💾 Installation

Quick Setup

1. Clone the repository

git clone https://github.com/yourusername/MPR-Viewer.git

2. Navigate to the project directory

cd MPR-Viewer

3. Install core dependencies

pip install -r requirements.txt

4. Install optional dependencies (for full features)

# For NIfTI file support
pip install nibabel

# For contour and oblique views
pip install scipy scikit-image

# For AI-enhanced detection
pip install torch torchvision

🚀 Usage

Starting the Application

python MPR_Viewer.py

Loading Medical Images

1. Click "📁 Load Files"
2. Choose loading method:
   • DICOM Folder: Select folder containing DICOM series
   • Single File: Load NIfTI (.nii, .nii.gz) or MHD (.mhd) files

Basic Navigation

Mouse Controls:

• Left Click + Drag Line: Move single crosshair line
• Right Click + Drag: Move both crosshair lines simultaneously
• Ctrl + Right Click + Drag: Pan view
• Scroll Wheel: Navigate through slices
• Ctrl + Scroll Wheel: Zoom in/out

View Selection:

• Use dropdown menus to change 4th view mode (Contour/Oblique)
• Select source plane for contour or oblique views

Using ROI Tools

1. Draw ROI:

   • Click "⬚ Draw ROI" button
   • Click and drag on any view to define rectangle
   • Release to confirm selection

2. Manual ROI Entry:

   • Click "📝 Manual ROI"
   • Enter precise Z, Y, X start/end values
   • Click OK to apply

3. Apply ROI:

   • Click "🔍 Apply ROI" to crop volume to selected region

4. Save/Load ROI:

   • Save current ROI: "💾 Save ROI" → Choose location
   • Load saved ROI: "📂 Load ROI" → Select JSON file

Creating Oblique Views

1. Switch to Oblique Mode:

   • Select "Oblique Plane" from 4th view dropdown

2. Choose Source Plane:

   • Select Axial, Coronal, or Sagittal from oblique dropdown

3. Draw Cutting Line:

   • Click "✏️ Draw Cutting Line"
   • Click and drag on source plane to define cutting angle
   • Oblique view updates in real-time

4. View Results:

   • See angle measurement in button label
   • Observe oblique slice in 4th panel

AI Detection

The application automatically analyzes loaded volumes:

• Displays detected organ type with confidence
• Shows orientation analysis for each plane
• Uses ResNet18 + heuristics (if PyTorch available) or enhanced heuristics alone
• Results shown in blue info banner at top

Detected Organs

• Brain: High contrast, complex edge structures
• Chest/Heart: Medium intensity, characteristic shape
• Abdomen/Liver: Moderate intensity, homogeneous regions
• Pelvis: High bone intensity, specific anatomical markers
• Spine: Very high bone intensity, elongated structure
• Extremities: Variable intensity, elongated proportions

Orientation Detection

Per-plane analysis provides:

• Confidence Scores: 70-95% range
• Anatomical Labels: Axial, Coronal, Sagittal
• Volume Analysis: Overall orientation assessment

Reset View

Click "🔄 Reset" to restore original volume and clear all modifications

🎮 Interactive Controls

Keyboard Shortcuts

Action	Shortcut
Pan View	Ctrl + Right Click + Drag
Zoom	Ctrl + Scroll Wheel
Next Slice	Scroll Up
Previous Slice	Scroll Down

Mouse Controls Reference

Action	Control
Drag Single Line	Left Click + Drag (on line)
Drag Both Lines	Right Click + Drag
Draw ROI	Left Click + Drag (in ROI mode)
Draw Oblique Line	Left Click + Drag (in oblique mode)
Pan View	Ctrl + Right Click + Drag
Zoom	Ctrl + Scroll Wheel
Change Slice	Scroll Wheel

Slider Controls

• Each view has a slider for precise slice navigation
• Slider values sync with mouse wheel scrolling
• Direct slider manipulation updates all views

📁 Supported Formats

DICOM (.dcm)

• Full DICOM series support
• Automatic slice ordering by position/instance number
• Voxel spacing extraction from metadata
• Multi-file series loading

NIfTI (.nii, .nii.gz)

• Compressed and uncompressed formats
• Automatic orientation handling
• Header metadata extraction
• 3D and 4D volume support (uses first timepoint)

MetaImage (.mhd + .raw/.zraw)

• Header + raw data pair
• Multiple data types (short, ushort, float, uchar, etc.)
• Spacing and dimension metadata
• Compressed (.zraw) support

🎨 Color Coding

• Yellow Lines: Coronal reference plane
• Magenta Lines: Sagittal reference plane
• Cyan Lines: Axial reference plane
• Green Box: ROI boundary
• Green Line: Oblique cutting line

📝 License

This project is provided as-is for educational and research purposes. Not intended for clinical diagnostic use.

🤝 Contributing

We welcome contributions! Please follow these steps:

1. Fork the repository
2. Create a feature branch

   git checkout -b feature/amazing-feature

3. Commit your changes

   git commit -m 'Add amazing feature'

4. Push to the branch

   git push origin feature/amazing-feature

5. Open a Pull Request

📧 Contact

Project Contributer: Radwa Hamdy

• Email: radwahamdy922@gmail.com
• GitHub
• LinkedIn

Project Contributer: Maryam Moustafa

• Email: maryam23shabaan@gmail.com
• GitHub
• LinkedIn

Project Contributer: Nour Ahmed

• GitHub
• LinkedIn

Project Contributer: Aya Sayed

• Email: aya.sayed14827@gmail.com
• GitHub