Prototype of Next-Gen Spatial Interaction
This is a full-stack creative engineering prototype — where GPU graphics, real-time gesture recognition, and mathematical motionconverge into a living holographic interface.
More than visuals: it’s an interactive spatial experience that responds to your movements like a futuristic lab assistant.
This system merges multiple advanced technologies:
-WebGL / Three.js → GPU-powered 3D rendering
-Custom GLSL shaders → Nanite-style particle swarm
-Mathematical physics → Smooth, fluid particle motion
-MediaPipe Hand Tracking → Real-time gesture interface
-Adaptive state machine → Automatic or user-controlled modes
The end result is an interface that feels alive, responsive, and intelligent.
-One Hand (Open): Orbit the model by moving your hand.
-Pinch (One Hand): Pinch thumb + index → triggers Next Phase >> (Sphere → Grid → Rings).
-Two Hands: Move hands closer → Zoom In; spread → Zoom Out.
-No Hands: Activates Auto-Pilot Mode — the system evolves formations on its own.
-Optional Future Gesture: Wave to reset or trigger hidden effects.
💡 Gestures are fully math-driven, no buttons required.
-Over 25,000 particles render in real-time without lag.
-Uses shader-based computation — avoids heavy meshes and textures.
-Glow and aura effects simulate pure energy, not just 3D points.
Extra touch: Particle intensity reacts subtly to movement speed, creating a living, breathing effect.
-Smooth motion achieved via LERP (Linear Interpolation).
-Particles ease naturally into new formations, no sudden jumps.
-Motion mimics magnetic, fluid-like inertia, giving each swarm a sense of life.
Extra touch: You can tweak “swarm elasticity” to create faster or slower morphing.
-Sphere: Spherical coordinates for perfect distribution
-Grid / Lattice: 3D structured arrangement
-Vortex / Spiral: Trigonometric + angular math creates dynamic spirals
-Future shapes: Potential for torus, Möbius strip, and organic formations
All shapes generated mathematically, allowing smooth transitions in real time.
-Tracks 21 landmarks per hand, fully in-browser
-Pinch, tilt, and hand position are quantitative, not button-based
-Hand data maps directly to particle speed, rotation, and camera angles
💡 Gesture recognition threshold is adaptive for different users.
-Converts webcam 2D hand coordinates to 3D world motion
-Normalization, axis inversion, and rotation mapping ensure natural response
-Hand movements influence:
-Camera orbit
-Particle behavior intensity
-Morphing speed
Extra touch: Smooth damping avoids jitter and creates cinematic motion.
Real-time loop (~60 FPS):
1. Detect hand presence
2. If hands are present → user-driven mode
3. If no hands → auto-pilot mode
4. Smooth transitions between modes
. Never freezes, never waits — always reactive and continuous.
.Potential future feature: Multi-user recognition with separate control states.
.Rendering: WebGL / Three.js
.Shaders: GLSL (vertex + fragment)
.Math: Linear algebra, LERP, vector transforms, spherical coordinates
.Computer Vision: MediaPipe Hand Landmarker
.Engine: State-machine-driven animation + gesture interaction
Optional future add-ons: Physics-based collisions, sound-reactive particles.
This prototype demonstrates a next-generation spatial interface that:
- Observes and responds to gestures
- Morphs formations mathematically in real-time
- Feels alive, reactive, and futuristic
- Can serve as a base for AR/VR, holographic interfaces, or immersive art
"And finally… the interface awakens. Mission accomplished.”