Migrate from Processing to p5.js#1
Conversation
- Add index.html with p5.js CDN and click-to-start overlay (browser autoplay policy) - Add sketch.js: p5.js port of soothingChantal.pde - Movie → createVideo() (HTML5 video via p5.MediaElement) - Brightest pixel detection via loadPixels()/pixels[] - Speed clamped to valid HTML5 range (0.0625–16) since negative playback is unsupported - Volume normalized to 0–1 range - Layered video playback and HUD visualizers preserved - Update README.md with p5.js description and running instructions - Keep original .pde file for reference Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Track video currentTime to detect new frames instead of checking intrinsic dimensions. This ensures the else-if branch fires between frames, preserving the generative video-layering behavior. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
| } else if (oldX <= 200) { | ||
| rand = floor(random(MAX_MOVIES)); | ||
| videosPlaying.push(myVideos[rand]); | ||
| videosPlaying[videosPlaying.length - 1].loop(); | ||
| } |
There was a problem hiding this comment.
🔴 Unbounded videosPlaying array growth causes progressive performance degradation
The else if block at sketch.js:109-113 pushes a video reference to videosPlaying on every frame where frameAvailable is false and oldX <= 200, but entries are never removed. Since draw() runs at ~60fps and frameAvailable is false whenever firstClipVideo.time() hasn't changed between frames (e.g., during buffering, at low playback speeds down to 0.0625x per line 125, or between video frames), this block can fire dozens of times per second. Notably, oldX initializes to 0 (sketch.js:19), which satisfies <= 200, so this fires continuously until the first video frame is processed. The drawing loop at sketch.js:116-122 iterates the entire array, calling image() for each entry. After running for even a minute, the array can accumulate thousands of redundant entries (all references to the same 6 video objects), causing the draw loop to perform thousands of image() calls per frame. The visual effect of repeated alpha-blended draws saturates after ~20 layers, so all additional draws are pure waste.
Impact at low playback speed
At speed 0.0625x (the minimum from map(oldX, 0, width, 0.0625, 2) when oldX is near 0), the video advances extremely slowly, so currentTime may report the same value across many consecutive draw() calls. This maximizes the rate of array growth, creating a feedback loop: low oldX → low speed → more frames with !frameAvailable → more pushes → more draws → worse performance.
Was this helpful? React with 👍 or 👎 to provide feedback.
Use createVideo() with an array of sources (ogv, mp4, webm) so the browser picks the first format it supports. Chrome dropped Theora support, so OGV-only fails silently (videoWidth/Height = 0). Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Browsers try sources in order and may not fall through from OGV if the container loads but the codec (Theora) is unsupported. MP4/H.264 is universally supported, so it should come first. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
| let currentTime = firstClipVideo.time(); | ||
| let frameAvailable = currentTime !== lastFrameTime && firstClipVideo.width > 0; |
There was a problem hiding this comment.
🔴 Frame availability detection using time() comparison never triggers the else-if branch during playback
The currentTime !== lastFrameTime check on line 75 does not replicate Processing's Movie.available() semantics. HTML5 video's currentTime (exposed by p5's .time()) advances continuously on the media timeline — it changes on virtually every animation frame during playback, even between decoded visual frames. This means frameAvailable is always true during normal playback, and the else if (oldX <= 200) branch at line 109 effectively never fires.
In the original Processing code (soothingChantal.pde:54), available() returns true only when the decoder delivers a new visual frame (~24-30fps), making it false on roughly half of draw() calls at 60fps. The else-if branch firing regularly is what creates the generative additive video-layering effect — the core artistic feature of the piece. With the current p5.js implementation, this layering never occurs during playback.
Prompt for agents
The frame availability detection at sketch.js:74-75 uses `firstClipVideo.time() !== lastFrameTime` to determine if a new frame is available. This doesn't replicate Processing's `Movie.available()` because HTML5 video's currentTime updates continuously (not per-decoded-frame). The else-if branch at line 109 (which adds generative video layers) essentially never fires during playback.
To fix this, you need to detect actual new visual frame delivery rather than timeline advancement. Options:
1. Use the HTMLVideoElement's `requestVideoFrameCallback()` API (modern browsers) to set a flag when a new decoded frame is available, and check/reset that flag in draw().
2. Use a threshold-based approach: treat a frame as 'new' only when currentTime has advanced by more than one video-frame-duration (e.g., 1/30 second), and set frameAvailable=false on intermediate draw() calls.
3. Draw the video to an off-screen graphics buffer and compare a hash/sample of pixels to detect actual visual changes.
Option 1 is the most faithful to the original Processing behavior. The pattern would be:
- In setup: register a callback via `firstClipVideo.elt.requestVideoFrameCallback(onNewFrame)` that sets a `newFrameReady = true` flag and re-registers itself.
- In draw: check `newFrameReady` instead of the time comparison, and reset it to false after processing.
Was this helpful? React with 👍 or 👎 to provide feedback.
1 participant
Summary
Migrates the generative lullaby sketch from Processing (
soothingChantal.pde) to p5.js, making it runnable in any modern browser without needing the Processing IDE.Key changes:
index.html– Loads p5.js v1.11.3 from CDN. Includes a click-to-start overlay to satisfy browser autoplay policies.sketch.js– Full p5.js port ofsoothingChantal.pde:Movie→createVideo()(HTML5<video>via p5.MediaElement)loadPixels()/pixels[](RGBA, 4 bytes per pixel)currentTimecomparison (mirrors Processing'sMovie.available())playbackRateREADME.md– Updated description and added running instructions.pdefile is kept for referenceReview & Testing Checklist for Human
data/lala.oggtheora.ogvand serve the directory (python -m http.server 8000), then open in a browser and click to start — verify the video plays and the HUD values updateoldX <= 200(matches original Processing logic)Notes
-0.5in the original) is not supported by HTML5 video; the mapped range is now0.0625–2instead of-0.5–2tint(255, 50)translucency is applied the same way via p5.jsframeRate(Processing global) →frameRate()(p5.js function call)video.time()changes, which mirrors Processing'sMovie.available()— only fires when a new frame has been decoded.pdeis intentionally preserved so the Processing version remains available for referenceLink to Devin session: https://app.devin.ai/sessions/9df6bd8fae6f461bbdd4d142f99829db
Requested by: @leandroestrella