Smoothed tracking

[NahuelGerthVK]

Problem Statement

Since the default tracking is kind of jittery, I often try to avoid this using lerp.

I faced this with my students when creating interactive installations. We wanted to avoid the jitteryness to make the interaction feel more smooth and organic.

It would be great and very benefitial to have a built in smoothing option where we could simply play with the factor and be ready to go.

Maybe there is a better way to do this though, since it only works for numbered keypoints (not the named ones).

Looking forward to your thoughts on this :-)

Proposed Solution

smoothing factor:

let activateSmoothing = true;
let smoothing = 0.8; // smaller = smoother

draw:

• named keypoint: not smooth
• numbered keypoint: smooth

// let thumb = hands[0].thumb_tip; // by name: not smooth
let thumb = hands[0].keypoints[4]; // by number: smooth

fill("blue");
ellipse(thumb.x, thumb.y, 20);

smoothed tracking:

// hand tracking
function gotHands(results) {
  // smoothing
  if (activateSmoothing) {
    if (hands.length > 0) {
      for (let i = 0; i < results.length; i++) {
        if (hands[i]) {
          let oldHand = hands[i];
          let newHand = results[i];
          for (let j = 0; j < newHand.keypoints.length; j++) {
            newHand.keypoints[j].x = lerp(
              oldHand.keypoints[j].x,
              newHand.keypoints[j].x,
              smoothing
            );
            newHand.keypoints[j].y = lerp(
              oldHand.keypoints[j].y,
              newHand.keypoints[j].y,
              smoothing
            );
          }
        }
      }
    }
  }

  hands = results; // array of tracked points
}

Alternative Solutions

No response

Code of Conduct

• I agree to follow this project's Code of Conduct