Consider Dyn-HaMR for hand tracking

[shivpalkang]

Hi, check this out for hand tracking, they released their code just a couple of days ago:

https://github.com/ZhengdiYu/Dyn-HaMR

And if you like it, you can add it to the list for hand mocap
They plan to make a blender addon for it too.

[AClon314]

Seems better than HaMeR, wish this could continuous track the hands like GVHMR yolo do. Currently WiLoR don't have continuous tracking natively.

[shivpalkang]

Yes it does track for two hands using ViTPose and employs the hand motion prior (HMP) as a generative, smooth motion hallucinator, to get temporal coherence. That's why no jittering. It also optimizes for interacting motion prior, which helps separating the and meshes, and preventing biomechanically implausible configurations. That's why it works great with occlusion and interacting hands.

The only problem for me to run it locally is that the ViTPose huge whole body checkpoints require too much VRAM. I keep running out of memory for pytorch (I have 8GB VRAM on my Nvidia 2070 GPU).

[shivpalkang]

I think if the initial step in Dyn-HaMR, which uses HaMeR for hand reconstruction, is replaced with WiLoR, it will make the whole algorithm run a lot faster.