Incorporating our differentiable contact manifold generation method to Newton

[bekeronur]

Hi!

My name is Onur Beker, I am a first-year PhD student in the Autonomous Learning Group of Prof. Georg Martius as part of the Max Planck Research School for Intelligent Systems in Germany.

I am a big fan of the Newton simulator and have learned a lot from studying it, thanks a lot for making everything open-source!

Our group has a new collision detection method that produces contact manifolds between non-convex surfaces in a smoothly differentiable manner, which is optimized for massive vectorization. It is described in detail in this pre-print on arxiv (which we are currently converting into a complete paper by adding robotics experiments).

We have just recently presented our method at the contact-rich robotics workshop in ICRA, where we had a chance to meet the team at NVIDIA that implemented the hydroelastic contact model (which can be found here). They were kind enough to encourage us about opening a thread on Github to discuss the possibility of incorporating our method to the Newton codebase. If the Newton team is also interested in this, we would be very happy to do so!

Our method essentially constitutes a middle ground between the mesh-sdf collision routine in Newton, and the hydroelastic-contact routine. As such, a potential implementation would look very similar to these two in terms of how it is incorporated into the codebase (i.e. as a single source file that lies under newton/_src/geometry). We believe it would also constitute a good synergy with the recent discussions about implemeting differentiable contacts as well.

Thanks a lot for your time, and please keep up the great open-source work!

[mzamoramora-nvidia]

Hi @bekeronur, thanks for reaching out and for sharing your work with us.

A contribution of this scope would fall under our guidelines for adding a solver or large extension. As such, the implementation should be first developed and made publicly available outside the main Newton repository.

Since a new collision-detection pipeline will likely require changes to Newton's internals, a public Newton fork may be the most practical starting point. If a standalone package is feasible, that would also be suitable.

The external project should ideally include tests, benchmarks against relevant existing collision methods, examples, and documentation describing its capabilities, limitations, and integration points. Please also add the newton-physics GitHub topic to the repository so that it is easier for the community to discover.

Having a public implementation will give the maintainers and the wider community a concrete basis for evaluating the approach, its integration requirements, and its performance. Once the implementation has matured and there is demonstrated interest, we can revisit whether incorporating all or part of it into Newton would be appropriate. As noted in the guidelines, upstream adoption is considered on a case-by-case basis and is not guaranteed.

[bekeronur]

Hello again,

Sounds good, thanks for the information!

We are currently working on a public implementation that combines the proposed collision detection routine with a new contact dynamics routine that takes advantage of it.
Our current plan is to release both parts (i.e. collision detection and contact dynamics) as a public repository before the upcoming iteration of the contact-rich robotics workshop at IROS (where we plan to present it), meaning the public repository should be out by September if everything goes as planned.

The implementation is a standalone package, where the collision detection part is written in NVIDIA-Warp, and the contact dynamics part is written in JAX (to take advantage of the existing numerical optimization and time integration libraries that are part of the JAX scientific-computing ecosystem 1,2,3,4,5)

After this is done, I would be happy to reach out again to get your feedback about our work, and to see if it makes sense to incorporate (parts of) it into the Newton package.

Thanks!