Skip to content

feat: Add Puara TouchArrayGestureDetector processor#11

Open
yrevash wants to merge 2 commits intoossia:masterfrom
yrevash:feature/puara-toucharray-processor
Open

feat: Add Puara TouchArrayGestureDetector processor#11
yrevash wants to merge 2 commits intoossia:masterfrom
yrevash:feature/puara-toucharray-processor

Conversation

@yrevash
Copy link
Copy Markdown
Contributor

@yrevash yrevash commented Jun 10, 2025

Implements an Avendish wrapper for puara_gestures::TouchArrayGestureDetector.

  • Takes a std::vector representing touch strip states as input.
  • Outputs total/top/middle/bottom touch averages, and total brush/rub intensities.
  • Internal puara-gestures parameters (maxBlobs, touchSizeEdge, leaky integrator settings) currently use library defaults.
  • Includes CMake integration.

Note: A safety check for empty input arrays is included in the wrapper. Discussion point: whether the underlying library is robust enough to handle this, potentially simplifying the wrapper.

@yrevash yrevash requested review from edumeneses and jcelerier June 10, 2025 07:23
@yrevash yrevash self-assigned this Jun 10, 2025
@yrevash
Copy link
Copy Markdown
Contributor Author

yrevash commented Jun 10, 2025

Hi @jcelerier and @edumeneses
I've added an initial implementation for the TouchArrayGestureDetector.

Current Implementation:

  • It takes a std::vector as the 'Touch Array' input.

  • It outputs totalTouchAverage, topTouchAverage, middleTouchAverage, bottomTouchAverage, totalBrush, and totalRub as floats.

  • The template parameters maxNumBlobs and touchSizeEdge for the underlying puara_gestures::TouchArrayGestureDetector are currently fixed at compile-time in the wrapper (to TAGD_MAX_BLOBS = 3 and TAGD_TOUCH_SIZE_EDGE = 2).

  • The leaky integrator parameters within the internal Brush and Rub objects currently use their defaults from the puara-gestures library, as there isn't an easy way to configure them externally without modifying puara-gestures/brushRub.h. I've added placeholder input ports for these in the .hpp if we decide to pursue making them configurable.

Testing:
I've been using the Arraygen node in SCORE to generate test arrays, which works well for static inputs. Simulating continuous movement for brush/rub requires manually changing Arraygen's expression for now.

Please let me know your thoughts on this initial version and any further changes or expansions you'd like to see

@jcelerier
Copy link
Copy Markdown
Member

Code looks good! @edumeneses what do you think? I'd be curious to see an example of how one would use it in practice, do we have a device we can try it with?

Otherwise sensors2osc also supports multitouch

@edumeneses
Copy link
Copy Markdown

This is a descriptor particular to touch arrays, e.g., the T-Stick or devices using the Trill touch sensors. I'm not sure it's really generalizable, but it could be interesting to explore possibilities with normalized arrays

@jcelerier
Copy link
Copy Markdown
Member

I see, so yeah Sensors2OSC should allow to try it :)

@yrevash
Copy link
Copy Markdown
Contributor Author

yrevash commented Jun 12, 2025

Hey @jcelerier, let me know of any changes, or if we should move with a merge.

{
// 1. Getting input touch array and its size
// Useing .value here as std::vector doesn't have an implicit conversion from halp::val_port
std::vector<int>& current_touch_array_data = inputs.touch_array_input.value;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd mark this as const auto& instead

Copy link
Copy Markdown
Contributor Author

@yrevash yrevash Jun 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @jcelerier ,

"Regarding the const auto& suggestion for current_touch_array_data (renamed to touch_data):

When I applied const auto&, touch_data.data() returns a const int*. However, the TouchArrayGestureDetector::update function is defined as

void update(int* touchArray, int touchSize)

The compiler threw an error: Cannot initialise a parameter of type 'int *' with an rvalue of type 'const value_type *' (aka 'const int *').

This is C++'s const-correctness preventing me from passing const data to a function that could potentially modify it (because it expects a non-const int*).

Therefore, the most const-correct solution would be to update the TouchArrayGestureDetector::update signature in puara/descriptors/touchArrayGestureDetector.h to:

void update(const int* touchArray, int touchSize);

Should I proceed with this change and commit? Or should I keep the default implementation as it is.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @jcelerier,

As I mentioned, I think modifying ToughArrayGestures.h is a valid option as TouchArrayGestureDetector::update function does not modify the touchArray data it receives. It only reads from it (via arrayAverage and blobDetector.detect1D).

Thus, it would be a good choice to change the call from

--> void update(int* touchArray, int touchSize)

to this

--> void update(const int* touchArray, int touchSize);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ideally we'd use std::span

// 1. Getting input touch array and its size
// Useing .value here as std::vector doesn't have an implicit conversion from halp::val_port
std::vector<int>& current_touch_array_data = inputs.touch_array_input.value;
const int touch_size = static_cast<int>(current_touch_array_data.size());
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not really necessary - you could rename the array "touch" and then do "touch.size()" throughout, it's not going to have any performance impact due to inlining

@yrevash
Copy link
Copy Markdown
Contributor Author

yrevash commented Jun 16, 2025

Hey @jcelerier, Requested Review, much more importantly, the part

When I applied const auto&, touch_data.data() returns a const int*. However, the TouchArrayGestureDetector::update function is defined as

void update(int touchArray, int touchSize)*

**The compiler threw an error: Cannot initialise a parameter of type 'int ' with an rvalue of type 'const value_type ' (aka 'const int ').
This is C++'s const-correctness preventing me from passing const data to a function that could potentially modify it (because it expects a non-const int
).

Therefore, the most correct solution would be to update the TouchArrayGestureDetector::update signature in puara/descriptors/touchArrayGestureDetector.h to:

_void update(const int touchArray, int touchSize);_*

@jcelerier jcelerier force-pushed the master branch 3 times, most recently from 397d5c4 to fa25dea Compare July 29, 2025 18:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants