Add API to support various thresholds

[miparnisari]

The current API

rtml.IsMemLimitReached()

is limiting. For my use case, I would like to prioritize in-flight work over incoming work, which means that if I get a call to an internal API (in-flight work), I want to let it through even if memory usage is over 90%. However, if I get a call to an external API (incoming work), I want to block it if memory usage is over 80%.

Could we add an API for this? For example:

rtml.IsMemLimitAboveThreshold(80)
rtml.IsMemLimitAboveThreshold(90)

[blumamir]

Hi @miparnisari

Sorry, I missed your issue, only now notice it.

go will trigger garbage collection and free up memory (mostly) only when we reach the GOMEMLIMIT. If you start rejecting external APIs when memory reaches 80%, and there are no in-flight operations to allocate more memory, that can lead to a dead lock (all requests will be rejected).

If you like, we can have a short zoom call to discuss this, as we (odigos) have similar challenges.