openEO UDP tuning tool

[jdries]

Throughout the initial phase of APEx, several needs were identified to improve the development workflow of a UDP:

• Determining the cost distribution of a UDP as a function of its input parameters. 
  

• Optimising job configuration options to balance performance against cost. 
  

To address these scenarios, APEx proposes developing a web-based interface that can be used during the algorithm propagation phase. This tool would perform a systematic grid search, executing multiple jobs with varying input parameters and job configuration options.

For each successful run, performance and cost metrics would be collected and visualised, enabling users to evaluate how an algorithm scales under different conditions. Importantly, the full grid-search results could also be made accessible to end users. This would allow them to directly explore how an algorithm scales with respect to temporal coverage, spatial extent, and parameter choices and improve transparency.

An important goal of this tool is also to test and improve robustness: failures that occur for specific combinations of input parameters can be detected more easily.

[Patrick1G]

@jdries algorithm robustness is an important issue!

looking at the long list of benchmark failures (see below),
we should test for robustness first before running the benchmark scenario tests:

[HansVRP]

Hi Patrick;

We revise these on a regular basis.

It is important to note that the reason for these failures is not necessarily dramatic. In a lot of cases its metadata e.g. the name of the underlying product which has changed.

The tooling therefore needs an update as it is too rigid for many cases since the openEO jobs did succeed and the output was the same.