PWD-driven FINALES tenant(s)

[edan-bainglass]

We discussed briefly a potential tenant for FINALES with a "broad" capability of yielding any PWD-emitted quantity from its corresponding PWD-driven method. This thread is now open to continue the discussion and draft a plan towards implementation/testing/deployment.

Pinging @jan-janssen @MonikaVo

[MonikaVo]

Thank you @edan-bainglass for opening this discussion. Why would you like to put all the capabilities in a single tenant rather than having several tenants?

[edan-bainglass]

In principle, it doesn't have to be, and perhaps not the way to do. It's more about providing the "capability" of leveraging any PWD-supporting engine to produce any PWD-derived quantity.

Example use case

• User A wants to compute the elastic constant (semantically Q) of some material (semantically STRUCT).
• There exists a PWD yielding Q via method M1 taking STRUCT as input (plus other inputs).
  • In principle, there could be n PWDs yielding Q via m methods Mm, presumably all requiring at least STRUCT as input.
• User A would like to submit a request to FINALES for Q via Mi (selected by user) providing STRUCT as input.

TBD

How to best register PWD-based capabilities with FINALES?

[edan-bainglass]

How to best register PWD-based capabilities with FINALES?

Can this be achieved via a single PWD-aware tenant? If so, does this add any complexity to FINALES or are any and all complexities isolated to said tenant? Does such encapsulation offer any benefits as compared to multiple tenants? If multiple tenants (one per engine?), each tenant can simply implement PWD-awareness (a few already do via @jan-janssen's (and company) efforts - see paper). If so, how do they announce this capability? Generically, i.e., "I can compute Q via Mm methods (with inputs)"? If each Mm requires a different set of inputs, this is likely not possible. There must be a clear schema for input/output! Alternatively, as part of PWD-awareness implementation, perhaps we can think of a general mechanism for dynamic capability registration? The PWD-aware tenant can auto-register each discovered method Mi associated with Q as capability Ci with its own set of inputs.

Anyhow, just some thoughts for now. Good to get @jan-janssen @giovannipizzi @MonikaVo's thoughts on this.

[jan-janssen]

As part of the Materials Digital project we started to work on a workflow store concept similar to the app stores we all use on our mobile phones:
https://workflows.material-digital.de

While this concept is not yet using the Python Workflow Definition, it is obviously a planned extension, right @schaarj ? The Material Digital consortium could be one maintainer of such a workflow store but ideally all the required source code would be open source so other consortia like POLARIS could host their own workflow store. Anyway the workflow store is just a meta data index, which hosts links to workflows published on Github or Zenodo. So larger consortia can decide to either host their own workflow store which only contains workflows they approve or they use one of the larger public community workflow stores.

At the moment these developments are on a prototypical stage but having a description of the workflow which can be communicated to FINALES could be a requirement of such a workflow store.

[edan-bainglass]

That's great @jan-janssen. Indeed, I envision a tenant (or each tenant) having access to such a store, from which it can fetch for a given PWD a set of instructions covering installation, setup, and usage. @giovannipizzi suggested one option would be that the tenant periodically pings the store for available workflows. As mentioned above, each fetched workflow could be dynamically registered with FINALES as a capability. @MonikaVo what do you think? Happy to collab on these dynamics.

[MonikaVo]

This is a rather different usecase from the ones we covered up to now. The dynamic registration of capabilities was not done yet to avoid the registration of very similar capabilities instead of using existing capabilities and the associated schemas. Up to now, I tried not to agglomerate too much functionality/too many capabilities in one tenant to enable the distribution of tasks by the principle of the next available tenant picking up the task. If we collect all the PWD-capabilities in one tenant, I guess we will require every newcomer to connect to this tenant and not directly to FINALES. If this single tenant is down, all the PWD-capabilities are not available. So splitting it up would also be an approach to reduce down time.

This is what came to my mind at this moment. There might be good reasons to deviate from these practices. I guess, I would need to understand the details of these workflows in more detail to see what could work or not.

[edan-bainglass]

If we collect all the PWD-capabilities in one tenant, I guess we will require every newcomer to connect to this tenant and not directly to FINALES

Can you explain what you mean by this? Is the newcomer a tenant? In any case, why the direct connection to the PWD tenant?

[MonikaVo]

In my train of thought, the newcomer would be a tenant. I understood that the PWD would be a (or the (?)) input for the computational workflow engines. Assuming this is correct, the workflow engines would all need to be able to process PWD as an input. If you mention only a single tenant that can collect PWD inputs from FINALES (which means it is the sole tenant that is registered with the PWD-capability), this means for me that all the workflow engines will be connected to this tenant and get their tasks assigned from there since they will not be registered as individual tenants with FINALES for the PWD-capability. If all the workflow engines are registered with FINALES, this means that we would have several tenants registered for the PWD-capability.

Feel free to correct me if I misunderstood your concept.