Common interfaces for ORD exchange across RDM platforms

[ml-evs]

Working group notes building on #2

[edan-bainglass]

We discussed the RO-Crate Schema Plus specification developed by openBIS, conceptually stress testing it as we establish understanding w.r.t its motivation and implementation details.

The current understanding is that, since openBIS is domain-less, there is a requirement for imported/exported data/metadata to also carry the corresponding schemas. This allows a domain-agnostic system such as openBIS to digest any metadata and construct from it ELN objects.

There remain some questions (that may already have answers somewhere):

1. Pro/con analysis of domain-less formats - what are the burden/implications of no predefined schemas?
2. Explicit vs. implicit - what is the motivation for explicit schema definition vs profile conformation?
3. Possibly more to be added later

We also discussed more concretely as a working example the use case of PREMISE - transfer of data+metadata between an ELN/LIMS and a WFMS in the context of a request for a simulation. Specifically, an ELN client requests a simulation given a structure description (e.g., molecule), with the final goal being a new simulation object in the ELN (metadata+semantics) with attached datasets. A demonstration (video) of such a process leveraging direct platform-specific APIs can be found here.

To decouple from the platform-specific APIs, @edan-bainglass proposed a crate-based two-way interaction:

1. ELN sends a request crate - sender/context metadata + action plan, which can be as minimal as "I want a simulation on this structure" and as detailed as "I want THIS simulation on this structure with THESE inputs", etc.
2. Intermediate comms for progress would be useful (likely JSON-based)
3. In the minimal case, a response could be a URL to the WFMS platform, for the user to visit and manually build the simulation on the already-consumed structure. This would be a phase I (at MADICES) target! Advancing this further is out of scope (detailed below).
4. On a successful simulation, WFMS sends a response crate, with complete metadata covering the full provenance of the simulation, following the openBIS standard - the crate is self-contained, providing also its metadata schema.
5. ELN constructs from the shipped crate the new simulation object and assigns the metadata + datasets.

Potential deliverables

• A minimal demonstrator of the above - here we can build on the interoperability demo developed in MADICES 2
• A request protocol
• A response protocol
• A final report of what was achieved/learned
• A plan for continued engagement on the topic beyond MADICES

[simontaurus]

Please also see the broader discussion general RO-Crate / .eln Profiles, e.g. using OO-LD

[edan-bainglass]

Thank you @simontaurus. It would be interesting to see if there is anything offered by the openBIS spec that cannot be covered by OO-LD via the JSON-Schema spec.

[edan-bainglass]

@simontaurus great if you can have a look here and give your feedback regarding the openBIS approach, and more importantly for me, if OO-LD can reproduce this fully. More broadly, can OO-LD capture all the requirements specified in the openBIS spec?

[simontaurus]

@edan-bainglass, @juan-fuentes-sis and @AndreasMeier12
Did I get it right that you want export the vocabular created in a particular OpenBIS instance in a RO-Create?
First of all it looks like you mix the weak logic of schema.org vocabs with the strict of OWL. Since OWL is more widly used and more expressive you may stick to it.
I would not link the vocabular as direct part of the Dataset, but as part of a top-level owl:Ontology entiry. You can then compile a JSON-LD context from it which you put in the RO-Crate @context or in a seperate file referenced within the RO-Crate @context.
To model owl:Ontologies (owl:Class, owl:Property) we already created OO-LD schema (see, e.g., OwlClass Demo, Schema), which also allows to model them with pydantic in python.

[edan-bainglass]

Thanks @simontaurus. Indeed, from my own experience playing around with this, I also leaned towards encapsulating schemas/ontologies as crate payloads, with context references added to the manifest context. So something like

crate/
  ro-crate-manifest.json
  paper.pdf
  data/
    results.csv
  schema/
    experiment.schema.json
  ontology/
    publication.ttl
  shapes/
    experiment.shapes.ttl
  ...

The organization is not something to enforce. But in general, I think not only this is good practice, but it also removes any requirements on format for schema/ontology authoring. Each crate producer (platform) can use their own tools. The standard comes via the manifest, where all is referenced. So OO-LD, linkML, etc. are left for each platform to decide on. But the manifest is the source of truth to be acted on.

What is missing in this approach?

[edan-bainglass]

@JFRudzinski representing the NOMAD ELN (FAIRmat) has expressed initial concern w.r.t the openBIS spec regarding how one resolves links? (@JFRudzinski correct me if I'm wrong). Further discussion is warranted on this, perhaps including @hampusnasstrom. Let's fix a call.

[edan-bainglass]

@jan-janssen is in favor of REST API interactions (motivated in part by a potentially inevitable agentic future). Through some discussion, also with @giovannipizzi, a current compromise is to avoid by default the exchange of raw files (possibly leaving the option) in favor of links to storage locations. This in principle reduces crates to their manifests (including file URLs), which still covers the requirements stated above.

This shifts the focus to two goals:

1. Still, how to standardize the request shape
2. Less of a priority, how to best handle files (technical, TBD later)

For 1, the discussion should consider along with ELN/WFMS, and now agents, also the FINALES broker (and other such systems), which register tenants via JSON schemas. @jan-janssen @giovannipizzi point here to a potential use case of semantics in FINALES to mark quantities and methods by meaning, possibly improving discovery and/or reducing the burden on the schema requirements. @jan-janssen @giovannipizzi if you can please say more?

[edan-bainglass]

Also, @jan-janssen made a comment that perhaps systems like FINALES could be the REST-API-based interaction he had in mind.

[edan-bainglass]

An alternative(?) has emerged as a means for mediating interoperability across platforms - FINALES. Briefly, recasting the PREMISE use case as a FINALES-brokered workflow:

1. PLATFORM 1 pings FINALES for available capabilities
2. PLATFORM 2 posts a request for a simulation following the schema of an available capability (provided by PLATFORM 2 - a registered FINALES tenant)
3. PLATFORM 2 picks up the job, processes it, finally responding back to FINALES
4. PLATFORM 1 picks up the response from FINALES

The above is a top-level overview of the interaction. However, several details can be further described w.r.t the PREMISE case, some not yet supported. For example, the concept of interactive capabilities has yet to be considered in FINALES. However, through discussions with @MonikaVo of FINALES, it is clear that the PREMISE use case is not the only one requiring such a mechanism. In other words, there exists motivation to "upgrade" FINALES for this support.

To keep this comment brief, much of the discussion on this topic is summarized in reports by @MonikaVo and @giovannipizzi, to be linked here. The approach was then discussed also with the MADICES community, which so far appears to be in favor.

@giovannipizzi developed a minimal PoC demonstrating the concept, to be linked here. Further discussion in the coming days will aim to flesh this PoC out w.r.t the PREMISE project, in collaboration with the FINALES team, also to test it in practice.

[MonikaVo]

Extension of FINALES with the concept of partial results

@MonikaVo, @giovannipizzi
MADICES 2025, PSI, 23 October 2025

Introduction

Here, we propose to extend FINALES with the concept of partial results (and possibly partial requests).

The idea is that the tenant that reserves a request and starts operating on it, can post partial results at any time until it completes (and thus posts the final result, setting the status of the request to RESOLVED).

If a capability supports partial results, their cntent must comply to a JSON schema. This schema is in general a third schema, i.e. different both from the request schema and from the results schema.

Note: In general, it could be possible that the results schema is the same as the partial results schema, in which case, the two schemas are just the same. But in general, they could be different (e.g. a "progress" float field ranging between 0% and 100% is not useful in the final results; in the same way, some final results can only be generated at the very end, and are never available as partial results). One could also consider to implement the partial results as a kind of "wrapper" around the regular result schema. In this way, a final result might be deduced from a partial result in case a process needs to be cancelled. In this context, the "progress" field could even make sense in final results.

From the point of view of the user, one only sees the most recent partial result. A new partial result posted by the tenant will typically overwrite the previous results. This is also a consideration of size and performance, since for a given experiment, there could be partial results posted more often.
FINALES can of course keep track of all posted requests, but there should be 1) some easy-to-use functionality to clean up all previous states except the last one, if space needs to be saved; 2) functionality (automated when a request is RESOLVED, or upon user request) to clean up also the latest partial result of any RESOLVED request that posted a result (since in general, one should at that point only look at the final results, that should contain all relevant information).

Probably, keeping the last partial result for RESOLVED requests should be on only if the user is running FINALES in a debugging mode; otherwise, it is clearer for the user if these are removed (as they might not contain all relevant data, but only data up to the last time the partial result was updated; the final results might contain also some final data points). Not cleaning it might result in confusion by the user, who might not know whether to check the partial results or the final results.

Goal

• communicate back information to the "caller user" (whoever posted the new request), to, e.g.:
  • monitor progress, so e.g. the user can see the % of the progress, can show intermediate plots of the data measured/simulated until now, ...
  • communicate a request for action. Examples:
    • ask for confirmation by user
    • ask user to fix an error on the tenant that requires human action
    • filling missing input information, e.g. if the user asks for a simulation of a molecule and a GUI is available to select the exact details)
    • early finshing a task if the data does not look promising (similar to a data stream) (could be interesting for long experiments such as cycling)
  • improve OVERLORT functionalities (e.g. OVERLORT can communicate back which sub-requests it submitted, OVERLORT processes can be monitored via FINALES (currently only the OVERLORT knows about the relation between the (sub-)requests and results until the final result is posed)).
    • note: OVERLORT can (and should) proxy the (relevant subset of) partial results from the partial results of the subtasks it spawned. Meaning it should monitor the process of the workflow and communicate relevant events to the user that submitted the request. The idea is that the "caller user" would in general look and monitor only the partial results of the requests it submitted, not of other related requests

How to use this mechanism to ask for human-feedback interaction

We envision three use cases in which a human interaction is needed.

Perform an action as part of the workflow ("human tenant")

In this case, an action is required as part of the workflow. The imporant aspects defining this usecase are:

• the human performing the action does not need to be aware of the full workflow: they just need to know which action to perform
• the human performing the action might not be the same user who triggered the top workflow.

An example: as part of a autonomous workflow to create and test batteries, one of the tasks is to move (at a given point) the battery from the robot to the cycler.

This use case is already covered by FINALES, and does not require partial results (even if of course they can be helpful to communicate partial results to the original user). This is implemented by OVERLORT. A new FINALES request is submitted for the human action; the human is notified in some way (e.g. email); the human performs the action, and at the end posts a response in FINALES to confirm the action has been successfully completed; any orchestrating tenant who submitted the request will notice that it is now RESOLVED, and will continue.

Ask for user feedback to the same user who submitted the request

In this case, instead, the action is required by the user who submitted the workflow.
This is typcically for action confirmation, to ask for explicit inspection before continuing (or killing the request if something is going wrong), or to ask for additional inputs that are not part of the request (e.g. the request is to simulate a molecule on a substrate, and more info is required by the tenant; the tenant provides a GUI to fill what is missing [e.g. selecting the substrate, positioning the molecule on the substrate, selecting numerical parameters, ...] and actually submit the calculaction).

In this case, the FINALES capability would also have the third schema for the partial results, and one of the field specified in the schema could be an optional URL string keyword, say "human_feedback_url". If this is specified and not null, it indicates that the workflow is paused and is waiting for more feedback. The feedback can be provided by opening the provided link in a browser, following instructions there, and posting/submitting them via that web page.

This web page is usually provided by the tenant. When the results are submitted, the tenant would know that it has to post a new patial result, removing the "human_feedback_url" field.
This effectively communicates back to the submitting user that no human interaction is required and the workflow is now continuing.

NOTE: at a later time, the same tenant can put a new "human_feedback_url" in the partial response, indicating that it needs further interaction (the information provided was not sufficient, or a later part of the workflow has been reached and more interaction is required). So in general, for a given request, several human interactions could be asked at different times, before RESOLVING the request.

Usage in a web GUI to submit requests to FINALES

The usage could be the following.
Let's imagine a ELN that has a button to submit a request (e.g. for a simulation) to a WFMS (workflow management system).

The ELN would post the request, knowing that the partial results exist, and one of them is about a "human_feedback_url", that when set, requires human interaction.
This means that the ELN is asking to perform an operation that can potentially require human interaction.

In any case, even without human interaction, the ELN would have to poll FINALES to know if the request is RESOLVED, and show the results to the user (and potentially import them).

If partial results exist, the ELN could show them visually to the user to indicate progress.

If a human feedback is required (of the second category discussed above, i.e. by the submitter), the ELN would know because of the schema of the (partial results associated to the) request, and thus also monitor those while polling, and specifically the field "human_feedback_url".

If a non-null value is found, this is shown in a very visible way (say, in a yellow box) to the user, explaining that interaction is required and they have to open the provided link.
Clicking will open the link in a new tab, and the user can perform the operation.
At the end, the tab can be closed (and the tenant, behind the scenes, will have updated the partial response removing the "human_feedback_url").

When going back to the ELN (and possibly waiting a few seconds and refreshing the page, or waiting for the ELN to perform another poll of the partial response), the user will not see anymore the yellow message, and will therefore just continue waiting (either to get the final results, or for a new "yellow-box" human interaction request to be shown).

Error handling in tenants requiring human intervention

In this usecase, a monitoring tenant could be implemented to notify a user in case the tenant runs into an error and an incomplete request therefore changes its status to human_intervention. It would be in the discretion of the person responsible for a tenant to decide which errors trigger a change in the status field to human_intervention and which ones don't.

There are errors that require direct access to the tenant for troubleshooting. Such tasks might lead to a change in status but the notification about the details of the error might not be sent to the user submitting the request but rather to the maintainer(s) of the tenant. Ideally, in this use case, if the URL is opened by the submitting user, they will not be authorized and should get a message that the action needs to be performed by an admin.

Other use cases, where the error might be that a process failed after several attempts. In this case a tenant could provide some reasoning why this is and provide hints to the user posting the request about how to alter its request to increase the chance of success.

Extension of FINALES with the concept of subrequests

Current situation

Workflows involving actions from various tenants to answer a request are managed by the OVERLORT. Although, this concept works, it is not possible to reconstruct the relation between the requests and results generated in a workflow from the FINALES database until the final workflow-result is posted that contains the list of involved requests and results.

Goal

The goal of subrequests is to make the links between requests that belong to a workflow explicit in the FINALES database. In this way, the current status of a workflow would be more clearly visible and a workflow could be traced within the FINALES database (e.g. by the user who submitted the request for the workflow but does not know about the details of the workflow). Subrequests and their related results should not be deleted when finishing the workflow, but their link to the "initial request" should remain.

Properties

In addition to a regular request, a subrequest requires a field, in which the parent request can be linked. Apart from that, it is a regular request, which gets answered by a result.

[giovannipizzi]

The mock-up code can be found in this repo: https://github.com/MADICES/interop-demo-human-interaction

[edan-bainglass]

@MonikaVo let's fix a call to discuss moving this from mock-up to proper PoC 💪