Benchmark: timeseries_statistics_s2_full

[github-actions]

Benchmark scenario ID: timeseries_statistics_s2_full
Benchmark scenario definition: https://github.com/ESA-APEx/apex_algorithms/blob/9fe9397b8f7afe69b3b7227b6391238ff3895914/algorithm_catalog/vito/timeseries_statistics/benchmark_scenarios/timeseries_statistics.json
openEO backend: openeo.dataspace.copernicus.eu

GitHub Actions workflow run: https://github.com/ESA-APEx/apex_algorithms/actions/runs/24815810963
Workflow artifacts: https://github.com/ESA-APEx/apex_algorithms/actions/runs/24815810963#artifacts

Test start: 2026-04-23 03:56:06.455388+00:00
Test duration: 0:00:03.059170
Test outcome: ❌ failed

Last successful test phase: connect
Failure in test phase: create-job

Contact Information

Name	Organization	Contact
VITO	n/a	SEE WEBSITE (VITO Website)

Process Graph

{
  "tsstats1": {
    "process_id": "timeseries_statistics",
    "namespace": "https://raw.githubusercontent.com/ESA-APEx/apex_algorithms/refs/heads/main/algorithm_catalog/vito/timeseries_statistics/openeo_udp/timeseries_statistics.json",
    "arguments": {
      "collection_id": "SENTINEL2_L2A",
      "bands": [
        "B04",
        "B08"
      ],
      "geometry": {
        "type": "Polygon",
        "coordinates": [
          [
            [
              4.97,
              51.19
            ],
            [
              5.07,
              51.19
            ],
            [
              5.07,
              51.28
            ],
            [
              4.97,
              51.28
            ],
            [
              4.97,
              51.19
            ]
          ]
        ]
      },
      "temporal_extent": [
        "2023-05-01",
        "2023-07-31"
      ]
    },
    "result": true
  }
}

Error Logs

scenario = BenchmarkScenario(id='timeseries_statistics_s2_full', description='Full-extent statistics for Sentinel-2 L2A Red and N...gorithms/apex_algorithms/algorithm_catalog/vito/timeseries_statistics/benchmark_scenarios/timeseries_statistics.json'))
connection_factory = <function connection_factory.<locals>.get_connection at 0x7f27e09d6840>
tmp_path = PosixPath('/home/runner/work/apex_algorithms/apex_algorithms/qa/benchmarks/tmp_path_root/test_run_benchmark_timeseries_0')
track_metric = <function track_metric.<locals>.track at 0x7f27e09d6980>
track_phase = <apex_algorithm_qa_tools.pytest.pytest_track_metrics._PhaseTracker object at 0x7f27e09ce3f0>
upload_assets_on_fail = <apex_algorithm_qa_tools.pytest.pytest_upload_assets.upload_assets_on_fail.<locals>._Collector object at 0x7f27e09ec3b0>
request = <FixtureRequest for <Function test_run_benchmark[timeseries_statistics_s2_full]>>

    @pytest.mark.parametrize(
        "scenario",
        [
            # Use scenario id as parameterization id to give nicer test names.
            pytest.param(uc, id=uc.id)
            for uc in get_benchmark_scenarios()
        ],
    )
    def test_run_benchmark(
        scenario: BenchmarkScenario,
        connection_factory,
        tmp_path: Path,
        track_metric,
        track_phase,
        upload_assets_on_fail,
        request,
    ):
        track_metric("scenario_id", scenario.id)

        with track_phase(phase="connect"):
            # Check if a backend override has been provided via cli options.
            override_backend = request.config.getoption("--override-backend")
            backend_filter = request.config.getoption("--backend-filter")
            if backend_filter and not re.match(backend_filter, scenario.backend):
                # TODO apply filter during scenario retrieval, but seems to be hard to retrieve cli param
                pytest.skip(
                    f"skipping scenario {scenario.id} because backend {scenario.backend} does not match filter {backend_filter!r}"
                )
            backend = scenario.backend
            if override_backend:
                _log.info(f"Overriding backend URL with {override_backend!r}")
                backend = override_backend

            connection: openeo.Connection = connection_factory(url=backend)

        report_path = None
        if request.config.getoption("--upload-benchmark-report"):
            report_path = tmp_path / "benchmark_report.json"
            report_path.write_text(json.dumps({
                "scenario_id": scenario.id,
                "scenario_description": scenario.description,
                "scenario_backend": scenario.backend,
                "scenario_source": str(scenario.source) if scenario.source else None,
                "reference_data": scenario.reference_data,
                "reference_options": scenario.reference_options,
            }, indent=2))
            upload_assets_on_fail(report_path)

        def _on_phase_exception(phase: str, exc: Exception):
            if report_path is not None:
                report = json.loads(report_path.read_text())
                report["test_failed"] = True
                report["test_failed_phase"] = phase
                report["test_error_message"] = str(exc)
                report_path.write_text(json.dumps(report, indent=2))
                cwd_report_dir = Path("benchmark_reports")
                cwd_report_dir.mkdir(exist_ok=True)
                (cwd_report_dir / f"{scenario.id}_benchmark_report.json").write_text(
                    json.dumps(report, indent=2)
                )
                report_url = upload_assets_on_fail.get_url(report_path)
                if report_url:
                    exc.add_note(f"Benchmark report: {report_url}")

        track_phase.on_exception = _on_phase_exception

        with track_phase(phase="create-job"):
            # TODO #14 scenario option to use synchronous instead of batch job mode?
>           job = connection.create_job(
                process_graph=scenario.process_graph,
                title=f"APEx benchmark {scenario.id}",
                additional=scenario.job_options,
            )

tests/test_benchmarks.py:92:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/openeo/rest/connection.py:1899: in create_job
    response = self.post("/jobs", json=pg_with_metadata, expected_status=HTTP_201_CREATED)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/openeo/rest/_connection.py:232: in post
    return self.request("post", path=path, json=json, allow_redirects=False, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/openeo/rest/connection.py:774: in request
    return _request()
           ^^^^^^^^^^
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/openeo/rest/connection.py:767: in _request
    return super(Connection, self).request(
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/openeo/rest/_connection.py:141: in request
    self._raise_api_error(resp)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <Connection to 'https://openeo.dataspace.copernicus.eu/openeo/1.2/' with OidcBearerAuth>
response = <Response [502]>

    def _raise_api_error(self, response: requests.Response):
        """Convert API error response to Python exception"""
        status_code = response.status_code
        try:
            info = response.json()
        except Exception:
            info = None

        # Valid JSON object with "code" and "message" fields indicates a proper openEO API error.
        if isinstance(info, dict):
            error_code = info.get("code")
            error_message = info.get("message")
            if error_code and isinstance(error_code, str) and error_message and isinstance(error_message, str):
                raise OpenEoApiError(
                    http_status_code=status_code,
                    code=error_code,
                    message=error_message,
                    id=info.get("id"),
                    url=info.get("url"),
                )

        # Failed to parse it as a compliant openEO API error: show body as-is in the exception.
        text = response.text
        error_message = None
        _log.warning(f"Failed to parse API error response: [{status_code}] {text!r} (headers: {response.headers})")

        # TODO: eliminate this VITO-backend specific error massaging?
        if status_code == HTTP_502_BAD_GATEWAY and "Proxy Error" in text:
            error_message = (
                "Received 502 Proxy Error."
                " This typically happens when a synchronous openEO processing request takes too long and is aborted."
                " Consider using a batch job instead."
            )

>       raise OpenEoApiPlainError(message=text, http_status_code=status_code, error_message=error_message)
E       openeo.rest.OpenEoApiPlainError: [502] Bad Gateway

/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/openeo/rest/_connection.py:184: OpenEoApiPlainError
------------------------------ Captured log call -------------------------------
INFO     conftest:conftest.py:145 Connecting to 'openeo.dataspace.copernicus.eu'
INFO     openeo.config:config.py:193 Loaded openEO client config from sources: []
INFO     conftest:conftest.py:158 Checking for auth_env_var='OPENEO_AUTH_CLIENT_CREDENTIALS_CDSEFED' to drive auth against url='openeo.dataspace.copernicus.eu'.
INFO     conftest:conftest.py:162 Extracted provider_id='CDSE' client_id='openeo-apex-benchmarks-service-account' from auth_env_var='OPENEO_AUTH_CLIENT_CREDENTIALS_CDSEFED'
INFO     openeo.rest.connection:connection.py:302 Found OIDC providers: ['CDSE']
INFO     openeo.rest.auth.oidc:oidc.py:410 Doing 'client_credentials' token request 'https://identity.dataspace.copernicus.eu/auth/realms/CDSE/protocol/openid-connect/token' with post data fields ['grant_type', 'client_id', 'client_secret', 'scope'] (client_id 'openeo-apex-benchmarks-service-account')
INFO     openeo.rest.connection:connection.py:401 Obtained tokens: ['token_type', 'access_token', 'expires_in', 'id_token', 'scope']
WARNING  openeo.rest._connection:_connection.py:174 Failed to parse API error response: [502] 'Bad Gateway' (headers: {'Date': 'Thu, 23 Apr 2026 03:56:09 GMT', 'Content-Length': '11'})