Test times out waiting for model

[kimwnasptd]

Bug Description

This looks to be the same issue as canonical/resource-dispatcher#86

While all the integration tests are passing though, only the bundle tests are the ones that fail

1. workflow yaml
2. test_charms.py

Locally though I haven't managed to reproduce as I'm bumping into a different error that I'll share in the comments.

https://github.com/canonical/notebook-operators/actions/runs/13542756086/job/37848778215?pr=435

To Reproduce

1. Trigger the on_pull event from commit c56a97f

Environment

1. Code from commit c56a97f
2. MicroK8s, JuJu

Relevant Log Output

tests/test_charms.py::test_prometheus_grafana_integration 
-------------------------------- live log call ---------------------------------
INFO     juju.model:model.py:2284 Deploying ch:amd64/focal/prometheus-k8s-228
...
INFO     juju.model:model.py:3211 Waiting for model:
  grafana-k8s/0 [idle] maintenance: installing charm software
INFO     juju.model:model.py:3211 Waiting for model:
  grafana-k8s/0 [idle] maintenance: installing charm software
FAILED
------------------------------ live log teardown -------------------------------
INFO     httpx:_client.py:1038 HTTP Request: DELETE https://10.1.0.18:16443/api/v1/namespaces/kubeflow-user "HTTP/1.1 200 OK"
INFO     httpx:_client.py:1038 HTTP Request: DELETE https://10.1.0.18:16443/api/v1/namespaces/kubeflow-user/serviceaccounts/default-editor "HTTP/1.1 200 OK"
INFO     pytest_operator.plugin:plugin.py:903 Model status:

Model     Controller                Cloud/Region        Version  SLA          Timestamp
kubeflow  github-pr-d0cb9-microk8s  microk8s/localhost  3.6.2    unsupported  12:20:21Z

App                           Version  Status       Scale  Charm                         Channel         Rev  Address         Exposed  Message
admission-webhook                      active           1  admission-webhook             latest/edge     404  10.152.183.83   no       
grafana-k8s                   9.5.3    maintenance      1  grafana-k8s                   latest/stable   133  10.152.183.92   no       installing charm software
istio-ingressgateway                   active           1  istio-gateway                 latest/edge    1378  10.152.183.129  no       
istio-pilot                            active           1  istio-pilot                   latest/edge    1321  10.152.183.193  no       
jupyter-controller                     active           1  jupyter-controller                              0  10.152.183.164  no       
jupyter-ui                             active           1  jupyter-ui                                      0  10.152.183.24   no       
kubeflow-dashboard                     active           1  kubeflow-dashboard            latest/edge     747  10.152.183.123  no       
kubeflow-profiles                      active           1  kubeflow-profiles             latest/edge     539  10.152.183.170  no       
prometheus-k8s                2.52.0   active           1  prometheus-k8s                latest/stable   228  10.152.183.101  no       
prometheus-scrape-config-k8s  n/a      active           1  prometheus-scrape-config-k8s  latest/stable    56  10.152.183.191  no       

Unit                             Workload     Agent  Address     Ports  Message
admission-webhook/0*             active       idle   10.1.72.80         
grafana-k8s/0*                   maintenance  idle   10.1.72.89         installing charm software
istio-ingressgateway/0*          active       idle   10.1.72.75         
istio-pilot/0*                   active       idle   10.1.72.74         
jupyter-controller/0*            active       idle   10.1.72.79         
jupyter-ui/0*                    active       idle   10.1.72.78         
kubeflow-dashboard/0*            active       idle   10.1.72.82         
kubeflow-profiles/0*             active       idle   10.1.72.81         
prometheus-k8s/0*                active       idle   10.1.72.88         
prometheus-scrape-config-k8s/0*  active       idle   10.1.72.85         
...

=================================== FAILURES ===================================
_____________________ test_prometheus_grafana_integration ______________________
Traceback (most recent call last):
...
  File "/home/runner/work/notebook-operators/notebook-operators/tests/test_charms.py", line 347, in test_prometheus_grafana_integration
    await ops_test.model.wait_for_idle(status="active", timeout=60 * 20)
  File "/home/runner/work/notebook-operators/notebook-operators/.tox/integration/lib/python3.8/site-packages/juju/model.py", line 3209, in wait_for_idle
    raise jasyncio.TimeoutError("Timed out waiting for model:\n" + busy)
asyncio.exceptions.TimeoutError: Timed out waiting for model:
grafana-k8s/0 [idle] maintenance: installing charm software

Additional Context

I can't manage to reproduce this error when running the tests locally though.

[syncronize-issues-to-jira]

Thank you for reporting your feedback to us!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-7041.

This message was autogenerated

[kimwnasptd]

Locally though, when I try to build the charms I'm bumping into the following error, but maybe it has to do with my env?

1. JuJu 3.6.3
2. charmcraft 3.4.3

tests/test_charms.py::test_build_and_deploy
--------------------------------------------------------------------------- live log setup ---------------------------------------------------------------------------
INFO     pytest_operator.plugin:plugin.py:766 Connecting to existing model sibyl:kubeflow on unspecified cloud
INFO     httpx:_client.py:1038 HTTP Request: POST https://127.0.0.1:16443/api/v1/namespaces "HTTP/1.1 201 Created"
INFO     httpx:_client.py:1038 HTTP Request: POST https://127.0.0.1:16443/api/v1/namespaces/kubeflow-user/serviceaccounts "HTTP/1.1 201 Created"
--------------------------------------------------------------------------- live log call ----------------------------------------------------------------------------
INFO     pytest_operator.plugin:plugin.py:583 Using tmp_path: /home/kimwnasptd/Code/git/notebook-operators/.tox/integration/tmp/pytest/kubeflow0
INFO     pytest_operator.plugin:plugin.py:1139 Building charm jupyter-controller
INFO     pytest_operator.plugin:plugin.py:1144 Built charm jupyter-controller in 13.40s
INFO     pytest_operator.plugin:plugin.py:583 Using tmp_path: /home/kimwnasptd/Code/git/notebook-operators/.tox/integration/tmp/pytest/kubeflow0
INFO     pytest_operator.plugin:plugin.py:1139 Building charm jupyter-ui
INFO     pytest_operator.plugin:plugin.py:1144 Built charm jupyter-ui in 13.56s
FAILED
...
  File "/home/kimwnasptd/Code/git/notebook-operators/tests/test_charms.py", line 110, in test_build_and_deploy
    await ops_test.model.deploy(
...
  File "/home/kimwnasptd/Code/git/notebook-operators/tests/test_charms.py", line 112, in test_build_and_deploy
    await ops_test.model.deploy(
  File "/home/kimwnasptd/Code/git/notebook-operators/.tox/integration/lib/python3.8/site-packages/juju/model.py", line 1871, in deploy
    res = await self.deploy_types[schema].resolve(
  File "/home/kimwnasptd/Code/git/notebook-operators/.tox/integration/lib/python3.8/site-packages/juju/model.py", line 605, in resolve
    charm_url, origin = await self.charm_resolver(
  File "/home/kimwnasptd/Code/git/notebook-operators/.tox/integration/lib/python3.8/site-packages/juju/model.py", line 2061, in _resolve_charm
    resp = await charms_facade.ResolveCharms(
  File "/home/kimwnasptd/Code/git/notebook-operators/.tox/integration/lib/python3.8/site-packages/juju/client/facade.py", line 483, in wrapper
    reply = await f(*args, **kwargs)
  File "/home/kimwnasptd/Code/git/notebook-operators/.tox/integration/lib/python3.8/site-packages/juju/client/_client6.py", line 276, in ResolveCharms
    reply = await self.rpc(msg)
  File "/home/kimwnasptd/Code/git/notebook-operators/.tox/integration/lib/python3.8/site-packages/juju/client/facade.py", line 662, in rpc
    result = await self.connection.rpc(msg, encoder=TypeEncoder)
  File "/home/kimwnasptd/Code/git/notebook-operators/.tox/integration/lib/python3.8/site-packages/juju/client/connection.py", line 597, in rpc
    raise errors.JujuAPIError(result)
juju.errors.JujuAPIError: transforming slice at index 0: cannot parse name and/or revision in URL "": name "" not valid
========================================================================== warnings summary ==========================================================================
.tox/integration/lib/python3.8/site-packages/seleniumwire/thirdparty/mitmproxy/contrib/kaitaistruct/tls_client_hello.py:10
  /home/kimwnasptd/Code/git/notebook-operators/.tox/integration/lib/python3.8/site-packages/seleniumwire/thirdparty/mitmproxy/contrib/kaitaistruct/tls_client_hello.py:10: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    from pkg_resources import parse_version

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
====================================================================== short test summary info =======================================================================
FAILED tests/test_charms.py::test_build_and_deploy - juju.errors.JujuAPIError: transforming slice at index 0: cannot parse name and/or revision in URL "": name "" not valid

Not sure why we get the error, when trying to deploy Istio charm

juju.errors.JujuAPIError: transforming slice at index 0: cannot parse name and/or revision in URL "": name "" not valid

---

In the end the above issue was because of my local DNS behaving weirdly. Trying to also deploy a charm directly with juju gave me the same error as canonical/charmhub.io#2072

But after rebooting all errors mentioned in this comment were resolved.

[kimwnasptd]

Also this issue is not 100% the same with canonical/resource-dispatcher#86

In the issue above the tests failed on test_deploy_resource_dispatcher, while here they failed on test_prometheus_grafana_integration tests

[kimwnasptd]

In the end after re-running the test multiple times it unblocked. But we still don't know the root cause of this issue

[kimwnasptd]

Bumped into the same issue in canonical/kfp-operators#727, in the corresponding PR

[kimwnasptd]

Running again into the same error in canonical/minio-operator#265