Skip to content

juju config option port is not working properly #202

Open
Open
juju config option port is not working properly#202
Labels
bugSomething isn't working

Description

@rgildein
rgildein
opened on Jun 18, 2024

Bug Description

Changing the port config option will change k8s service, but not applied on kubeflow-dashboard container.

To Reproduce

$ tox -e integration -- --model kubeflow --keep-models
$ juju switch kubeflow
$ kubectl get services -n kubeflow | grep 8082
kubeflow-dashboard ClusterIP 10.152.183.57 8082/TCP 37m
$ kubectl logs -n kubeflow kubeflow-dashboard-0 --container kubeflow-dashboard --tail 1000 | grep '[kubeflow-dashboard]'
2024-06-18T09:44:44.331Z [kubeflow-dashboard]
2024-06-18T09:44:44.331Z [kubeflow-dashboard] > kubeflow-centraldashboard@0.0.2 start /app
2024-06-18T09:44:44.331Z [kubeflow-dashboard] > npm run serve
2024-06-18T09:44:44.331Z [kubeflow-dashboard]
2024-06-18T09:44:44.537Z [kubeflow-dashboard]
2024-06-18T09:44:44.537Z [kubeflow-dashboard] > kubeflow-centraldashboard@0.0.2 serve /app
2024-06-18T09:44:44.537Z [kubeflow-dashboard] > node dist/server.js
2024-06-18T09:44:44.537Z [kubeflow-dashboard]
2024-06-18T09:44:45.022Z [kubeflow-dashboard] Initializing Kubernetes configuration
2024-06-18T09:44:45.053Z [kubeflow-dashboard] Unable to fetch Application information: 404 page not found
2024-06-18T09:44:45.053Z [kubeflow-dashboard]
2024-06-18T09:44:45.056Z [kubeflow-dashboard] "other" is not a supported platform for Metrics
2024-06-18T09:44:45.057Z [kubeflow-dashboard] Using Profiles service at http://kubeflow-profiles.kubeflow:8081/kfam
2024-06-18T09:44:45.060Z [kubeflow-dashboard] Server listening on port http://localhost:8082 (in production mode)
$ curl 10.152.183.57:8082
<!doctype html><title>Kubeflow Central Dashboard</title><script src="webcomponentsjs/webcomponents-loader.js"></script><script src="webcomponentsjs/custom-elements-es5-adapter.js"></script>Please enable JavaScript to view this website.<script src="vendor.bundle.js" defer="defer"></script><script src="app.bundle.js" defer="defer"></script><script src="dashboard_lib.bundle.js" defer="defer"></script>%
$ juju config kubeflow-dashboard port=8090
$ jhack fire kubeflow-dashboard/0 update-status
Fired update-status on kubeflow-dashboard/0.
$ kubectl get services -n kubeflow | grep 8090
kubeflow-dashboard ClusterIP 10.152.183.57 8090/TCP 39m
$ kubectl logs -n kubeflow kubeflow-dashboard-0 --container kubeflow-dashboard --tail 1000 | grep '[kubeflow-dashboard]' # logs did not change
$ curl 10.152.183.57:8090
curl: (7) Failed to connect to 10.152.183.57 port 8090 after 5 ms: Couldn't connect to server

Environment

$ juju --version
3.4.3-genericlinux-amd64
$ juju status --model controller
Model Controller Cloud/Region Version SLA Timestamp
controller u1-k8s microk8s/localhost 3.4.2 unsupported 12:23:49+02:00
$ microk8s version
MicroK8s v1.28.10 revision 6829

Relevant Log Output

2024-06-18T09:43:12.318Z [pebble] HTTP API server listening on ":38813".
2024-06-18T09:43:12.318Z [pebble] Started daemon.
2024-06-18T09:43:42.411Z [pebble] GET /v1/notices?timeout=30s 30.002412765s 200
2024-06-18T09:44:07.191Z [pebble] GET /v1/plan?format=yaml 117.266µs 200
2024-06-18T09:44:07.193Z [pebble] POST /v1/layers 227.128µs 200
2024-06-18T09:44:12.411Z [pebble] GET /v1/notices?timeout=30s 30.000449346s 200
2024-06-18T09:44:42.412Z [pebble] GET /v1/notices?timeout=30s 30.000406686s 200
2024-06-18T09:44:44.102Z [pebble] GET /v1/plan?format=yaml 138.219µs 200
2024-06-18T09:44:44.116Z [pebble] POST /v1/layers 202.753µs 200
2024-06-18T09:44:44.123Z [pebble] POST /v1/services 2.106944ms 202
2024-06-18T09:44:44.125Z [pebble] Service "kubeflow-dashboard" starting: npm start
2024-06-18T09:44:44.331Z [kubeflow-dashboard] 
2024-06-18T09:44:44.331Z [kubeflow-dashboard] > kubeflow-centraldashboard@0.0.2 start /app
2024-06-18T09:44:44.331Z [kubeflow-dashboard] > npm run serve
2024-06-18T09:44:44.331Z [kubeflow-dashboard] 
2024-06-18T09:44:44.537Z [kubeflow-dashboard] 
2024-06-18T09:44:44.537Z [kubeflow-dashboard] > kubeflow-centraldashboard@0.0.2 serve /app
2024-06-18T09:44:44.537Z [kubeflow-dashboard] > node dist/server.js
2024-06-18T09:44:44.537Z [kubeflow-dashboard] 
2024-06-18T09:44:45.022Z [kubeflow-dashboard] Initializing Kubernetes configuration
2024-06-18T09:44:45.053Z [kubeflow-dashboard] Unable to fetch Application information: 404 page not found
2024-06-18T09:44:45.053Z [kubeflow-dashboard] 
2024-06-18T09:44:45.056Z [kubeflow-dashboard] "other" is not a supported platform for Metrics
2024-06-18T09:44:45.057Z [kubeflow-dashboard] Using Profiles service at http://kubeflow-profiles.kubeflow:8081/kfam
2024-06-18T09:44:45.060Z [kubeflow-dashboard] Server listening on port http://localhost:8082 (in production mode)
2024-06-18T09:44:45.128Z [pebble] GET /v1/changes/1/wait?timeout=4.000s 1.00448347s 200
2024-06-18T09:44:45.333Z [pebble] Cannot flush logs to target "grafana-agent-k8s/0": Post "http://grafana-agent-k8s-0.grafana-agent-k8s-endpoints.kubeflow.svc.cluster.local:3500/loki/api/v1/push": dial tcp 10.1.23.214:3500: connect: connection refused
2024-06-18T09:45:12.413Z [pebble] GET /v1/notices?timeout=30s 30.000981728s 200
2024-06-18T09:45:32.524Z [pebble] GET /v1/plan?format=yaml 169.997µs 200
2024-06-18T09:45:32.541Z [pebble] POST /v1/layers 242.493µs 200
2024-06-18T09:45:32.541Z [pebble] Cannot flush logs to target "grafana-agent-k8s/0": Post "http://grafana-agent-k8s-0.grafana-agent-k8s-endpoints.kubeflow.svc.cluster.local:3500/loki/api/v1/push": dial tcp 10.1.23.214:3500: connect: connection refused
2024-06-18T09:45:32.548Z [pebble] POST /v1/services 2.143053ms 202
2024-06-18T09:45:32.554Z [pebble] GET /v1/changes/2/wait?timeout=4.000s 5.191055ms 200
2024-06-18T09:45:33.543Z [pebble] Cannot flush logs to target "grafana-agent-k8s/0": Post "http://grafana-agent-k8s-0.grafana-agent-k8s-endpoints.kubeflow.svc.cluster.local:3500/loki/api/v1/push": dial tcp 10.1.23.214:3500: connect: connection refused
2024-06-18T09:45:34.060Z [pebble] GET /v1/plan?format=yaml 177.959µs 200
2024-06-18T09:45:34.075Z [pebble] POST /v1/layers 193.255µs 200
2024-06-18T09:45:34.076Z [pebble] Cannot flush logs to target "grafana-agent-k8s/0": Post "http://grafana-agent-k8s-0.grafana-agent-k8s-endpoints.kubeflow.svc.cluster.local:3500/loki/api/v1/push": dial tcp 10.1.23.214:3500: connect: connection refused
2024-06-18T09:45:34.084Z [pebble] POST /v1/services 3.110373ms 202
2024-06-18T09:45:34.093Z [pebble] GET /v1/changes/3/wait?timeout=4.000s 9.215389ms 200
2024-06-18T09:45:35.077Z [pebble] Cannot flush logs to target "grafana-agent-k8s/0": Post "http://grafana-agent-k8s-0.grafana-agent-k8s-endpoints.kubeflow.svc.cluster.local:3500/loki/api/v1/push": dial tcp 10.1.23.214:3500: connect: connection refused
2024-06-18T09:45:35.703Z [pebble] GET /v1/plan?format=yaml 165.248µs 200
2024-06-18T09:45:35.719Z [pebble] POST /v1/layers 191.578µs 200
2024-06-18T09:45:35.720Z [pebble] Cannot flush logs to target "grafana-agent-k8s/0": Post "http://grafana-agent-k8s-0.grafana-agent-k8s-endpoints.kubeflow.svc.cluster.local:3500/loki/api/v1/push": dial tcp 10.1.23.214:3500: connect: connection refused
2024-06-18T09:45:35.727Z [pebble] POST /v1/services 2.300198ms 202
2024-06-18T09:45:35.737Z [pebble] GET /v1/changes/4/wait?timeout=4.000s 8.853604ms 200
2024-06-18T09:45:36.722Z [pebble] Cannot flush logs to target "grafana-agent-k8s/0": Post "http://grafana-agent-k8s-0.grafana-agent-k8s-endpoints.kubeflow.svc.cluster.local:3500/loki/api/v1/push": dial tcp 10.1.23.214:3500: connect: connection refused
2024-06-18T09:45:37.294Z [pebble] GET /v1/plan?format=yaml 198.912µs 200
2024-06-18T09:45:37.311Z [pebble] POST /v1/layers 188.366µs 200
2024-06-18T09:45:37.312Z [pebble] Cannot flush logs to target "grafana-agent-k8s/0": Post "http://grafana-agent-k8s-0.grafana-agent-k8s-endpoints.kubeflow.svc.cluster.local:3500/loki/api/v1/push": dial tcp 10.1.23.214:3500: connect: connection refused
2024-06-18T09:45:37.318Z [pebble] POST /v1/services 2.09486ms 202
2024-06-18T09:45:37.324Z [pebble] GET /v1/changes/5/wait?timeout=4.000s 5.26481ms 200
2024-06-18T09:45:38.313Z [pebble] Cannot flush logs to target "grafana-agent-k8s/0": Post "http://grafana-agent-k8s-0.grafana-agent-k8s-endpoints.kubeflow.svc.cluster.local:3500/loki/api/v1/push": dial tcp 10.1.23.214:3500: connect: connection refused
2024-06-18T09:45:39.481Z [pebble] GET /v1/plan?format=yaml 155.958µs 200
2024-06-18T09:45:39.496Z [pebble] POST /v1/layers 200.379µs 200
2024-06-18T09:45:39.510Z [pebble] Cannot flush logs to target "grafana-agent-k8s/0": Post "http://grafana-agent-k8s-0.grafana-agent-k8s-endpoints.kubeflow.svc.cluster.local:3500/loki/api/v1/push": dial tcp 10.1.23.214:3500: connect: connection refused
2024-06-18T09:45:39.513Z [pebble] POST /v1/services 2.184189ms 202
2024-06-18T09:45:39.519Z [pebble] GET /v1/changes/6/wait?timeout=4.000s 5.361681ms 200
2024-06-18T09:45:40.512Z [pebble] Cannot flush logs to target "grafana-agent-k8s/0": Post "http://grafana-agent-k8s-0.grafana-agent-k8s-endpoints.kubeflow.svc.cluster.local:3500/loki/api/v1/push": dial tcp 10.1.23.214:3500: connect: connection refused
2024-06-18T09:45:42.414Z [pebble] GET /v1/notices?timeout=30s 30.000359753s 200
2024-06-18T09:45:58.064Z [pebble] GET /v1/plan?format=yaml 141.711µs 200
2024-06-18T09:45:58.079Z [pebble] POST /v1/layers 204.639µs 200
2024-06-18T09:45:58.079Z [pebble] Cannot flush logs to target "grafana-agent-k8s/0": Post "http://grafana-agent-k8s-0.grafana-agent-k8s-endpoints.kubeflow.svc.cluster.local:3500/loki/api/v1/push": dial tcp 10.1.23.214:3500: connect: connection refused
2024-06-18T09:45:58.086Z [pebble] POST /v1/services 2.099261ms 202
2024-06-18T09:45:58.092Z [pebble] GET /v1/changes/7/wait?timeout=4.000s 5.182814ms 200
2024-06-18T09:45:59.081Z [pebble] Cannot flush logs to target "grafana-agent-k8s/0": Post "http://grafana-agent-k8s-0.grafana-agent-k8s-endpoints.kubeflow.svc.cluster.local:3500/loki/api/v1/push": dial tcp 10.1.23.214:3500: connect: connection refused
2024-06-18T09:46:12.415Z [pebble] GET /v1/notices?timeout=30s 30.000797234s 200
2024-06-18T09:46:15.877Z [pebble] GET /v1/plan?format=yaml 146.39µs 200
2024-06-18T09:46:15.894Z [pebble] POST /v1/layers 210.157µs 200
2024-06-18T09:46:15.895Z [pebble] Cannot flush logs to target "grafana-agent-k8s/0": Post "http://grafana-agent-k8s-0.grafana-agent-k8s-endpoints.kubeflow.svc.cluster.local:3500/loki/api/v1/push": dial tcp 10.1.23.214:3500: connect: connection refused
2024-06-18T09:46:15.902Z [pebble] POST /v1/services 2.208006ms 202
2024-06-18T09:46:15.909Z [pebble] GET /v1/changes/8/wait?timeout=4.000s 5.839544ms 200
2024-06-18T09:46:16.897Z [pebble] Cannot flush logs to target "grafana-agent-k8s/0": Post "http://grafana-agent-k8s-0.grafana-agent-k8s-endpoints.kubeflow.svc.cluster.local:3500/loki/api/v1/push": dial tcp 10.1.23.214:3500: connect: connection refused
2024-06-18T09:46:42.416Z [pebble] GET /v1/notices?timeout=30s 30.000941947s 200
2024-06-18T09:46:43.166Z [pebble] GET /v1/plan?format=yaml 239.072µs 200
2024-06-18T09:46:43.183Z [pebble] POST /v1/layers 220.494µs 200
2024-06-18T09:46:43.184Z [pebble] Cannot flush logs to target "grafana-agent-k8s/0": Post "http://grafana-agent-k8s-0.grafana-agent-k8s-endpoints.kubeflow.svc.cluster.local:3500/loki/api/v1/push": dial tcp 10.1.23.214:3500: connect: connection refused
2024-06-18T09:46:43.191Z [pebble] POST /v1/services 2.268909ms 202
2024-06-18T09:46:43.197Z [pebble] GET /v1/changes/9/wait?timeout=4.000s 5.563456ms 200
2024-06-18T09:46:44.186Z [pebble] Cannot flush logs to target "grafana-agent-k8s/0": Post "http://grafana-agent-k8s-0.grafana-agent-k8s-endpoints.kubeflow.svc.cluster.local:3500/loki/api/v1/push": dial tcp 10.1.23.214:3500: connect: connection refused
2024-06-18T09:47:12.418Z [pebble] GET /v1/notices?timeout=30s 30.001109441s 200
2024-06-18T09:47:42.419Z [pebble] GET /v1/notices?timeout=30s 30.000875421s 200
2024-06-18T09:48:12.421Z [pebble] GET /v1/notices?timeout=30s 30.001162787s 200
2024-06-18T09:48:42.421Z [pebble] GET /v1/notices?timeout=30s 30.000194951s 200
2024-06-18T09:49:12.422Z [pebble] GET /v1/notices?timeout=30s 30.00056056s 200
2024-06-18T09:49:42.423Z [pebble] GET /v1/notices?timeout=30s 30.000895098s 200
2024-06-18T09:50:12.425Z [pebble] GET /v1/notices?timeout=30s 30.001060486s 200
2024-06-18T09:50:42.425Z [pebble] GET /v1/notices?timeout=30s 30.000130538s 200
2024-06-18T09:51:12.427Z [pebble] GET /v1/notices?timeout=30s 30.001094523s 200
2024-06-18T09:51:42.427Z [pebble] GET /v1/notices?timeout=30s 30.000175696s 200
2024-06-18T09:52:12.428Z [pebble] GET /v1/notices?timeout=30s 30.000665392s 200
2024-06-18T09:52:17.466Z [pebble] GET /v1/plan?format=yaml 149.883µs 200
2024-06-18T09:52:17.481Z [pebble] POST /v1/layers 182.22µs 200
2024-06-18T09:52:17.488Z [pebble] POST /v1/services 2.19278ms 202
2024-06-18T09:52:17.494Z [pebble] GET /v1/changes/10/wait?timeout=4.000s 5.435155ms 200
2024-06-18T09:52:42.429Z [pebble] GET /v1/notices?timeout=30s 30.000716342s 200
2024-06-18T09:53:12.430Z [pebble] GET /v1/notices?timeout=30s 30.001018454s 200
2024-06-18T09:53:42.431Z [pebble] GET /v1/notices?timeout=30s 30.000506415s 200
2024-06-18T09:54:12.432Z [pebble] GET /v1/notices?timeout=30s 30.000363081s 200
2024-06-18T09:54:42.433Z [pebble] GET /v1/notices?timeout=30s 30.001054231s 200
2024-06-18T09:55:12.434Z [pebble] GET /v1/notices?timeout=30s 30.000625878s 200
2024-06-18T09:55:42.435Z [pebble] GET /v1/notices?timeout=30s 30.00065891s 200
2024-06-18T09:56:12.437Z [pebble] GET /v1/notices?timeout=30s 30.001204033s 200
2024-06-18T09:56:42.438Z [pebble] GET /v1/notices?timeout=30s 30.000840367s 200
2024-06-18T09:57:12.438Z [pebble] GET /v1/notices?timeout=30s 30.000190071s 200
2024-06-18T09:57:42.439Z [pebble] GET /v1/notices?timeout=30s 30.00059119s 200
2024-06-18T09:58:12.441Z [pebble] GET /v1/notices?timeout=30s 30.000764834s 200
2024-06-18T09:58:42.442Z [pebble] GET /v1/notices?timeout=30s 30.00122721s 200
2024-06-18T09:58:58.281Z [pebble] GET /v1/plan?format=yaml 151.418µs 200
2024-06-18T09:58:58.296Z [pebble] POST /v1/layers 203.382µs 200
2024-06-18T09:58:58.304Z [pebble] POST /v1/services 2.420118ms 202
2024-06-18T09:58:58.310Z [pebble] GET /v1/changes/11/wait?timeout=4.000s 5.901076ms 200
2024-06-18T09:59:12.443Z [pebble] GET /v1/notices?timeout=30s 30.000301886s 200
2024-06-18T09:59:30.859Z [pebble] GET /v1/plan?format=yaml 174.188µs 200
2024-06-18T09:59:30.875Z [pebble] POST /v1/layers 320.298µs 200
2024-06-18T09:59:30.884Z [pebble] POST /v1/services 2.202069ms 202
2024-06-18T09:59:30.890Z [pebble] GET /v1/changes/12/wait?timeout=4.000s 5.414552ms 200
2024-06-18T09:59:42.443Z [pebble] GET /v1/notices?timeout=30s 30.000424065s 200
2024-06-18T10:00:12.444Z [pebble] GET /v1/notices?timeout=30s 30.000407818s 200
2024-06-18T10:00:42.445Z [pebble] GET /v1/notices?timeout=30s 30.000827392s 200
2024-06-18T10:01:12.446Z [pebble] GET /v1/notices?timeout=30s 30.000355707s 200
2024-06-18T10:01:42.447Z [pebble] GET /v1/notices?timeout=30s 30.000163463s 200
2024-06-18T10:02:12.448Z [pebble] GET /v1/notices?timeout=30s 30.000839087s 200
2024-06-18T10:02:42.449Z [pebble] GET /v1/notices?timeout=30s 30.000700068s 200
2024-06-18T10:03:12.449Z [pebble] GET /v1/notices?timeout=30s 30.00038053s 200
2024-06-18T10:03:42.450Z [pebble] GET /v1/notices?timeout=30s 30.00071035s 200
2024-06-18T10:04:12.451Z [pebble] GET /v1/notices?timeout=30s 30.0007493s 200
2024-06-18T10:04:42.453Z [pebble] GET /v1/notices?timeout=30s 30.001166365s 200
2024-06-18T10:05:12.454Z [pebble] GET /v1/notices?timeout=30s 30.000334558s 200
2024-06-18T10:05:42.454Z [pebble] GET /v1/notices?timeout=30s 30.000183138s 200
2024-06-18T10:06:12.455Z [pebble] GET /v1/notices?timeout=30s 30.000888248s 200
2024-06-18T10:06:42.456Z [pebble] GET /v1/notices?timeout=30s 30.000639211s 200
2024-06-18T10:07:12.457Z [pebble] GET /v1/notices?timeout=30s 30.000771628s 200
2024-06-18T10:07:42.458Z [pebble] GET /v1/notices?timeout=30s 30.000478603s 200
2024-06-18T10:07:56.018Z [pebble] GET /v1/plan?format=yaml 199.541µs 200
2024-06-18T10:07:56.035Z [pebble] POST /v1/layers 204.15µs 200
2024-06-18T10:07:56.044Z [pebble] POST /v1/services 2.530471ms 202
2024-06-18T10:07:56.051Z [pebble] GET /v1/changes/13/wait?timeout=4.000s 6.351423ms 200
2024-06-18T10:08:12.459Z [pebble] GET /v1/notices?timeout=30s 30.000806029s 200
2024-06-18T10:08:42.461Z [pebble] GET /v1/notices?timeout=30s 30.00104039s 200
2024-06-18T10:09:12.462Z [pebble] GET /v1/notices?timeout=30s 30.000914465s 200
2024-06-18T10:09:42.463Z [pebble] GET /v1/notices?timeout=30s 30.001090783s 200
2024-06-18T10:10:12.464Z [pebble] GET /v1/notices?timeout=30s 30.000404153s 200
2024-06-18T10:10:17.695Z [kubeflow-dashboard] Unable to fetch Application information: 404 page not found
2024-06-18T10:10:17.695Z [kubeflow-dashboard] 
2024-06-18T10:10:42.466Z [pebble] GET /v1/notices?timeout=30s 30.001231253s 200
2024-06-18T10:11:12.466Z [pebble] GET /v1/notices?timeout=30s 30.000181147s 200
2024-06-18T10:11:42.467Z [pebble] GET /v1/notices?timeout=30s 30.000655732s 200
2024-06-18T10:12:12.468Z [pebble] GET /v1/notices?timeout=30s 30.000332947s 200
2024-06-18T10:12:42.469Z [pebble] GET /v1/notices?timeout=30s 30.001081222s 200
2024-06-18T10:13:12.470Z [pebble] GET /v1/notices?timeout=30s 30.001071306s 200
2024-06-18T10:13:42.471Z [pebble] GET /v1/notices?timeout=30s 30.000396298s 200
2024-06-18T10:14:12.472Z [pebble] GET /v1/notices?timeout=30s 30.000469812s 200
2024-06-18T10:14:42.473Z [pebble] GET /v1/notices?timeout=30s 30.000676813s 200
2024-06-18T10:15:12.474Z [pebble] GET /v1/notices?timeout=30s 30.000856522s 200
2024-06-18T10:15:42.475Z [pebble] GET /v1/notices?timeout=30s 30.000158744s 200
2024-06-18T10:16:12.476Z [pebble] GET /v1/notices?timeout=30s 30.001161301s 200
2024-06-18T10:16:42.477Z [pebble] GET /v1/notices?timeout=30s 30.000702413s 200
2024-06-18T10:17:12.478Z [pebble] GET /v1/notices?timeout=30s 30.000501677s 200
2024-06-18T10:17:42.479Z [pebble] GET /v1/notices?timeout=30s 30.000815624s 200
2024-06-18T10:18:12.480Z [pebble] GET /v1/notices?timeout=30s 30.001008056s 200
2024-06-18T10:18:42.481Z [pebble] GET /v1/notices?timeout=30s 30.000355195s 200
2024-06-18T10:19:12.482Z [pebble] GET /v1/notices?timeout=30s 30.00058229s 200
2024-06-18T10:19:42.484Z [pebble] GET /v1/notices?timeout=30s 30.001088417s 200
2024-06-18T10:20:12.485Z [pebble] GET /v1/notices?timeout=30s 30.000591892s 200
2024-06-18T10:20:42.486Z [pebble] GET /v1/notices?timeout=30s 30.000717604s 200
2024-06-18T10:20:58.837Z [pebble] GET /v1/plan?format=yaml 192.067µs 200
2024-06-18T10:20:58.851Z [pebble] POST /v1/layers 183.546µs 200
2024-06-18T10:20:58.858Z [pebble] POST /v1/services 2.278128ms 202
2024-06-18T10:20:58.865Z [pebble] GET /v1/changes/14/wait?timeout=4.000s 6.421544ms 200
2024-06-18T10:21:12.486Z [pebble] GET /v1/notices?timeout=30s 30.000116822s 200
2024-06-18T10:21:42.488Z [pebble] GET /v1/notices?timeout=30s 30.000973733s 200
2024-06-18T10:22:12.488Z [pebble] GET /v1/notices?timeout=30s 30.000187602s 200
2024-06-18T10:22:42.489Z [pebble] GET /v1/notices?timeout=30s 30.000381721s 200
2024-06-18T10:23:12.489Z [pebble] GET /v1/notices?timeout=30s 30.000228379s 200
2024-06-18T10:23:42.490Z [pebble] GET /v1/notices?timeout=30s 30.000343936s 200
2024-06-18T10:24:12.491Z [pebble] GET /v1/notices?timeout=30s 30.000807944s 200
2024-06-18T10:24:42.492Z [pebble] GET /v1/notices?timeout=30s 30.000221455s 200
2024-06-18T10:25:12.493Z [pebble] GET /v1/notices?timeout=30s 30.000755248s 200

unit-kubeflow-dashboard-0: 11:46:16 INFO juju.worker.uniter.operation ran "config-changed" hook (via hook dispatching script: dispatch)
unit-kubeflow-dashboard-0: 11:46:42 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/apis/apiextensions.k8s.io/v1/customresourcedefinitions "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:46:42 INFO unit.kubeflow-dashboard/0.juju-log Rendering manifests
unit-kubeflow-dashboard-0: 11:46:42 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/apis/rbac.authorization.k8s.io/v1/clusterroles/kubeflow-dashboard?force=true&fieldManager=lightkube "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:46:42 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/apis/rbac.authorization.k8s.io/v1/clusterrolebindings/kubeflow-dashboard?force=true&fieldManager=lightkube "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:46:42 INFO unit.kubeflow-dashboard/0.juju-log Reconcile completed successfully
unit-kubeflow-dashboard-0: 11:46:42 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/apis/apiextensions.k8s.io/v1/customresourcedefinitions "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:46:43 INFO unit.kubeflow-dashboard/0.juju-log Rendering manifests
unit-kubeflow-dashboard-0: 11:46:43 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/api/v1/namespaces/kubeflow/configmaps/centraldashboard-config?force=true&fieldManager=lightkube "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:46:43 INFO unit.kubeflow-dashboard/0.juju-log Reconcile completed successfully
unit-kubeflow-dashboard-0: 11:46:43 INFO unit.kubeflow-dashboard/0.juju-log Pebble plan updated with new configuration, replaning
unit-kubeflow-dashboard-0: 11:46:43 INFO juju.worker.uniter.operation ran "config-changed" hook (via hook dispatching script: dispatch)
unit-kubeflow-dashboard-0: 11:48:21 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/api/v1/namespaces/kubeflow/services/kubeflow-dashboard "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:48:21 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/api/v1/namespaces/kubeflow/services/kubeflow-dashboard "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:48:21 INFO unit.kubeflow-dashboard/0.juju-log Kubernetes service 'kubeflow-dashboard' patched successfully
unit-kubeflow-dashboard-0: 11:48:21 INFO juju.worker.uniter.operation ran "update-status" hook (via hook dispatching script: dispatch)
unit-kubeflow-dashboard-0: 11:52:17 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/apis/apiextensions.k8s.io/v1/customresourcedefinitions "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:52:17 INFO unit.kubeflow-dashboard/0.juju-log Rendering manifests
unit-kubeflow-dashboard-0: 11:52:17 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/apis/rbac.authorization.k8s.io/v1/clusterroles/kubeflow-dashboard?force=true&fieldManager=lightkube "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:52:17 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/apis/rbac.authorization.k8s.io/v1/clusterrolebindings/kubeflow-dashboard?force=true&fieldManager=lightkube "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:52:17 INFO unit.kubeflow-dashboard/0.juju-log Reconcile completed successfully
unit-kubeflow-dashboard-0: 11:52:17 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/apis/apiextensions.k8s.io/v1/customresourcedefinitions "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:52:17 INFO unit.kubeflow-dashboard/0.juju-log Rendering manifests
unit-kubeflow-dashboard-0: 11:52:17 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/api/v1/namespaces/kubeflow/configmaps/centraldashboard-config?force=true&fieldManager=lightkube "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:52:17 INFO unit.kubeflow-dashboard/0.juju-log Reconcile completed successfully
unit-kubeflow-dashboard-0: 11:52:17 INFO unit.kubeflow-dashboard/0.juju-log Pebble plan updated with new configuration, replaning
unit-kubeflow-dashboard-0: 11:52:17 INFO juju.worker.uniter.operation ran "config-changed" hook (via hook dispatching script: dispatch)
unit-kubeflow-dashboard-0: 11:53:45 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/api/v1/namespaces/kubeflow/services/kubeflow-dashboard "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:53:45 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/api/v1/namespaces/kubeflow/services/kubeflow-dashboard "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:53:45 INFO unit.kubeflow-dashboard/0.juju-log Kubernetes service 'kubeflow-dashboard' patched successfully
unit-kubeflow-dashboard-0: 11:53:45 INFO juju.worker.uniter.operation ran "update-status" hook (via hook dispatching script: dispatch)
unit-kubeflow-dashboard-0: 11:54:33 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/api/v1/namespaces/kubeflow/services/kubeflow-dashboard "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:54:33 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/api/v1/namespaces/kubeflow/services/kubeflow-dashboard "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:54:34 INFO unit.kubeflow-dashboard/0.juju-log Kubernetes service 'kubeflow-dashboard' patched successfully
unit-kubeflow-dashboard-0: 11:58:16 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/api/v1/namespaces/kubeflow/services/kubeflow-dashboard "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:58:16 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/api/v1/namespaces/kubeflow/services/kubeflow-dashboard "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:58:16 INFO unit.kubeflow-dashboard/0.juju-log Kubernetes service 'kubeflow-dashboard' patched successfully
unit-kubeflow-dashboard-0: 11:58:16 INFO juju.worker.uniter.operation ran "update-status" hook (via hook dispatching script: dispatch)
unit-kubeflow-dashboard-0: 11:58:57 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/apis/apiextensions.k8s.io/v1/customresourcedefinitions "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:58:57 INFO unit.kubeflow-dashboard/0.juju-log Rendering manifests
unit-kubeflow-dashboard-0: 11:58:57 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/apis/rbac.authorization.k8s.io/v1/clusterroles/kubeflow-dashboard?force=true&fieldManager=lightkube "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:58:57 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/apis/rbac.authorization.k8s.io/v1/clusterrolebindings/kubeflow-dashboard?force=true&fieldManager=lightkube "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:58:58 INFO unit.kubeflow-dashboard/0.juju-log Reconcile completed successfully
unit-kubeflow-dashboard-0: 11:58:58 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/apis/apiextensions.k8s.io/v1/customresourcedefinitions "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:58:58 INFO unit.kubeflow-dashboard/0.juju-log Rendering manifests
unit-kubeflow-dashboard-0: 11:58:58 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/api/v1/namespaces/kubeflow/configmaps/centraldashboard-config?force=true&fieldManager=lightkube "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:58:58 INFO unit.kubeflow-dashboard/0.juju-log Reconcile completed successfully
unit-kubeflow-dashboard-0: 11:58:58 INFO unit.kubeflow-dashboard/0.juju-log Pebble plan updated with new configuration, replaning
unit-kubeflow-dashboard-0: 11:59:30 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/apis/apiextensions.k8s.io/v1/customresourcedefinitions "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:59:30 INFO unit.kubeflow-dashboard/0.juju-log Rendering manifests
unit-kubeflow-dashboard-0: 11:59:30 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/apis/rbac.authorization.k8s.io/v1/clusterroles/kubeflow-dashboard?force=true&fieldManager=lightkube "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:59:30 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/apis/rbac.authorization.k8s.io/v1/clusterrolebindings/kubeflow-dashboard?force=true&fieldManager=lightkube "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:59:30 INFO unit.kubeflow-dashboard/0.juju-log Reconcile completed successfully
unit-kubeflow-dashboard-0: 11:59:30 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/apis/apiextensions.k8s.io/v1/customresourcedefinitions "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:59:30 INFO unit.kubeflow-dashboard/0.juju-log Rendering manifests
unit-kubeflow-dashboard-0: 11:59:30 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/api/v1/namespaces/kubeflow/configmaps/centraldashboard-config?force=true&fieldManager=lightkube "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 11:59:30 INFO unit.kubeflow-dashboard/0.juju-log Reconcile completed successfully
unit-kubeflow-dashboard-0: 11:59:30 INFO unit.kubeflow-dashboard/0.juju-log Pebble plan updated with new configuration, replaning
unit-kubeflow-dashboard-0: 11:59:31 INFO juju.worker.uniter.operation ran "config-changed" hook (via hook dispatching script: dispatch)
unit-kubeflow-dashboard-0: 12:04:08 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/api/v1/namespaces/kubeflow/services/kubeflow-dashboard "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:04:08 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/api/v1/namespaces/kubeflow/services/kubeflow-dashboard "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:04:08 INFO unit.kubeflow-dashboard/0.juju-log Kubernetes service 'kubeflow-dashboard' patched successfully
unit-kubeflow-dashboard-0: 12:04:09 INFO juju.worker.uniter.operation ran "update-status" hook (via hook dispatching script: dispatch)
unit-kubeflow-dashboard-0: 12:07:55 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/apis/apiextensions.k8s.io/v1/customresourcedefinitions "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:07:55 INFO unit.kubeflow-dashboard/0.juju-log Rendering manifests
unit-kubeflow-dashboard-0: 12:07:55 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/apis/rbac.authorization.k8s.io/v1/clusterroles/kubeflow-dashboard?force=true&fieldManager=lightkube "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:07:55 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/apis/rbac.authorization.k8s.io/v1/clusterrolebindings/kubeflow-dashboard?force=true&fieldManager=lightkube "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:07:55 INFO unit.kubeflow-dashboard/0.juju-log Reconcile completed successfully
unit-kubeflow-dashboard-0: 12:07:55 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/apis/apiextensions.k8s.io/v1/customresourcedefinitions "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:07:55 INFO unit.kubeflow-dashboard/0.juju-log Rendering manifests
unit-kubeflow-dashboard-0: 12:07:55 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/api/v1/namespaces/kubeflow/configmaps/centraldashboard-config?force=true&fieldManager=lightkube "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:07:55 INFO unit.kubeflow-dashboard/0.juju-log Reconcile completed successfully
unit-kubeflow-dashboard-0: 12:07:56 INFO unit.kubeflow-dashboard/0.juju-log Pebble plan updated with new configuration, replaning
unit-kubeflow-dashboard-0: 12:07:56 INFO juju.worker.uniter.operation ran "config-changed" hook (via hook dispatching script: dispatch)
unit-kubeflow-dashboard-0: 12:10:00 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/api/v1/namespaces/kubeflow/services/kubeflow-dashboard "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:10:00 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/api/v1/namespaces/kubeflow/services/kubeflow-dashboard "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:10:00 INFO unit.kubeflow-dashboard/0.juju-log Kubernetes service 'kubeflow-dashboard' patched successfully
unit-kubeflow-dashboard-0: 12:10:00 INFO juju.worker.uniter.operation ran "update-status" hook (via hook dispatching script: dispatch)
unit-kubeflow-dashboard-0: 12:14:46 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/api/v1/namespaces/kubeflow/services/kubeflow-dashboard "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:14:46 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/api/v1/namespaces/kubeflow/services/kubeflow-dashboard "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:14:46 INFO unit.kubeflow-dashboard/0.juju-log Kubernetes service 'kubeflow-dashboard' patched successfully
unit-kubeflow-dashboard-0: 12:14:46 INFO juju.worker.uniter.operation ran "update-status" hook (via hook dispatching script: dispatch)
unit-kubeflow-dashboard-0: 12:20:37 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/api/v1/namespaces/kubeflow/services/kubeflow-dashboard "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:20:37 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/api/v1/namespaces/kubeflow/services/kubeflow-dashboard "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:20:37 INFO unit.kubeflow-dashboard/0.juju-log Kubernetes service 'kubeflow-dashboard' patched successfully
unit-kubeflow-dashboard-0: 12:20:37 INFO juju.worker.uniter.operation ran "update-status" hook (via hook dispatching script: dispatch)
unit-kubeflow-dashboard-0: 12:20:58 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/apis/apiextensions.k8s.io/v1/customresourcedefinitions "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:20:58 INFO unit.kubeflow-dashboard/0.juju-log Rendering manifests
unit-kubeflow-dashboard-0: 12:20:58 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/apis/rbac.authorization.k8s.io/v1/clusterroles/kubeflow-dashboard?force=true&fieldManager=lightkube "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:20:58 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/apis/rbac.authorization.k8s.io/v1/clusterrolebindings/kubeflow-dashboard?force=true&fieldManager=lightkube "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:20:58 INFO unit.kubeflow-dashboard/0.juju-log Reconcile completed successfully
unit-kubeflow-dashboard-0: 12:20:58 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/apis/apiextensions.k8s.io/v1/customresourcedefinitions "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:20:58 INFO unit.kubeflow-dashboard/0.juju-log Rendering manifests
unit-kubeflow-dashboard-0: 12:20:58 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/api/v1/namespaces/kubeflow/configmaps/centraldashboard-config?force=true&fieldManager=lightkube "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:20:58 INFO unit.kubeflow-dashboard/0.juju-log Reconcile completed successfully
unit-kubeflow-dashboard-0: 12:20:58 INFO unit.kubeflow-dashboard/0.juju-log Pebble plan updated with new configuration, replaning
unit-kubeflow-dashboard-0: 12:20:59 INFO juju.worker.uniter.operation ran "config-changed" hook (via hook dispatching script: dispatch)
unit-kubeflow-dashboard-0: 12:22:13 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: GET https://10.152.183.1/api/v1/namespaces/kubeflow/services/kubeflow-dashboard "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:22:13 INFO unit.kubeflow-dashboard/0.juju-log HTTP Request: PATCH https://10.152.183.1/api/v1/namespaces/kubeflow/services/kubeflow-dashboard "HTTP/1.1 200 OK"
unit-kubeflow-dashboard-0: 12:22:13 INFO unit.kubeflow-dashboard/0.juju-log Kubernetes service 'kubeflow-dashboard' patched successfully

Additional Context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions