Skip to content

Commit bb16533

Browse files
authored
Add PHP runtime PHM meter analyzer, e2e, and documentation. (#13928)
Introduce php-runtime.yaml MAL rules for six instance_php_* meters, MAL tests, PHP e2e meter assertions, and backend docs. Pin SW_AGENT_PHP_COMMIT to apache/skywalking-php@de311c9 (PHM #145). Use metrics_report_period=31 in e2e php.ini so meter gRPC streams idle-close and OAP persists PHM data.
1 parent 4226f14 commit bb16533

11 files changed

Lines changed: 247 additions & 3 deletions

File tree

docs/en/changes/changes.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -245,6 +245,10 @@
245245
admin-host only" entry above for the public REST retirement.
246246

247247
#### OAP Server
248+
* Add PHP runtime PHM meter analyzer (`php-runtime.yaml`) for SkyWalking PHP agent process
249+
metrics (CPU, memory, virtual memory, thread count, open file descriptors sampled from
250+
`/proc` on Linux). Registers six `meter_instance_php_*` metrics on the General Service
251+
layer; `php-runtime` is included in the default `meterAnalyzerActiveFiles`.
248252
* Batch the BanyanDB schema fence per runtime-rule apply. A runtime-rule file changes dozens of rules at once, but the post-DDL fence (`SchemaWatcher.awaitRevisionApplied`) ran once per metric/downsampling, so a large file did `K×M` sequential ≤2s fences — on a laggy cluster that overran the apply's REST budget. The main-node apply path now uses `StorageManipulationOpt.withSchemaChangeDeferredFence()`: the installer records each resource's `mod_revision` without fencing and registers a single flush that the apply runs once on the file's max revision, collapsing the whole file to one barrier. The flush is one-shot — a reconciler tick reuses one opt across every rule file, so after a file flushes, the closure and accumulated revision reset and each file fences on its own DDL only. Drops still fence inline on the dropped resource's own delete revision — or, when that delete recorded no tombstone (`mod_revision == 0`), on a key-based deletion barrier (`AwaitSchemaDeleted`) — never on the shared opt's cumulative revision, so a tombstone-less delete in a multi-file tick is still confirmed removed. On the operator REST apply the single create/update fence runs on a configurable, generous budget (default 180s) in the background **before** the rule row is persisted and dispatch resumes — it gates the persist + local commit + peer resume so the durable commit point is only reached once the schema is confirmed cluster-wide, and writes never resume against an un-propagated schema (see the apply-status entry below); the reconciler tick keeps the short inline 2s fence (a background reconcile must not wait minutes per file). Peer / `withoutSchemaChange` applies are unaffected (no fence).
249253
* Add a runtime-rule apply-status query. The cluster main now tracks each structural apply through a phase machine (`SchemaApplyCoordinator`: pending → DDL → fencing → rolling-out → applied, with `degraded` for a committed-but-unconfirmed apply — the cluster schema fence did not confirm within the timeout, in which case the lagging data-node ids are surfaced as `fenceLaggards` and dispatch is resumed anyway, or the local commit-tail threw — and `failed` carrying the specific reason). The schema fence runs on a configurable, generous budget (`receiver-runtime-rule.deferredFenceTimeoutSeconds`, default 180s) and **gates everything durable or visible**: because an un-propagated write is silently dropped at the data node, the order after a successful DDL is suspend → DDL → **fence → persist → commit → resume**. The rule row (the durable commit point) is written only AFTER the fence confirms, so "durable" implies "schema propagated cluster-wide" — a main crash before persist leaves no row (peers/crash-recovery stay safely on the old content; the orphaned measure is inert), and any durable row is guaranteed fence-confirmed, so convergence never resumes dispatch against an unpropagated schema. The fence + persist + resume run in the background so they never block the HTTP response — `POST /addOrUpdate` returns its `applyId` immediately at `fencing` (accepted, not yet durable; dispatch for that rule still paused — a clean gap, not dropped writes), and the operator polls `GET /runtime/rule/status` to watch `fencing → rolling-out → applied` (or `degraded`/`failed`); on a genuine laggard, dispatch resumes after the budget so one stuck node can't park the metric forever. A `GetApplyStatus` admin-internal gRPC served by the main backs the query — by `applyId`, or by `catalog`+`name` (+ optional `contentHash`, the durable identity) once the handle is gone after a page refresh. When the live status is gone (apply-id evicted, main restarted, or the main is unreachable), the query degrades to the durable rule row: a matching `ACTIVE` row reports `applied` derived from the content hash (a durable row is, by the fence-then-persist order, already propagation-confirmed). Non-main nodes route the read to the deterministic main; status is in-memory by design, with the content hash reconstructing truth after a restart.
250254
* Push runtime-rule convergence to peers on commit. After a successful structural apply — and on the `commit_deferred` path, where the DB row is durable but this node's commit-tail threw — the main broadcasts a `NotifyApplied` admin-internal RPC so peers reconcile against the just-persisted DB row immediately, instead of waiting up to one refresh tick (~30s) to notice it. The fan-out runs off the REST response thread (fire-and-forget on a daemon executor) so an unreachable peer's per-call deadline never adds to the operator's apply latency. On the peer side the notify-triggered reconcile is coalesced: a burst of notifies (a multi-rule file, or several applies) collapses to a single queued full reconcile rather than one redundant `dao.getAll()` scan per notify. The notify is best-effort and idempotent (the peer runs its normal per-file-locked reconcile; a lost notify is harmless — the peer still self-converges on its next tick), so it tightens the cluster-convergence window without adding a hard dependency on the main being reachable.
@@ -328,5 +332,6 @@
328332
* Add WeChat / Alipay Mini Program monitoring setup documentation, plus a client-side-monitoring section in the security guide covering public-internet ingress (OTLP + `/v3/segments`) for mobile / browser / mini-program SDKs.
329333
* Improve downsampling documentation
330334
* Fix the docker-compose quickstart: OAP healthcheck no longer calls `curl` (absent from the JRE image) and probes the query port via bash `/dev/tcp`; the Horizon UI service maps the correct container port (8081) and mounts a `horizon.yaml` (binding `0.0.0.0`, OAP URLs, demo `admin`/`admin` login) instead of non-existent `SW_*_ADDRESS` env vars.
335+
* Add PHP runtime metrics (PHM) dashboard documentation (agent setup, OAP `php-runtime` MAL rules, Horizon UI widgets).
331336

332337
All issues and pull requests are [here](https://github.com/apache/skywalking/issues?q=milestone:11.0.0)

docs/en/setup/backend/backend-meter.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@ All following agents and components have built-in meters reporting to the OAP th
5353
5. Java agent for thread-pool metrics
5454
6. Rover(eBPF) agent for metrics used continues profiling
5555
7. Satellite proxy self-observability metrics
56+
8. PHP agent for PHM (PHP Health Metrics) runtime metrics — **Linux only** (`/proc` sampling)
5657

5758
## Configuration file
5859

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
# PHP runtime metrics (PHM)
2+
3+
The SkyWalking PHP agent can report **PHP Health Metrics (PHM)** through the native Meter protocol.
4+
OAP parses them with MAL rules in `php-runtime.yaml` and stores
5+
them as `meter_*` metrics on the **General Service** layer.
6+
7+
Requires a PHP agent build that includes PHM (merged in `apache/skywalking-php` 1.2.0+).
8+
9+
## Platform support
10+
11+
PHM process meters are **Linux only**. In `grpc` / `kafka` reporter mode, the forked reporter worker
12+
samples the **parent PHP process** (`getppid()`) through `/proc` (`/proc/{pid}/status`, `stat`, and
13+
`fd`). They are not collected on macOS or Windows, and PHM does not run when `reporter_type =
14+
standalone`. Instance dashboard widgets stay hidden when no `meter_instance_php_*` data exists.
15+
16+
## Data flow
17+
18+
1. PHM is **On by default on Linux** when the agent is active (`skywalking_agent.enable = On`). Set
19+
`skywalking_agent.metrics_enable = Off` to disable.
20+
2. The forked reporter worker boots `skywalking::metrics::Metricer` in `start_worker` and samples
21+
`/proc` on `metrics_report_period` (default 30 seconds). No HTTP traffic is required.
22+
3. OAP loads `meter-analyzer-config/php-runtime.yaml` when `php-runtime` is listed in
23+
`agent-analyzer.default.meterAnalyzerActiveFiles`.
24+
4. Horizon UI renders widgets on **General Service → Instance → Dashboard** when the corresponding
25+
`meter_instance_php_*` metrics exist.
26+
27+
## Agent setup
28+
29+
```ini
30+
; Default On on Linux when the agent is active.
31+
; skywalking_agent.metrics_enable = Off
32+
33+
skywalking_agent.metrics_report_period = 30
34+
```
35+
36+
Refer to the PHP agent README and INI settings documentation for details.
37+
38+
## OAP setup
39+
40+
Ensure `php-runtime` is active (included by default when using the stock `application.yml`):
41+
42+
```yaml
43+
agent-analyzer:
44+
default:
45+
meterAnalyzerActiveFiles: ...,php-runtime,...
46+
```
47+
48+
## UI location
49+
50+
**Layer:** General Service (`GENERAL`)
51+
52+
**Path:** select a PHP service → **Instance** → **Dashboard**
53+
54+
Widgets appear only when PHM data is present (`visibleWhen` checks each `meter_instance_php_*` expression).
55+
56+
## Runtime metrics
57+
58+
Agent meter names (reported by the PHP agent) are rewritten by OAP MAL `metricPrefix: meter`:
59+
60+
| Unit | Agent meter name | OAP / UI metric name | Description | Data Source |
61+
|-------|------------------------------------------|--------------------------------------------|-----------------------------------------|----------------------|
62+
| % | instance_php_process_cpu_utilization | meter_instance_php_process_cpu_utilization | Process CPU utilization | SkyWalking PHP Agent |
63+
| MB | instance_php_memory_used_mb | meter_instance_php_memory_used_mb | Resident memory (VmRSS from /proc) | SkyWalking PHP Agent |
64+
| MB | instance_php_memory_peak_mb | meter_instance_php_memory_peak_mb | Peak resident memory (VmHWM from /proc) | SkyWalking PHP Agent |
65+
| MB | instance_php_virtual_memory_mb | meter_instance_php_virtual_memory_mb | Virtual memory (VmSize from /proc) | SkyWalking PHP Agent |
66+
| — | instance_php_thread_count | meter_instance_php_thread_count | OS thread count (Threads from /proc) | SkyWalking PHP Agent |
67+
| — | instance_php_open_fd_count | meter_instance_php_open_fd_count | Open file descriptor count | SkyWalking PHP Agent |
68+
69+
## Customizations
70+
71+
You can customize MAL expressions or dashboard panels. Metric definitions and expression rules are in
72+
`/meter-analyzer-config/php-runtime.yaml`. Instance dashboard widget templates ship from the
73+
SkyWalking Horizon UI bundle (`general.json` in apache/skywalking-horizon-ui).

docs/menu.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -260,6 +260,8 @@ catalog:
260260
path: "/en/setup/backend/backend-zabbix"
261261
- name: "Meter Analysis"
262262
path: "/en/setup/backend/backend-meter"
263+
- name: "PHP runtime metrics (PHM)"
264+
path: "/en/setup/backend/dashboards-php-runtime"
263265
- name: "Telegraf Metrics"
264266
path: "/en/setup/backend/telegraf-receiver"
265267
- name: "Apdex Threshold"
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one or more
2+
# contributor license agreements. See the NOTICE file distributed with
3+
# this work for additional information regarding copyright ownership.
4+
# The ASF licenses this file to You under the Apache License, Version 2.0
5+
# (the "License"); you may not use this file except in compliance with
6+
# the License. You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
script: oap-server/server-starter/src/main/resources/meter-analyzer-config/php-runtime.yaml
17+
input:
18+
instance_php_process_cpu_utilization:
19+
- labels:
20+
instance: test-instance
21+
value: 100.0
22+
instance_php_memory_used_mb:
23+
- labels:
24+
instance: test-instance
25+
value: 256.0
26+
instance_php_memory_peak_mb:
27+
- labels:
28+
instance: test-instance
29+
value: 512.0
30+
instance_php_virtual_memory_mb:
31+
- labels:
32+
instance: test-instance
33+
value: 1024.0
34+
instance_php_thread_count:
35+
- labels:
36+
instance: test-instance
37+
value: 4.0
38+
instance_php_open_fd_count:
39+
- labels:
40+
instance: test-instance
41+
value: 32.0
42+
expected:
43+
meter_instance_php_process_cpu_utilization:
44+
entities:
45+
- scope: SERVICE_INSTANCE
46+
instance: test-instance
47+
layer: GENERAL
48+
samples:
49+
- labels:
50+
instance: test-instance
51+
value: 100.0
52+
meter_instance_php_memory_used_mb:
53+
entities:
54+
- scope: SERVICE_INSTANCE
55+
instance: test-instance
56+
layer: GENERAL
57+
samples:
58+
- labels:
59+
instance: test-instance
60+
value: 256.0
61+
meter_instance_php_memory_peak_mb:
62+
entities:
63+
- scope: SERVICE_INSTANCE
64+
instance: test-instance
65+
layer: GENERAL
66+
samples:
67+
- labels:
68+
instance: test-instance
69+
value: 512.0
70+
meter_instance_php_virtual_memory_mb:
71+
entities:
72+
- scope: SERVICE_INSTANCE
73+
instance: test-instance
74+
layer: GENERAL
75+
samples:
76+
- labels:
77+
instance: test-instance
78+
value: 1024.0
79+
meter_instance_php_thread_count:
80+
entities:
81+
- scope: SERVICE_INSTANCE
82+
instance: test-instance
83+
layer: GENERAL
84+
samples:
85+
- labels:
86+
instance: test-instance
87+
value: 4.0
88+
meter_instance_php_open_fd_count:
89+
entities:
90+
- scope: SERVICE_INSTANCE
91+
instance: test-instance
92+
layer: GENERAL
93+
samples:
94+
- labels:
95+
instance: test-instance
96+
value: 32.0

oap-server/server-starter/src/main/resources/application.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -229,7 +229,7 @@ agent-analyzer:
229229
# Nginx and Envoy agents can't get the real remote address.
230230
# Exit spans with the component in the list would not generate the client-side instance relation metrics.
231231
noUpstreamRealAddressAgents: ${SW_NO_UPSTREAM_REAL_ADDRESS:6000,9000}
232-
meterAnalyzerActiveFiles: ${SW_METER_ANALYZER_ACTIVE_FILES:datasource,threadpool,satellite,go-runtime,python-runtime,continuous-profiling,java-agent,go-agent,ruby-runtime} # Which files could be meter analyzed, files split by ","
232+
meterAnalyzerActiveFiles: ${SW_METER_ANALYZER_ACTIVE_FILES:datasource,threadpool,satellite,go-runtime,python-runtime,continuous-profiling,java-agent,go-agent,ruby-runtime,php-runtime} # Which files could be meter analyzed, files split by ","
233233
slowCacheReadThreshold: ${SW_SLOW_CACHE_SLOW_READ_THRESHOLD:default:20,redis:10} # The slow cache read operation thresholds. Unit ms.
234234
slowCacheWriteThreshold: ${SW_SLOW_CACHE_SLOW_WRITE_THRESHOLD:default:20,redis:10} # The slow cache write operation thresholds. Unit ms.
235235

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one or more
2+
# contributor license agreements. See the NOTICE file distributed with
3+
# this work for additional information regarding copyright ownership.
4+
# The ASF licenses this file to You under the Apache License, Version 2.0
5+
# (the "License"); you may not use this file except in compliance with
6+
# the License. You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
expSuffix: instance(['service'], ['instance'], Layer.GENERAL)
17+
metricPrefix: meter
18+
metricsRules:
19+
# CPU
20+
- name: instance_php_process_cpu_utilization
21+
exp: instance_php_process_cpu_utilization
22+
# Memory
23+
- name: instance_php_memory_used_mb
24+
exp: instance_php_memory_used_mb
25+
- name: instance_php_memory_peak_mb
26+
exp: instance_php_memory_peak_mb
27+
- name: instance_php_virtual_memory_mb
28+
exp: instance_php_virtual_memory_mb
29+
# Process resources
30+
- name: instance_php_thread_count
31+
exp: instance_php_thread_count
32+
- name: instance_php_open_fd_count
33+
exp: instance_php_open_fd_count

test/e2e-v2/cases/php/e2e.yaml

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,3 +107,34 @@ verify:
107107
swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql instance list --service-name=php | yq e '.[0].name' - ) \
108108
--service-name=php --dest-instance-name=provider1 --dest-service-name=e2e-service-provider
109109
expected: expected/metrics-has-value.yml
110+
# PHP Health Metrics (PHM) — instance meters via native MeterReportService
111+
- query: |
112+
swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_instance_php_process_cpu_utilization --instance-name=$( \
113+
swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql instance list --service-name=php | yq e '.[0].name' - ) \
114+
--service-name=php
115+
expected: expected/metrics-has-value.yml
116+
- query: |
117+
swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_instance_php_memory_used_mb --instance-name=$( \
118+
swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql instance list --service-name=php | yq e '.[0].name' - ) \
119+
--service-name=php
120+
expected: expected/metrics-has-value.yml
121+
- query: |
122+
swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_instance_php_memory_peak_mb --instance-name=$( \
123+
swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql instance list --service-name=php | yq e '.[0].name' - ) \
124+
--service-name=php
125+
expected: expected/metrics-has-value.yml
126+
- query: |
127+
swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_instance_php_virtual_memory_mb --instance-name=$( \
128+
swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql instance list --service-name=php | yq e '.[0].name' - ) \
129+
--service-name=php
130+
expected: expected/metrics-has-value.yml
131+
- query: |
132+
swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_instance_php_thread_count --instance-name=$( \
133+
swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql instance list --service-name=php | yq e '.[0].name' - ) \
134+
--service-name=php
135+
expected: expected/metrics-has-value.yml
136+
- query: |
137+
swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_instance_php_open_fd_count --instance-name=$( \
138+
swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql instance list --service-name=php | yq e '.[0].name' - ) \
139+
--service-name=php
140+
expected: expected/metrics-has-value.yml

test/e2e-v2/cases/php/php.ini

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,3 +25,6 @@ skywalking_agent.server_addr = oap:11800
2525
skywalking_agent.skywalking_version = 9
2626
skywalking_agent.runtime_dir = /tmp
2727

28+
skywalking_agent.metrics_enable = 1
29+
; CI: 31s (> skywalking-rs 30s stream idle) so OAP meter process() runs
30+
skywalking_agent.metrics_report_period = 31

test/e2e-v2/cases/storage/expected/config-dump.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@
3232
"admin-server.default.port": "17128",
3333
"admin-server.provider": "default",
3434
"agent-analyzer.default.forceSampleErrorSegment": "true",
35-
"agent-analyzer.default.meterAnalyzerActiveFiles": "datasource,threadpool,satellite,go-runtime,python-runtime,continuous-profiling,java-agent,go-agent,ruby-runtime",
35+
"agent-analyzer.default.meterAnalyzerActiveFiles": "datasource,threadpool,satellite,go-runtime,python-runtime,continuous-profiling,java-agent,go-agent,ruby-runtime,php-runtime",
3636
"agent-analyzer.default.noUpstreamRealAddressAgents": "6000,9000",
3737
"agent-analyzer.default.segmentStatusAnalysisStrategy": "FROM_SPAN_STATUS",
3838
"agent-analyzer.default.slowCacheReadThreshold": "default:20,redis:10",

0 commit comments

Comments
 (0)