Skip to content

Commit cad265b

Browse files
author
Thomas Rzen
committed
Add OPNsense by REST API
A second OPNsense template next to OPNsense by HTTP-JSON, derived from it and considerably wider. It carries its own name and its own uuids, so both can be linked to the same host and importing one does not touch the other. Compared with the template it grew out of it adds, measured against a live OPNsense 26.7 with a restricted monitoring key: * processor utilisation split into user, system and interrupt, core count and load per core derived from it * the packet filter in depth: state and source tracking tables with their limits, pf counters, table entries against the configured ceiling, and the loaded ruleset with size, fingerprint, evaluation rate and unmatched rules * kernel network memory and mbuf clusters, netisr queue drops per protocol, IP and TCP protocol error rates * clock synchronisation with offset, stratum and reachable peers * service run state, swap per device and temperature per sensor, each discovered * inbound errors and link state per interface, and the blocked share of the firewall log * a dashboard of eight pages in place of three Three defects of the original are fixed here rather than carried over. Two urls used camelCase where the privilege pattern is exact and ACL.php matches it case sensitively, so a monitoring key got HTTP 403 on the firewall state and all memory items. IPsec phase 2 asked with a GET although searchPhase2Action reads its connection from getPost('id'), so it could only ever return an empty row set; it is an item prototype inside the phase 1 rule now, posting one request per connection. A gateway without a monitor address had 9999 written into round trip time, loss and deviation, which landed in history and made the gateway graphs unreadable; the reading is discarded instead. The export declares format 7.0 rather than 7.4, so it imports into Zabbix 7.0 and everything above. Every widget type it uses exists in 7.0. The privileges the monitoring user needs are listed in the README with their internal ids, measured with a restricted key. Eight of the ten are read only, and the README says what the other two permit and how to do without them.
1 parent 640b659 commit cad265b

2 files changed

Lines changed: 7024 additions & 0 deletions

File tree

‎Network_Devices/OPNsense/template_opnsense_by_rest_api/7.0/README.md‎

Lines changed: 246 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,246 @@
1+
# OPNsense by REST API
2+
3+
Zabbix template that monitors an OPNsense firewall over its own REST API. No SNMP, no Zabbix
4+
agent on the firewall, one API key with read privileges.
5+
6+
Import into **Zabbix 7.0 or later**. The export declares format 7.0, which every later
7+
release accepts.
8+
9+
Everything in here was measured against a live OPNsense 26.7 with a restricted monitoring
10+
key, not read off the documentation. Where the API turned out to behave differently than
11+
expected, that is written down in [Notes](#notes) rather than smoothed over.
12+
13+
## Contents
14+
15+
- [What it monitors](#what-it-monitors)
16+
- [Setup](#setup)
17+
- [Privileges](#privileges)
18+
- [Macros](#macros)
19+
- [Discovery](#discovery)
20+
- [Triggers](#triggers)
21+
- [Dashboard](#dashboard)
22+
- [Notes](#notes)
23+
- [License](#license)
24+
25+
## What it monitors
26+
27+
107 items, 14 discovery rules with 70 item prototypes, 52 triggers, 36 macros and a dashboard
28+
with 8 pages.
29+
30+
| Area | Source endpoint |
31+
|------|-----------------|
32+
| Processor utilisation, split into user, system and interrupt | `diagnostics/activity/get_activity` |
33+
| Core count, load average over 1, 5 and 15 minutes, load per core | `diagnostics/cpu_usage/get_c_p_u_type`, `diagnostics/system/system_time` |
34+
| Memory, swap per device, filesystems | `diagnostics/system/system_resources`, `system_swap`, `system_disk` |
35+
| Temperature per sensor | `diagnostics/system/system_temperature` |
36+
| Kernel network memory, mbuf clusters, denied requests | `diagnostics/interface/get_memory_statistics` |
37+
| netisr queue drops, total and per protocol | `diagnostics/interface/get_netisr_statistics` |
38+
| IP and TCP protocol error rates | `diagnostics/interface/get_protocol_statistics` |
39+
| Interface traffic, errors, link state, per interface blocks | `diagnostics/traffic/_interface`, `diagnostics/firewall/pf_statistics/interfaces` |
40+
| pf state table and source tracking, with their limits | `diagnostics/firewall/pf_states`, `pf_statistics/info`, `pf_statistics/memory` |
41+
| pf counters, SYN floods, malformed packets | `diagnostics/firewall/pf_statistics/info` |
42+
| Ruleset size, fingerprint, evaluation rate, unmatched rules | `diagnostics/firewall/pf_statistics/rules` |
43+
| pf table entries against the configured ceiling | `firewall/alias/get_table_size` |
44+
| Blocked share of the firewall log | `diagnostics/firewall/stats?group_by=action` |
45+
| Gateways, address, status, round trip time, loss, deviation | `routes/gateway/status` |
46+
| CARP status per address, demotion factor, maintenance mode | `diagnostics/interface/get_vip_status` |
47+
| IPsec phase 1 and phase 2, per connection | `ipsec/sessions/searchPhase1`, `search_phase2` |
48+
| WireGuard instances and peers | `wireguard/service/show` |
49+
| OpenVPN instances and clients | `openvpn/service/search_sessions` |
50+
| Services, discovered and checked | `core/service/search` |
51+
| Clock synchronisation, offset, stratum, reachable peers | `ntpd/service/status` |
52+
| Firmware state, pending updates, business license, version | `core/firmware/status`, `core/firmware/info` |
53+
| Configuration change timestamp, uptime | `diagnostics/system/system_time` |
54+
| UPS, battery, load, runtime, voltage | `nut/diagnostics/upsstatus` |
55+
56+
## Setup
57+
58+
### 1. Create the API key
59+
60+
In the OPNsense GUI under **System, Access, Users**, create a user without a login shell, for
61+
example `zabbix`, give it the privileges from the next section, then add an API key. The
62+
download contains the key and the secret.
63+
64+
### 2. Import the template
65+
66+
**Data collection, Templates, Import**, then select `template_opnsense_by_rest_api.yaml`.
67+
68+
### 3. Create the host
69+
70+
Create a host with an agent interface carrying the firewall address, link the template, and
71+
set the three connection macros:
72+
73+
| Macro | Value |
74+
|-------|-------|
75+
| `{$OPNS.KEY}` | the API key |
76+
| `{$OPNS.SECRET}` | the API secret |
77+
| `{$OPNS.PORT}` | the HTTPS port of the web interface, default 443 |
78+
79+
The items address the firewall through `{HOST.IP}`, so the interface address is what counts,
80+
not the DNS name.
81+
82+
## Privileges
83+
84+
Measured with a restricted key, not taken from the documentation. Ten privileges are needed,
85+
eight of them read only.
86+
87+
| Privilege in the GUI | Internal ID | Needed for | Read only |
88+
|----------------------|-------------|------------|-----------|
89+
| Lobby: Dashboard | `page-system-login-logout` | system resources, time, disks, swap, temperature, core count, pf states | yes |
90+
| Diagnostics: Netstat | `page-diagnostics-netstat` | interface statistics, mbuf pool, netisr queues, protocol errors | yes |
91+
| Diagnostics: Firewall statistics | `page-diagnostics-pf-info` | pf counters, source tracking, ruleset | yes |
92+
| Diagnostics: System Activity | `page-diagnostics-system-activity` | processor utilisation | yes |
93+
| Status: NTP | `page-status-ntp` | clock synchronisation | yes |
94+
| Diagnostics: Logs: Firewall: Summary View | `page-diagnostics-logs-firewall-summary` | blocked share of the firewall log | yes |
95+
| Firewall: Aliases | `page-firewall-aliases` | pf table entry usage, also exposes every alias content | yes |
96+
| Reporting: Traffic | `page-status-trafficgraph` | interface discovery, traffic and counters | yes |
97+
| Status: Services | `page-status-services` | service run state | **no**, also permits starting and stopping services |
98+
| System: Firmware | `page-system-firmware-manualupdate` | firmware state, pending updates, business license, version | **no**, its pattern `api/core/firmware/*` also covers reboot, poweroff, install and remove |
99+
100+
Two of them grant more than reading. Without **Status: Services** the service discovery stays
101+
empty and nothing else breaks. Without **System: Firmware** the five firmware items, the
102+
license item and the version item stay unsupported; the version is also available from
103+
`diagnostics/system/system_information`, which Lobby: Dashboard already covers.
104+
105+
`page-nut` is needed only if a UPS is monitored. It comes with the NUT plugin and covers
106+
`api/nut/*`. Without it the UPS master returns HTTP 403 even with the item enabled.
107+
108+
## Macros
109+
110+
### Connection
111+
112+
| Macro | Default | Purpose |
113+
|-------|---------|---------|
114+
| `{$OPNS.KEY}` | empty | API key |
115+
| `{$OPNS.SECRET}` | empty | API secret |
116+
| `{$OPNS.PORT}` | `443` | HTTPS port of the web interface |
117+
118+
### Thresholds
119+
120+
| Macro | Default | Purpose |
121+
|-------|---------|---------|
122+
| `{$OPNS.CPU.UTIL.WARN}` | `85` | Processor utilisation counting as high, averaged over ten minutes |
123+
| `{$OPNS.CPU.LOAD.MAX}` | `2` | Load average counting as high |
124+
| `{$OPNS.LOAD.AVG5.WARN}` | `4` | Absolute five minute load average counting as high |
125+
| `{$OPNS.LOAD.PERCORE.WARN}` | `1` | Five minute load per core, 1 means the cores are exactly saturated |
126+
| `{$OPNS.MEMORY.UTIL.MAX}` | `90` | Memory utilisation counting as high |
127+
| `{$OPNS.MBUF.UTIL.WARN}` | `80` | Share of the mbuf cluster limit counting as filling up |
128+
| `{$OPNS.SWAP.UTIL.WARN}` | `5` | Swap in use counting as a problem. A firewall should not swap at all, so this is deliberately low. Per device through macro context |
129+
| `{$OPNS.TEMP.CRIT}` | `80` | Sensor temperature in degrees Celsius. Per sensor through context, for example `{$OPNS.TEMP.CRIT:"dev.cpu.0.temperature"}` |
130+
| `{$OPNS.FS.PUSED.MAX.WARN}` | `90` | Filesystem utilisation, warning |
131+
| `{$OPNS.FS.PUSED.MAX.CRIT}` | `95` | Filesystem utilisation, critical |
132+
| `{$OPNS.STATE.TABLE.UTIL.MAX}` | `90` | pf state table fill level counting as high |
133+
| `{$OPNS.PF.SRCNODES.UTIL.CRIT}` | `90` | Source tracking table fill level counting as critical |
134+
| `{$OPNS.PF.TABLES.UTIL.WARN}` | `80` | Share of the pf table entry budget counting as filling up. Relevant with block lists or GeoIP aliases |
135+
| `{$OPNS.GW.MIN.PACKET.LOSS}` | `10` | Gateway packet loss counting as a problem |
136+
| `{$OPNS.GW.HIGH.PACKET.LOSS}` | `50` | Gateway packet loss counting as severe |
137+
| `{$OPNS.NTP.OFFSET.WARN}` | `100` | Clock offset in milliseconds. Generous for a LAN time source and still far below what breaks Kerberos or IPsec |
138+
| `{$OPNS.LICENSE.EXPIRY.WARN}` | `30` | Days before the business license expires |
139+
| `{$OPNS.NUT.BAT.LOW}` | `30` | UPS battery charge counting as low |
140+
| `{$OPNS.NUT.BAT.RUNTIME}` | `600` | Remaining UPS runtime in seconds |
141+
| `{$OPNS.NUT.HIGH.LOAD}` | `80` | UPS load counting as high |
142+
143+
### Discovery filters and switches
144+
145+
| Macro | Default | Purpose |
146+
|-------|---------|---------|
147+
| `{$OPNS.IF.CONTROL}` | `1` | Set to 0, per interface through context, to silence the interface down trigger |
148+
| `{$OPNS.IF.NAME.NOT_MATCHES}` | `^(pflog\|pfsync\|enc\|lo)\d*$` | Interfaces excluded from discovery |
149+
| `{$OPNS.SERVICE.ID.NOT_MATCHES}` | `^$` | Services excluded from discovery, excludes nothing by default |
150+
| `{$OPNS.OPENVPN.ID.NOT_MATCHES}` | `^$` | OpenVPN instances excluded from discovery |
151+
| `{$OPNS.OPENVPN.CONTROL}` | `1` | Set to 0, per instance through context, to silence the OpenVPN down trigger |
152+
| `{$OPNS.WG.INSTANCE.MATCHES}` / `.NOT_MATCHES` | `.+` / `^$` | WireGuard instance discovery filter |
153+
| `{$OPNS.WG.PEER.MATCHES}` / `.NOT_MATCHES` | `.+` / `^$` | WireGuard peer discovery filter |
154+
| `{$OPNS.FS.FSNAME.MATCHES}` / `.NOT_MATCHES` | `.+` / `^(/dev\|/sys\|/run\|/proc\|.+/shm$)` | Filesystem discovery filter by mount point |
155+
| `{$OPNS.FS.FSTYPE.MATCHES}` / `.NOT_MATCHES` | filesystem list / `^\s$` | Filesystem discovery filter by type |
156+
157+
## Discovery
158+
159+
| Rule | Discovers |
160+
|------|-----------|
161+
| Interface Stats Discovery | traffic, errors and per interface pf blocks |
162+
| Network interfaces | link state and inbound errors |
163+
| Gateway Discovery | address, status, round trip time, loss, deviation |
164+
| Interface CARP Discovery | CARP status per virtual address |
165+
| Disk Discovery | filesystems |
166+
| Swap devices | swap per device |
167+
| Temperature sensors | one item per sensor |
168+
| netisr queues | queue drops per protocol |
169+
| Services | run state per service |
170+
| FW Action Discovery | firewall log actions |
171+
| IPsec Phase1 Discovery | tunnels, with phase 2 per connection |
172+
| WireGuard Instance Discovery | instances |
173+
| WireGuard Peer Discovery | peers, traffic and handshake age |
174+
| OpenVPN instances and clients | instances and connected clients |
175+
176+
## Triggers
177+
178+
52 in total: 2 Disaster, 13 High, 11 Average, 18 Warning, 8 Info.
179+
180+
Disaster is reserved for a gateway that is down and for a low UPS battery. High covers the
181+
cases where the firewall is losing packets or a tunnel is gone: denied kernel network memory,
182+
pf dropping packets for lack of memory, the state limit being hit, a filling source tracking
183+
table, a hot sensor, a CARP status change, a dead WireGuard instance or peer, an unconnected
184+
IPsec tunnel, an unreachable API and the UPS running on battery.
185+
186+
Warning is the working range: utilisation and load, clock offset, swap in use, malformed
187+
packets, pf table entries filling up, protocol level errors. Info reports facts rather than
188+
faults: configuration changes, ruleset and rule count changes, a version change, available
189+
firmware updates, a reboot, and a gateway whose monitoring is switched off.
190+
191+
## Dashboard
192+
193+
Eight pages, 84 widgets. The grid is the full 72 columns wide.
194+
195+
| Page | Shows |
196+
|------|-------|
197+
| Overview | memory, state table, processor and load per core as gauges, six tiles, firewall actions over time, CARP status, filesystems as a honeycomb |
198+
| Packet filter | state table, source tracking and pf table utilisation, twelve tiles, state churn and searches, rule matches against evaluations, drops and limit hits, malformed packets |
199+
| Interfaces | link state per interface, traffic, blocked and passed bytes side by side, errors, queue drops and collisions |
200+
| Gateways | status honeycomb, round trip time and packet loss side by side, deviation below |
201+
| VPN | WireGuard peers and instances, peer traffic, IPsec phase 1 tunnels and phase 2 traffic |
202+
| System | processor, memory and load per core, uptime, version, cores, configuration change timestamp, both load averages, utilisation split, temperature and swap honeycombs |
203+
| Kernel and protocols | mbuf utilisation, clusters in use, denied requests, netisr drops per protocol, IP and TCP error rates |
204+
| Services, clock and power | every service as a honeycomb, clock synchronisation, offset, stratum, reachable peers, CARP demotion and maintenance, UPS battery, status, load and runtime |
205+
206+
## Notes
207+
208+
**snake_case in API paths is not always optional.** OPNsense routes `systemResources` and
209+
`system_resources` to the same controller action, so an administrator key never notices a
210+
difference. Some privilege patterns are exact, and `ACL.php` matches them with `preg_match`
211+
without the `i` modifier, so a monitoring key gets HTTP 403 on the camelCase spelling. That
212+
is the case for `diagnostics/system/system_resources` and `diagnostics/firewall/pf_states`.
213+
Other areas carry a wildcard pattern and take either spelling, which is why
214+
`ipsec/sessions/searchPhase1` works and is left as it is. Test privileges with the restricted
215+
key, never with an admin key.
216+
217+
**Gateway monitoring switched off.** A gateway without a monitor address reports a tilde for
218+
round trip time, loss and deviation, and a status of `none` that the API translates to
219+
`Online`. The three items discard the reading in that case and stay empty instead of
220+
substituting a number. While monitoring is off, an outage of that gateway raises nothing,
221+
because the down trigger works on packet loss. That is what the Info trigger is for.
222+
223+
**IPsec phase 2** needs one request per connection. `searchPhase2Action` reads its connection
224+
from `getPost('id')` and returns an empty set for a plain GET, so phase 2 is an HTTP agent
225+
prototype inside the phase 1 rule that posts `id={#IPSECNAME}`. Where the API leaves
226+
`phase1desc` empty, the tunnel name stands in, because the discovery uses that field as the
227+
entity identity and identical keys would collapse into one.
228+
229+
**Physical interface counters** come from `diagnostics/traffic/_interface`, not from
230+
`netstat`, whose counters reset on every read.
231+
232+
**JavaScript is kept to a minimum.** Parsing happens once on the master item, dependent items
233+
read the result with plain JSONPath. A step on a prototype would run per value and per
234+
discovered instance.
235+
236+
**The pf ruleset item** polls every ten minutes on purpose. The response carries the full text
237+
of every rule, and rule changes are not a per minute concern.
238+
239+
## License
240+
241+
MIT, like this repository. The template is derived from `OPNsense by HTTP-JSON` in this same
242+
repository (MIT, Copyright 2021 Zabbix), with the greater part of the current content added on
243+
top. It carries its own name and its own uuids, so both can be linked to the same host and an
244+
import of one does not touch the other.
245+
246+
Maintained at https://github.com/Garfieldttt/opnsense-zabbix-template.

0 commit comments

Comments
 (0)