Support the JSON firmware family of issue #6

@jauling reverse engineered a Goodtop ZX310S-8T2XS and posted the protocol
and a payload sample. That is enough to implement it:

  GET /authorize?loginusr=<md5(user)>&loginpwd=<md5(pass)>
  GET /port_statistics.json

Selected with a new firmware key, defaulting to html so nothing changes for
the existing devices. switchclient.JSONClient satisfies the same
collector.SwitchClient interface as the HTML client, so the collector does
not know which family is in use; the package split from the refactor is
what makes this a new file rather than a rewrite.

Notable details

The credentials are hashed separately here, loginusr and loginpwd, unlike
the HTML family which sends one md5(username+password) token. A test pins
that the combined token is never sent to this firmware.

Ports arrive as top-level Port_N keys rather than an array, so they are
collected and sorted numerically. Go map iteration is random and Port_10
sorts before Port_2 lexically, either of which would attach counters to the
wrong label.

Link_Status carries speed and duplex together ("2500MbpsFull"), so the
family publishes port_link_speed_mbps. A 2.5G port that negotiated 1G is
one of the more useful things to alert on with these switches. The metric
is omitted for the HTML family, which does not report it, and for any port
without a link.

No PoE endpoint has been reported, so poe with firmware: json is refused at
startup rather than failing every scrape.

Testing

A fixture built from the payload in the issue, a maxlinear fakeswitch
profile with its own self-tests, and an acceptance case that runs the real
exporter against it. Verified end to end: port_link_speed_mbps reports
1000, 2500, 10000 and 100 for the emulated ports and is absent for the
unlinked ones.

Implemented from a report, not from hardware. The device table marks it
untested and asks for confirmation.

Also corrects the Sodola contributor back to @dennyreiter, which an earlier
commit in this branch had changed by mistake.

Author: Paolo Velati

AGENTS.md

@@ -30,7 +30,7 @@ mis-typechecks the sources and reports nonsense. The pinned version is in
 |------|----------------|
 | `main.go` | Flags, logging, HTTP server, process lifecycle |
 | `internal/config` | Schema, validation, secrets, redaction |
-| `internal/switchclient` | Device transport and HTML interpretation |
+| `internal/switchclient` | Device transport and response interpretation, both firmware families |
 | `internal/collector` | `prometheus.Collector`, nothing else |
 | `internal/fakeswitch` | Device emulator shared by tests and `cmd/fakeswitch` |
 | `cmd/fakeswitch` | Standalone emulator for manual testing and demos |
@@ -66,6 +66,17 @@ it is offered. Renaming the counters is the real fix and needs a major version.
 `exporter_build_info` must appear even when the device is unreachable. A series
 that disappears exactly when things break cannot be alerted on.
 
+**Two firmware families share one package.** `switchclient.Client` scrapes HTML
+and `switchclient.JSONClient` reads `/port_statistics.json`. Both satisfy
+`collector.SwitchClient`, so the collector does not know which is in use. Shared
+helpers (`parseUint`, `parseEnabled`, `normalizePortName`, `md5Hex`) are reused
+deliberately; keep new device support inside this seam rather than teaching the
+collector about firmwares.
+
+Their credential schemes differ and must not be unified: the HTML family sends
+one `md5(username+password)` token, the JSON family hashes the two separately as
+`loginusr` and `loginpwd`.
+
 **Device quirks are load-bearing.** Each of these is required by a specific
 firmware and must not be tidied away:
 

README.md

@@ -21,6 +21,7 @@ Many budget-friendly network switches do not support standard SNMP monitoring. T
 | Horaco | HC-SWTGW124AS | Verified | @arthurbarton |
 | KeepLink | KP-9000-9XHPML-X | Verified | @jfallot and @adamchabin |
 | Sodola | SL-SWTG124AS | Verified | @dennyreiter |
+| Goodtop | ZX310S-8T2XS | Untested, `firmware: json` | @jauling |
 
 ## Installation
 
@@ -105,6 +106,7 @@ password: "password"             # Web interface password
 poll_rate_seconds: 10            # Minimum delay between two polls of the switch
 timeout_seconds: 5               # Per-request timeout
 poe: false                       # Scrape the PoE pages (1/0 also accepted)
+firmware: html                   # html (default) or json, see below
 ```
 
 Unknown keys are rejected, so a misspelled option fails at startup instead of being ignored.
@@ -137,6 +139,26 @@ web:
 
 TLS is served with a minimum version of TLS 1.2. `/healthz` is intentionally left unauthenticated so container health checks keep working. When the endpoint is bound to a non-loopback address without authentication, the exporter logs a warning at startup.
 
+### Firmware families
+
+Two device families are supported. `firmware: html` is the default and covers
+every device in the table above: an RTL8373 style web interface whose statistics
+are scraped from `/port.cgi`.
+
+`firmware: json` covers MaxLinear based devices that authenticate through
+`/authorize` and serve `/port_statistics.json`, reported for the Goodtop
+ZX310S-8T2XS in [issue #6](https://github.com/pvelati/cheap-switch-exporter/issues/6).
+Two differences worth knowing:
+
+- It reports the negotiated link speed, so `port_link_speed_mbps` is published.
+  The HTML family does not expose it and omits the metric.
+- No PoE endpoint has been reported for it, so `poe: true` together with
+  `firmware: json` is refused at startup rather than failing every scrape.
+
+This family is implemented from the protocol and payload in the issue and
+verified against an emulated device. Nobody has confirmed it against real
+hardware yet; reports welcome.
+
 ### Polling behaviour
 
 `poll_rate_seconds` is the minimum delay between two polls of the switch. Scrapes arriving inside that window are answered from the previous result, including failures. That keeps a redundant pair of Prometheus servers, or a short scrape interval, from overwhelming a device whose web interface serves one session at a time. Set it to `0` to poll on every scrape.
@@ -153,6 +175,7 @@ The exporter now authenticates lazily: it requests the statistics page, and only
 
 - `port_state`: Port enabled/disabled status (1=Enable, 0=Disable)
 - `port_link_status`: Port link up/down status (1=Link Up, 0=Link Down)
+- `port_link_speed_mbps`: Negotiated speed in Mbps, only with `firmware: json`
 - `port_tx_good_pkt`: Transmitted good packets
 - `port_tx_bad_pkt`: Transmitted bad packets
 - `port_rx_good_pkt`: Received good packets
@@ -244,6 +267,7 @@ go run ./cmd/fakeswitch -list-profiles
 | `unauthorized` | HTTP 401 |
 | `slow` | Never answers in time |
 | `flaky` | Drops the session every other request |
+| `maxlinear` | The JSON firmware of issue #6, with negotiated link speeds |
 
 ### Test layers
 

acceptance_test.go

@@ -73,6 +73,7 @@ func TestAcceptanceProfiles(t *testing.T) {
 	tests := []struct {
 		profile  fakeswitch.Profile
 		poe      bool
+		firmware string
 		timeout  int
 		wantUp   float64
 		contains []string
@@ -148,6 +149,19 @@ func TestAcceptanceProfiles(t *testing.T) {
 			contains: []string{`exporter_up 0`},
 			absent:   []string{`port_state{`},
 		},
+		{
+			// Issue #6: JSON firmware, authorised through /authorize.
+			profile: fakeswitch.ProfileMaxLinear, firmware: "json", timeout: 2, wantUp: 1,
+			contains: []string{
+				`port_state{port="1"} 1`,
+				`port_link_status{port="1"} 1`,
+				`port_tx_good_pkt{port="1"}`,
+				// Only this family reports a negotiated speed.
+				`port_link_speed_mbps{port="1"}`,
+				`exporter_up 1`,
+			},
+			absent: []string{`poe_port_watts`},
+		},
 	}
 
 	for _, tc := range tests {
@@ -159,10 +173,14 @@ func TestAcceptanceProfiles(t *testing.T) {
 				Delay:   3 * time.Second,
 			})
 			listenAddr := freeAddr(t)
+			firmware := tc.firmware
+			if firmware == "" {
+				firmware = "html"
+			}
 			configPath := writeConfig(t, fmt.Sprintf(
 				"address: %q\nusername: admin\npassword: admin\n"+
-					"poll_rate_seconds: 0\ntimeout_seconds: %d\npoe: %t\n",
-				switchAddr, tc.timeout, tc.poe))
+					"poll_rate_seconds: 0\ntimeout_seconds: %d\npoe: %t\nfirmware: %s\n",
+				switchAddr, tc.timeout, tc.poe, firmware))
 
 			startExporter(t, configPath, listenAddr)
 			body := scrape(t, listenAddr)
@@ -409,3 +427,53 @@ func TestAcceptanceConcurrentScrapes(t *testing.T) {
 		}
 	}
 }
+
+// Only the JSON firmware reports a negotiated link speed, so the metric must be
+// present for it and absent for the HTML family rather than reported as zero.
+func TestAcceptanceLinkSpeedOnlyWhereReported(t *testing.T) {
+	for _, tc := range []struct {
+		profile  fakeswitch.Profile
+		firmware string
+		wantHave bool
+	}{
+		{fakeswitch.ProfileMaxLinear, "json", true},
+		{fakeswitch.ProfileStandard, "html", false},
+	} {
+		t.Run(string(tc.profile), func(t *testing.T) {
+			clearEnv(t)
+			switchAddr, _ := startFake(t, fakeswitch.Options{Profile: tc.profile, Seed: 13})
+			listenAddr := freeAddr(t)
+			configPath := writeConfig(t, fmt.Sprintf(
+				"address: %q\nusername: admin\npassword: admin\n"+
+					"poll_rate_seconds: 0\ntimeout_seconds: 2\nfirmware: %s\n",
+				switchAddr, tc.firmware))
+			startExporter(t, configPath, listenAddr)
+
+			body := scrape(t, listenAddr)
+			_, have := sample(t, body, "port_link_speed_mbps")
+			if have != tc.wantHave {
+				t.Errorf("port_link_speed_mbps present = %t, want %t", have, tc.wantHave)
+			}
+			// Port 1 is the first linked port in the emulator and negotiates 1G.
+			if tc.wantHave && !strings.Contains(body, `port_link_speed_mbps{port="1"} 1000`) {
+				t.Errorf("want port 1 to report 1000 Mbps, got:\n%s",
+					grepLines(body, "port_link_speed_mbps"))
+			}
+			// A port with no link must report no speed at all, not zero.
+			if tc.wantHave && strings.Contains(body, `port_link_speed_mbps{port="2"}`) {
+				t.Error("port 2 has no link and must not report a speed")
+			}
+		})
+	}
+}
+
+// grepLines returns the exposition lines containing needle, for error messages.
+func grepLines(body, needle string) string {
+	var out []string
+	for _, line := range strings.Split(body, "\n") {
+		if strings.Contains(line, needle) && !strings.HasPrefix(line, "#") {
+			out = append(out, "  "+line)
+		}
+	}
+	return strings.Join(out, "\n")
+}

config.yaml.example

@@ -16,6 +16,12 @@ timeout_seconds: 5
 # Scrape the PoE pages. Accepts true/false; 1/0 also still works.
 poe: false
 
+# How the device exposes its statistics.
+#   html  the RTL8373 style web interface, scraped from /port.cgi (default)
+#   json  MaxLinear based devices serving /port_statistics.json
+# The json family also reports port_link_speed_mbps and does not support poe.
+firmware: html
+
 # Optional hardening of the exporter's own endpoint. Everything below can be
 # left out entirely.
 #

internal/collector/collector.go

@@ -69,6 +69,7 @@ type Collector struct {
 
 	portState      *prometheus.Desc
 	portLinkStatus *prometheus.Desc
+	portLinkSpeed  *prometheus.Desc
 	portTxGoodPkt  *prometheus.Desc
 	portTxBadPkt   *prometheus.Desc
 	portRxGoodPkt  *prometheus.Desc
@@ -130,6 +131,8 @@ func New(ctx context.Context, opts Options) *Collector {
 			"State of the port (1=Enable, 0=Disable).", portLabels, nil),
 		portLinkStatus: prometheus.NewDesc("port_link_status",
 			"Link status of the port (1=Link Up, 0=Link Down).", portLabels, nil),
+		portLinkSpeed: prometheus.NewDesc("port_link_speed_mbps",
+			"Negotiated link speed in Mbps, when the firmware reports it.", portLabels, nil),
 		portTxGoodPkt: prometheus.NewDesc("port_tx_good_pkt",
 			"Number of good packets transmitted on the port.", portLabels, nil),
 		portTxBadPkt: prometheus.NewDesc("port_tx_bad_pkt",
@@ -171,8 +174,8 @@ func New(ctx context.Context, opts Options) *Collector {
 // registration time.
 func (c *Collector) Describe(ch chan<- *prometheus.Desc) {
 	for _, d := range []*prometheus.Desc{
-		c.portState, c.portLinkStatus, c.portTxGoodPkt, c.portTxBadPkt,
-		c.portRxGoodPkt, c.portRxBadPkt,
+		c.portState, c.portLinkStatus, c.portLinkSpeed,
+		c.portTxGoodPkt, c.portTxBadPkt, c.portRxGoodPkt, c.portRxBadPkt,
 		c.poeSystemConsumption, c.poeState, c.poePower, c.poeType,
 		c.poeWatts, c.poeVoltage, c.poeCurrent,
 		c.up, c.scrapeDuration, c.scrapeErrorsD, c.buildInfo,
@@ -198,6 +201,7 @@ func (c *Collector) Collect(ch chan<- prometheus.Metric) {
 	for _, p := range res.ports {
 		emitBool(ch, c.portState, p.Enabled, p.Name)
 		emitBool(ch, c.portLinkStatus, p.LinkUp, p.Name)
+		emit(ch, c.portLinkSpeed, prometheus.GaugeValue, p.LinkSpeedMbps, p.Name)
 		emit(ch, c.portTxGoodPkt, prometheus.CounterValue, p.TxGoodPkt, p.Name)
 		emit(ch, c.portTxBadPkt, prometheus.CounterValue, p.TxBadPkt, p.Name)
 		emit(ch, c.portRxGoodPkt, prometheus.CounterValue, p.RxGoodPkt, p.Name)

internal/config/config.go

@@ -61,6 +61,10 @@ type Config struct {
 	// PoE enables scraping of the PoE pages. Accepts true/false as well as
 	// the historical 1/0 spelling.
 	PoE FlexBool `yaml:"poe"`
+	// Firmware selects how the device exposes its statistics. Empty means
+	// FirmwareHTML, which is what every device supported before this key
+	// existed.
+	Firmware string `yaml:"firmware"`
 	// Web configures the exporter's own HTTP endpoint.
 	Web WebConfig `yaml:"web"`
 }
@@ -76,6 +80,23 @@ type WebConfig struct {
 	AuthPassword string `yaml:"auth_password"`
 }
 
+// Firmware families the exporter can talk to.
+const (
+	// FirmwareHTML scrapes the HTML pages of the RTL8373 style web interface.
+	FirmwareHTML = "html"
+	// FirmwareJSON reads /port_statistics.json, as served by MaxLinear based
+	// devices such as the Goodtop ZX310S-8T2XS. See issue #6.
+	FirmwareJSON = "json"
+)
+
+// FirmwareOrDefault returns the configured firmware family, defaulting to HTML.
+func (c Config) FirmwareOrDefault() string {
+	if c.Firmware == "" {
+		return FirmwareHTML
+	}
+	return strings.ToLower(strings.TrimSpace(c.Firmware))
+}
+
 // PollRate returns the minimum delay between two polls of the switch.
 func (c Config) PollRate() time.Duration {
 	if c.PollRateSeconds == nil {
@@ -110,7 +131,7 @@ func (c Config) String() string {
 	return fmt.Sprintf("address=%s username=%s password=%s poll_rate=%s "+
 		"timeout=%s poe=%t tls=%t auth=%t",
 		c.Address, c.Username, c.Password, c.PollRate(),
-		c.Timeout(), bool(c.PoE), c.TLSEnabled(), c.AuthEnabled())
+		c.Timeout(), bool(c.PoE), c.TLSEnabled(), c.AuthEnabled()) + " firmware=" + c.FirmwareOrDefault()
 }
 
 func redactIfSet(s string) string {
@@ -194,6 +215,17 @@ func (c Config) Validate() error {
 	if c.PollRateSeconds != nil && *c.PollRateSeconds < 0 {
 		return fmt.Errorf("poll_rate_seconds must not be negative, got %d", *c.PollRateSeconds)
 	}
+	switch c.FirmwareOrDefault() {
+	case FirmwareHTML:
+	case FirmwareJSON:
+		// No PoE endpoint has been reported for this family, so asking for it
+		// would fail every scrape.
+		if bool(c.PoE) {
+			return fmt.Errorf("poe is not supported with firmware: %s", FirmwareJSON)
+		}
+	default:
+		return fmt.Errorf("firmware must be %q or %q, got %q", FirmwareHTML, FirmwareJSON, c.Firmware)
+	}
 	return c.Web.validate()
 }
 

internal/config/config_test.go

@@ -367,3 +367,47 @@ func TestCheckPermissions(t *testing.T) {
 		t.Errorf("CheckPermissions(absent) = %v, want nil", err)
 	}
 }
+
+func TestFirmwareSelection(t *testing.T) {
+	clearEnv(t)
+
+	// Absent means the HTML family, which is what every earlier release did.
+	path := writeConfig(t, "address: h\nusername: u\npassword: p\n")
+	cfg, err := Load(path)
+	if err != nil {
+		t.Fatalf("Load: %v", err)
+	}
+	if cfg.FirmwareOrDefault() != FirmwareHTML {
+		t.Errorf("default firmware = %q, want %q", cfg.FirmwareOrDefault(), FirmwareHTML)
+	}
+
+	for _, spelling := range []string{"json", "JSON", " json "} {
+		path := writeConfig(t, "address: h\nusername: u\npassword: p\nfirmware: \""+spelling+"\"\n")
+		cfg, err := Load(path)
+		if err != nil {
+			t.Fatalf("Load(%q): %v", spelling, err)
+		}
+		if cfg.FirmwareOrDefault() != FirmwareJSON {
+			t.Errorf("firmware %q resolved to %q", spelling, cfg.FirmwareOrDefault())
+		}
+	}
+
+	path = writeConfig(t, "address: h\nusername: u\npassword: p\nfirmware: xml\n")
+	if _, err := Load(path); err == nil {
+		t.Error("want an error for an unknown firmware")
+	}
+}
+
+// No PoE endpoint has been reported for the JSON family, so asking for it would
+// fail every scrape. Better to refuse at startup.
+func TestFirmwareJSONRejectsPoE(t *testing.T) {
+	clearEnv(t)
+	path := writeConfig(t, "address: h\nusername: u\npassword: p\nfirmware: json\npoe: true\n")
+	_, err := Load(path)
+	if err == nil {
+		t.Fatal("want an error for poe with firmware: json")
+	}
+	if !strings.Contains(err.Error(), "poe is not supported") {
+		t.Errorf("err = %v, want it to explain the combination", err)
+	}
+}