Skip to content

Commit ba33bf8

Browse files
committed
vNUMA dom0: refactor commits, implement gntdev NUMA awareness
Signed-off-by: Steven Noonan <steven@edera.dev>
1 parent bde97e4 commit ba33bf8

15 files changed

Lines changed: 935 additions & 634 deletions

config.yaml

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -103,15 +103,18 @@ patches:
103103
- patch: 0003-x86-amd_node-fix-null-pointer-dereference-if-amd_smn.patch
104104
lower: '6.17'
105105
- patches:
106-
- 0001-xen-events-add-_on_node-variants-of-the-lateeoi-bind.patch
107-
- 0002-xen-xenbus-expose-host-NUMA-node-of-a-mapped-ring.patch
108-
- 0003-xen-netback-place-per-queue-kthreads-and-IRQs-near-t.patch
109-
- 0004-xen-blkback-place-per-ring-kthread-and-IRQ-near-the-.patch
110-
- 0005-xen-make-xen_alloc_unpopulated_pages-NUMA-aware.patch
111-
- 0006-xen-xenbus-collapse-xenbus_ring_host_node-to-a-page_.patch
106+
- 0001-xen-evtchn-diagnose-ring_overflow-with-post-barrier-.patch
107+
- 0002-xen-add-xen_mfn_to_node-to-resolve-a-foreign-frame-s.patch
108+
- 0003-xen-make-xen_alloc_unpopulated_pages-NUMA-aware.patch
109+
- 0004-xen-grant-table-add-gnttab_alloc_pages_node.patch
110+
- 0005-xen-events-add-_on_node-variants-of-the-lateeoi-bind.patch
111+
- 0006-xen-xenbus-home-ring-mappings-on-the-foreign-frame-s.patch
112112
- 0007-xen-xenbus-add-xenbus_setup_ring_node-for-per-node-r.patch
113-
- 0008-xen-netfront-place-per-queue-rings-on-per-queue-node.patch
114-
- 0009-xen-blkfront-place-per-ring-buffers-on-per-hctx-node.patch
113+
- 0008-xen-netback-place-per-queue-kthreads-and-IRQs-near-t.patch
114+
- 0009-xen-blkback-place-per-ring-kthread-and-IRQ-near-the-.patch
115+
- 0010-xen-netfront-place-per-queue-rings-on-per-queue-node.patch
116+
- 0011-xen-blkfront-place-per-ring-buffers-on-per-hctx-node.patch
117+
- 0012-xen-gntdev-home-grant-map-placeholder-pages-on-the-f.patch
115118
lower: '6.18'
116119
- patches:
117120
- 0001-xen-xlate_mmu-relocate-remap_pfn-utils.patch
Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
From 35760e39333ee8b3786ca54eb8e23ad72358b035 Mon Sep 17 00:00:00 2001
2+
From: Steven Noonan <steven@edera.dev>
3+
Date: Mon, 20 Jul 2026 13:49:47 -0700
4+
Subject: [PATCH 01/12] xen/evtchn: diagnose ring_overflow with post-barrier
5+
cons re-read
6+
7+
We've been seeing /dev/xen/evtchn fds enter the ring_overflow=1
8+
state under sustained event load (specifically: many ports bound
9+
to a single fd, all firing concurrently during a daemon-side
10+
metrics-collection storm). Once the flag latches, every read
11+
returns -EFBIG and the fd is dead for event delivery until reset.
12+
13+
The per-port masking invariant (each bound port can occupy at
14+
most one ring slot, because lateeoi_ack_dynirq masks the channel
15+
at Xen level before evtchn_interrupt runs and userspace can't
16+
unmask until it reads + writes the port back) combined with the
17+
ring being sized to nr_evtchns means overflow should not be
18+
reachable. No "Interrupt for port N, but apparently not enabled"
19+
WARNs precede the EFBIG state on hosts where this happens, so
20+
the masking invariant is holding.
21+
22+
One plausible candidate: the producer's READ_ONCE(ring_cons) in
23+
evtchn_interrupt has no acquire pairing with the consumer's
24+
WRITE_ONCE in evtchn_read -- they take ring_prod_lock and
25+
ring_cons_mutex respectively, neither of which synchronizes
26+
ring_cons between them. The smp_wmb/smp_rmb pair already
27+
present covers ring entry visibility against ring_prod, not
28+
ring_cons. A producer that fires right after the consumer
29+
drains can observe a stale ring_cons and compute
30+
(prod - stale_cons) >= ring_size when the ring actually has
31+
room. On x86-TSO the staleness window is bounded to
32+
store-buffer drain time but is non-zero; on weakly-ordered
33+
hardware it's wider.
34+
35+
Add an observe-only diagnostic at the overflow point: before
36+
setting ring_overflow, issue an smp_mb() and re-read ring_cons.
37+
Emit a rate-limited pr_warn with prod, both cons values, both
38+
depths, ring_size, nr_evtchns, and the triggering port's
39+
enabled/unbinding flags, tagging the log with " SPURIOUS" when
40+
the post-barrier depth is below ring_size (i.e., the original
41+
overflow detection was based on a stale view).
42+
43+
This does not change runtime behavior other than emitting a log
44+
line at the overflow boundary (which already required entering
45+
a path that wedges the fd, so the printk cost is irrelevant).
46+
If the diagnostic confirms the stale-cons hypothesis, the real
47+
fix is to switch ring_cons access to smp_load_acquire /
48+
smp_store_release pairing.
49+
50+
Signed-off-by: Steven Noonan <steven@edera.dev>
51+
---
52+
drivers/xen/evtchn.c | 31 ++++++++++++++++++++++++++++++-
53+
1 file changed, 30 insertions(+), 1 deletion(-)
54+
55+
diff --git a/drivers/xen/evtchn.c b/drivers/xen/evtchn.c
56+
index 7e4a13e632dc..85619b041ee8 100644
57+
--- a/drivers/xen/evtchn.c
58+
+++ b/drivers/xen/evtchn.c
59+
@@ -189,8 +189,37 @@ static irqreturn_t evtchn_interrupt(int irq, void *data)
60+
kill_fasync(&u->evtchn_async_queue,
61+
SIGIO, POLL_IN);
62+
}
63+
- } else
64+
+ } else {
65+
+ unsigned int cons_now;
66+
+
67+
+ /*
68+
+ * Diagnostic: the READ_ONCE of ring_cons above has no
69+
+ * acquire pairing with the WRITE_ONCE in evtchn_read --
70+
+ * the producer holds ring_prod_lock and the consumer
71+
+ * holds ring_cons_mutex, so they don't synchronize on
72+
+ * ring_cons. On weakly-ordered hardware (and within
73+
+ * the x86 store-buffer drain window) the producer can
74+
+ * observe a stale ring_cons and conclude the ring is
75+
+ * full when it actually has room. Re-read with a full
76+
+ * barrier before declaring overflow and tag the log as
77+
+ * SPURIOUS when the post-barrier read shows the ring
78+
+ * had room -- that pins the cause on the cons
79+
+ * visibility race rather than a real fill-to-capacity
80+
+ * or a broken per-port masking invariant.
81+
+ */
82+
+ smp_mb();
83+
+ cons_now = READ_ONCE(u->ring_cons);
84+
+
85+
+ pr_warn_ratelimited(
86+
+ "xen-evtchn overflow on %s: port=%u prod=%u cons=%u cons_after_mb=%u depth=%u depth_after_mb=%u ring_size=%u nr_evtchns=%u enabled=%d unbinding=%d%s\n",
87+
+ u->name, evtchn->port, prod, cons, cons_now,
88+
+ prod - cons, prod - cons_now,
89+
+ u->ring_size, u->nr_evtchns,
90+
+ (int)evtchn->enabled, (int)evtchn->unbinding,
91+
+ (prod - cons_now) < u->ring_size ? " SPURIOUS" : "");
92+
+
93+
u->ring_overflow = 1;
94+
+ }
95+
96+
spin_unlock(&u->ring_prod_lock);
97+
98+
--
99+
2.55.0
100+
Lines changed: 268 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,268 @@
1+
From b675cdf0b8448a460ff207a708aa272532b8a9a3 Mon Sep 17 00:00:00 2001
2+
From: Steven Noonan <steven@edera.dev>
3+
Date: Mon, 20 Jul 2026 13:51:31 -0700
4+
Subject: [PATCH 02/12] xen: add xen_mfn_to_node to resolve a foreign frame's
5+
host NUMA node
6+
7+
Dom0 code that grant-maps foreign pages -- xenbus ring mappings,
8+
gntdev -- has no way to learn which host NUMA node backs a foreign
9+
frame, so nothing downstream (kthread and IRQ placement, placeholder
10+
page homing) can be made node-local to the guest being served. Every
11+
backend kthread for every queue piles onto whatever node it happens
12+
to land on, regardless of the guest's vNUMA layout.
13+
14+
Add xen_mfn_to_node(), resolving one host MFN to a Linux node id via
15+
the new XENMEM_get_mfn_pxms hypercall, along with the hypercall's
16+
interface definition. It lives in grant-table.c because its callers
17+
are the grant-mapping paths that need to home placeholder pages.
18+
19+
Three NUMA identifier namespaces are involved. Xen returns host PXM
20+
(firmware-supplied), matching what dom0's own SRAT already uses;
21+
pxm_to_node() converts to a Linux dom0 node id, which is what callers
22+
feed to Linux helpers like cpumask_of_node() and
23+
kthread_create_on_node(). Keeping the Xen-side ABI in PXM-space lets
24+
callers translate with one standard lookup instead of maintaining
25+
their own Xen-nid -> Linux-node table.
26+
27+
The query is meaningful only in the hardware domain: Xen restricts
28+
XENMEM_get_mfn_pxms to it, and a guest has no view of host topology
29+
to localize against in any case. xen_mfn_to_node returns
30+
NUMA_NO_NODE immediately when !xen_initial_domain(), without issuing
31+
the hypercall.
32+
33+
Older or non-Edera hypervisors lack the hypercall. The first call
34+
returns -ENOSYS, which latches a global "unsupported" flag; every
35+
subsequent call returns NUMA_NO_NODE without entering the
36+
hypervisor. -EPERM (XSM policy, a late hardware domain) latches the
37+
same way, so no boot pays more than one refused attempt. Callers
38+
seeing NUMA_NO_NODE fall back to NUMA-oblivious behaviour, making the
39+
kernel-side change safe to ship without a lockstep hypervisor update.
40+
41+
Gated by CONFIG_XEN_BACKEND_NUMA_AFFINITY, which depends on
42+
XEN_BACKEND, NUMA, ACPI_NUMA, and XEN_UNPOPULATED_ALLOC and defaults
43+
to y. XEN_UNPOPULATED_ALLOC is a hard dependency rather than a soft
44+
one because the placement work the rest of this series performs is
45+
only meaningful when grant-map placeholders come from a per-node
46+
pool: with the generic balloon allocator, page_to_nid() reflects only
47+
where dom0's free RAM happened to be, and per-ring NUMA placement
48+
would commit to wrong nodes confidently rather than silently no-op.
49+
Disabling the option compiles the query out; the header supplies a
50+
NUMA_NO_NODE stub so callers need no ifdefs.
51+
52+
Sampling policy is left to callers. The hypercall accepts a batch of
53+
MFNs, though the consumers in this series query only a ring's first
54+
frame.
55+
56+
Signed-off-by: Steven Noonan <steven@edera.dev>
57+
---
58+
drivers/xen/Kconfig | 25 ++++++++++
59+
drivers/xen/grant-table.c | 86 ++++++++++++++++++++++++++++++++++
60+
include/xen/interface/memory.h | 26 ++++++++++
61+
include/xen/xen.h | 16 +++++++
62+
4 files changed, 153 insertions(+)
63+
64+
diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
65+
index f9a35ed266ec..147e6acb231b 100644
66+
--- a/drivers/xen/Kconfig
67+
+++ b/drivers/xen/Kconfig
68+
@@ -96,6 +96,31 @@ config XEN_BACKEND
69+
Support for backend device drivers that provide I/O services
70+
to other virtual machines.
71+
72+
+config XEN_BACKEND_NUMA_AFFINITY
73+
+ bool "NUMA affinity for Xen backend drivers"
74+
+ depends on XEN_BACKEND && NUMA && ACPI_NUMA && XEN_UNPOPULATED_ALLOC
75+
+ default y
76+
+ help
77+
+ Allow Xen backend drivers (netback, blkback, gntdev consumers)
78+
+ to discover the host NUMA node that hosts a grant-mapped ring
79+
+ page, and to place their service threads and IRQs on that node.
80+
+
81+
+ XEN_UNPOPULATED_ALLOC provides the per-node placeholder-page pool
82+
+ the relocation logic in xenbus_map_ring_valloc() draws from.
83+
+ Without it, placeholders come from the generic balloon allocator,
84+
+ whose page_to_nid() reflects only where dom0's free RAM happened
85+
+ to be -- not the host node of the foreign frame the placeholder
86+
+ will end up backing. In that configuration the per-ring
87+
+ placement decisions would be confidently wrong rather than just
88+
+ absent, so the Kconfig hard-depends on XEN_UNPOPULATED_ALLOC
89+
+ rather than silently degrading.
90+
+
91+
+ Requires hypervisor support for the XENMEM_get_mfn_pxms
92+
+ hypercall. Without that support the feature is silently a
93+
+ no-op, equivalent to NUMA-oblivious behaviour.
94+
+
95+
+ If unsure, say Y.
96+
+
97+
config XENFS
98+
tristate "Xen filesystem"
99+
select XEN_PRIVCMD
100+
diff --git a/drivers/xen/grant-table.c b/drivers/xen/grant-table.c
101+
index 478d2ad725ac..f8b86a8679de 100644
102+
--- a/drivers/xen/grant-table.c
103+
+++ b/drivers/xen/grant-table.c
104+
@@ -67,6 +67,10 @@
105+
106+
#include <asm/sync_bitops.h>
107+
108+
+#ifdef CONFIG_XEN_BACKEND_NUMA_AFFINITY
109+
+#include <acpi/acpi_numa.h>
110+
+#endif
111+
+
112+
#define GNTTAB_LIST_END 0xffffffff
113+
114+
static grant_ref_t **gnttab_list;
115+
@@ -881,6 +885,88 @@ int gnttab_pages_set_private(int nr_pages, struct page **pages)
116+
}
117+
EXPORT_SYMBOL_GPL(gnttab_pages_set_private);
118+
119+
+#ifdef CONFIG_XEN_BACKEND_NUMA_AFFINITY
120+
+/*
121+
+ * Tri-state cache for XENMEM_get_mfn_pxms availability. -ENOSYS
122+
+ * (hypervisor without the hypercall) or -EPERM (Xen restricts the
123+
+ * query to the hardware domain) from the first attempt latches
124+
+ * "unsupported", short-circuiting future calls. Any positive ACK
125+
+ * (including the legitimate XEN_INVALID_NUMA_ID answer for an MFN Xen
126+
+ * does not know about) latches "supported".
127+
+ *
128+
+ * Lock-free: at most one transition each direction, and "unsupported"
129+
+ * is a stable terminal state once entered. A racing reader might
130+
+ * issue one redundant hypercall before observing the cached state,
131+
+ * which is harmless.
132+
+ */
133+
+#define XEN_MFN_PXM_UNKNOWN 0
134+
+#define XEN_MFN_PXM_SUPPORTED 1
135+
+#define XEN_MFN_PXM_UNSUPPORTED 2
136+
+
137+
+static int xen_mfn_pxm_state = XEN_MFN_PXM_UNKNOWN;
138+
+
139+
+/*
140+
+ * Resolve one foreign MFN to a Linux node id. Returns NUMA_NO_NODE
141+
+ * for any failure mode: hypercall unsupported, MFN unknown to Xen,
142+
+ * PXM not registered with the dom0 ACPI namespace.
143+
+ *
144+
+ * Three NUMA identifier namespaces are involved here. Xen returns
145+
+ * host PXM (firmware-supplied). pxm_to_node() translates to a Linux
146+
+ * dom0 node id. Callers then use the result against Linux helpers
147+
+ * like cpumask_of_node() and kthread_create_on_node().
148+
+ */
149+
+int xen_mfn_to_node(unsigned long mfn)
150+
+{
151+
+ struct xen_get_mfn_pxms req;
152+
+ xen_pfn_t mfn_arg = mfn;
153+
+ uint32_t pxm = XEN_INVALID_NUMA_ID;
154+
+ int rc;
155+
+
156+
+ /*
157+
+ * Xen answers this query only for the hardware domain. A domU
158+
+ * mapping a foreign ring (a driver domain, say) would just earn
159+
+ * an -EPERM per connect; skip the doomed hypercall and stay
160+
+ * node-oblivious, which is the only honest answer for a guest
161+
+ * with no view of host topology anyway.
162+
+ */
163+
+ if (!xen_initial_domain())
164+
+ return NUMA_NO_NODE;
165+
+
166+
+ if (READ_ONCE(xen_mfn_pxm_state) == XEN_MFN_PXM_UNSUPPORTED)
167+
+ return NUMA_NO_NODE;
168+
+
169+
+ memset(&req, 0, sizeof(req));
170+
+ set_xen_guest_handle(req.mfns, &mfn_arg);
171+
+ set_xen_guest_handle(req.pxms, &pxm);
172+
+ req.nr_mfns = 1;
173+
+
174+
+ rc = HYPERVISOR_memory_op(XENMEM_get_mfn_pxms, &req);
175+
+ if (rc < 0) {
176+
+ /*
177+
+ * Both errnos are stable properties of this boot: -ENOSYS
178+
+ * means the hypervisor lacks the hypercall (no CONFIG_NUMA
179+
+ * or too old), -EPERM that it refuses us despite the
180+
+ * initial-domain check above (XSM policy, or a late
181+
+ * hardware domain). Latch either so we never retry.
182+
+ */
183+
+ if (rc == -ENOSYS || rc == -EPERM) {
184+
+ WRITE_ONCE(xen_mfn_pxm_state, XEN_MFN_PXM_UNSUPPORTED);
185+
+ pr_info("XENMEM_get_mfn_pxms unavailable (%d), NUMA affinity for grant mappings disabled\n",
186+
+ rc);
187+
+ }
188+
+ return NUMA_NO_NODE;
189+
+ }
190+
+
191+
+ WRITE_ONCE(xen_mfn_pxm_state, XEN_MFN_PXM_SUPPORTED);
192+
+
193+
+ if (pxm == XEN_INVALID_NUMA_ID)
194+
+ return NUMA_NO_NODE;
195+
+
196+
+ return pxm_to_node(pxm);
197+
+}
198+
+EXPORT_SYMBOL_GPL(xen_mfn_to_node);
199+
+#endif /* CONFIG_XEN_BACKEND_NUMA_AFFINITY */
200+
+
201+
/**
202+
* gnttab_alloc_pages - alloc pages suitable for grant mapping into
203+
* @nr_pages: number of pages to alloc
204+
diff --git a/include/xen/interface/memory.h b/include/xen/interface/memory.h
205+
index 1a371a825c55..1998a12a9465 100644
206+
--- a/include/xen/interface/memory.h
207+
+++ b/include/xen/interface/memory.h
208+
@@ -325,4 +325,30 @@ struct xen_mem_acquire_resource {
209+
};
210+
DEFINE_GUEST_HANDLE_STRUCT(xen_mem_acquire_resource);
211+
212+
+/*
213+
+ * XENMEM_get_mfn_pxms: resolve a batch of host MFNs to their firmware
214+
+ * proximity-domain identifiers (host PXM on x86 ACPI).
215+
+ *
216+
+ * Returned values are in the host PXM namespace (the same value space
217+
+ * dom0's own SRAT uses), not Xen's internal node id. Callers convert
218+
+ * to a Linux node id with pxm_to_node(). Slots Xen has no node info
219+
+ * for receive XEN_INVALID_NUMA_ID rather than failing the whole batch.
220+
+ *
221+
+ * Restricted to the hardware domain. On hypervisors that do not
222+
+ * provide this op (older or non-Edera Xen, or builds without
223+
+ * CONFIG_NUMA), the hypercall returns -ENOSYS; callers treat that as
224+
+ * "feature unavailable" and fall back to NUMA-oblivious behaviour.
225+
+ */
226+
+#define XENMEM_get_mfn_pxms 40
227+
+
228+
+#define XEN_INVALID_NUMA_ID (~(uint32_t)0)
229+
+
230+
+struct xen_get_mfn_pxms {
231+
+ GUEST_HANDLE(xen_pfn_t) mfns;
232+
+ GUEST_HANDLE(uint32_t) pxms;
233+
+ uint32_t nr_mfns;
234+
+ uint32_t flags;
235+
+};
236+
+DEFINE_GUEST_HANDLE_STRUCT(xen_get_mfn_pxms);
237+
+
238+
#endif /* __XEN_PUBLIC_MEMORY_H__ */
239+
diff --git a/include/xen/xen.h b/include/xen/xen.h
240+
index f280c5dcf923..f7f3dd7a3abe 100644
241+
--- a/include/xen/xen.h
242+
+++ b/include/xen/xen.h
243+
@@ -89,6 +89,22 @@ static inline void xen_free_unpopulated_pages(unsigned int nr_pages,
244+
}
245+
#endif
246+
247+
+/*
248+
+ * Resolve a foreign frame's host MFN to the Linux node id of the memory
249+
+ * backing it, via XENMEM_get_mfn_pxms. NUMA_NO_NODE for any failure
250+
+ * mode (hypercall unsupported or refused, MFN unknown to Xen, PXM not
251+
+ * in the ACPI namespace) -- callers degrade to node-oblivious behaviour.
252+
+ */
253+
+#include <linux/numa.h>
254+
+#ifdef CONFIG_XEN_BACKEND_NUMA_AFFINITY
255+
+int xen_mfn_to_node(unsigned long mfn);
256+
+#else
257+
+static inline int xen_mfn_to_node(unsigned long mfn)
258+
+{
259+
+ return NUMA_NO_NODE;
260+
+}
261+
+#endif
262+
+
263+
#if defined(CONFIG_XEN_DOM0) && defined(CONFIG_ACPI) && defined(CONFIG_X86)
264+
bool __init xen_processor_present(uint32_t acpi_id);
265+
#else
266+
--
267+
2.55.0
268+

0 commit comments

Comments
 (0)