Commit 81cb947
committed
smp_load_acquire(ring->cq.khead) in __io_uring_peek_cqe() for
consistent dereferencing
__io_uring_peek_cqe() is on the critical path for the user to consume
completion-queue entries. It load_acquires the tail, loads the head,
uses the head to index into the cq ring.
The C memory consistency model does not enforce ordering for address
dependencies, like the one seen in:
```
LD1: unsigned head = *ring->cq.khead;
...
LD2: cqe = &ring->cq.cqes[(head & mask) << shift];
```
where LD2 is address-dependent on LD1. This means that LD2 could
load a stale cqe due to LD1 being ordered after LD2. If instead,
LD1 was a smp_load_acquire, the ordering would be enforced and
correct cqes would be loaded.
Signed-off by: Soham Bagchi <soham.bagchi@utah.edu>
Co-authored by: Marco Elver <elver@google.com>
Co-authored by: Vijay Nagarajan <vijay@cs.utah.edu>
Co-authored by: Ryan Stutsman <stutsman@cs.utah.edu>1 parent 2fd7904 commit 81cb947
1 file changed
Lines changed: 6 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1576 | 1576 | | |
1577 | 1577 | | |
1578 | 1578 | | |
1579 | | - | |
| 1579 | + | |
| 1580 | + | |
| 1581 | + | |
| 1582 | + | |
| 1583 | + | |
| 1584 | + | |
1580 | 1585 | | |
1581 | 1586 | | |
1582 | 1587 | | |
| |||
0 commit comments