-
Notifications
You must be signed in to change notification settings - Fork 13
feat: ec module #72
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
derekbit
merged 103 commits into
longhorn:longhorn-v25.09
from
c3y1huang:1061-v2-sharding
Jul 9, 2026
Merged
feat: ec module #72
Changes from all commits
Commits
Show all changes
103 commits
Select commit
Hold shift + click to select a range
927efd6
bdev/nvme: refresh namespace blockcnt during reset reconnect
c3y1huang e442143
bdev/ec: introduce module scaffolding and full ec_bdev data model (WI…
c3y1huang b3b0707
bdev/ec: bring up create/delete bdev lifecycle
c3y1huang c869ea7
bdev/ec: report bdev config and runtime info via JSON dump
c3y1huang ef5ab87
bdev/ec: detect base-bdev failure and tear down the slot
c3y1huang adf0e9f
bdev/ec: hot-swap a failed base bdev
c3y1huang 4cc9777
bdev/ec: serve the read path with degraded reconstruction
c3y1huang 7b6756f
bdev/ec: serve the full-stripe write path
c3y1huang fcf647f
bdev/ec: protect parity with a front-placed Write-Intent Bitmap
c3y1huang 1373dfc
bdev/ec: load and merge the WIB across parity disks on startup
c3y1huang fa5b517
bdev/ec: serve sub-stripe writes via RMW under WIB protection
c3y1huang 138c773
bdev/ec: serialize concurrent writers per stripe with stripe-busy claims
c3y1huang dda16a7
bdev/ec: back-pressure RMW around active and deferred scrub
c3y1huang b8d0966
bdev/ec: defer WRITE_ZEROES to SPDK emulation via is_zero_fill shortcut
c3y1huang 7db5811
bdev/ec: add ec_submit_rmw_zero_fill_range entry point on RMW
c3y1huang c4a131a
bdev/ec: reconstruct a failed slot via background rebuild walk
c3y1huang 82f049a
bdev/ec: control the rebuild (start/stop/QoS/progress)
c3y1huang c0718a5
bdev/ec: transition rebuilt slots to NORMAL in rebuild_finish
c3y1huang 96875d1
bdev/ec: scrub dirty regions on startup
c3y1huang 153eb61
bdev/ec: start deferred scrub after rebuild restores failed slots
c3y1huang de204f5
bdev/ec: grow capacity in place via bdev_ec_resize
c3y1huang 0c07370
bdev/ec: add the in-band unmapped-bitmap on-disk format
c3y1huang 8702c00
bdev/ec: persist the unmapped bitmap across disks
c3y1huang 69936fc
bdev/ec: load the unmapped bitmap on startup
c3y1huang 123505a
bdev/ec: clear unmapped bits asynchronously via the bit-clear waiter …
c3y1huang cc233b1
bdev/ec: serve multi-stripe aligned UNMAP via the bitmap, route singl…
c3y1huang 7f344f6
bdev/ec: dispatch unaligned multi-stripe UNMAP via head/inner/tail se…
c3y1huang 465fe28
bdev/ec: synthesize zeros on read of unmapped stripe
c3y1huang 8118d20
bdev/ec: route writes into unmapped stripes via zero-fill and deferre…
c3y1huang a4562c2
bdev/ec: expose lifecycle and data-plane JSON-RPC with cumulative I/O…
c3y1huang f20ed99
bdev/ec: add unmaps_failed counter so the UNMAP accounting identity c…
c3y1huang 9fc190a
bdev/ec: dispatch I/O via a type-indexed table instead of an inline s…
c3y1huang b5db991
bdev/ec: expose recovery, resize, and status JSON-RPC
c3y1huang 2a20355
bdev/ec: add Python CLI and RPC client bindings
c3y1huang 4da9af0
bdev/ec: refuse delete while rebuild or resize is active
c3y1huang 122c3ac
bdev/ec: add bdev_ec unit-test suite
c3y1huang 7844d20
bdev/ec: capture the submitter thread on every bdev_io
c3y1huang e06293f
bdev/ec: make cross-thread bitmap and inflight counters atomic
c3y1huang 7863a6a
bdev/ec: apply the UNMAP staged shadow via per-stripe release-store
c3y1huang d61e86a
bdev/ec: assert the submitter thread at every base-I/O dispatch site
c3y1huang 6b8ae4a
bdev/ec: route consumer submit entries to the home thread
c3y1huang dbaf849
bdev/ec: split ec_rmw_submit_core into home setup + submitter dispatch
c3y1huang 1ff61d4
bdev/ec: route the full-stripe write fan-out to the submitter thread
c3y1huang c3e89fb
bdev/ec: route the RMW write fan-out to the submitter thread
c3y1huang 24e349d
bdev/ec: make submitter-incremented counters atomic
c3y1huang 073b4d1
bdev/ec: correct the thread-safety rationale on ec_bitmap_apply_buf
c3y1huang 73de46a
bdev/ec: run the RMW WIB persist before the reads
c3y1huang 1fb36f3
bdev/ec: route the RMW read dispatch to the submitter thread
c3y1huang 2d0a3da
bdev/ec: route the write-into-unmapped fan-out to the submitter thread
c3y1huang 95ecacc
bdev/ec: route the UNMAP base fan-out and inner-completion to the sub…
c3y1huang a5a3a1f
bdev/ec: route ec_rmw_complete to the submitter thread
c3y1huang d41c693
bdev/ec: route the multi-segment UNMAP chain across home and submitter
c3y1huang c3a95e1
bdev/ec: fix wib_buf_bytes overflow on large strip_size
c3y1huang f8dac1e
bdev/ec: drop single-reactor framing and finalise the threading-model…
c3y1huang fae06cc
bdev/ec: extract ec_io_release_state to share EC-layer teardown
c3y1huang 01660e1
bdev/ec: extract ec_scrub_blocks_stripe guard for parity-write paths
c3y1huang 3000387
bdev/ec: extract ec_rmw_teardown to share RMW cleanup
c3y1huang 61e1b9d
bdev/ec: unify background-walk heartbeat between rebuild and scrub
c3y1huang 971442c
bdev/ec: unify parity reconstruct-then-re-encode helper
c3y1huang 8d3c45e
bdev/ec: adopt struct-returning progress getters for rebuild and scru…
c3y1huang a8b1259
bdev/ec: skip unmapped stripes in the scrub and rebuild walks
c3y1huang 8d68920
bdev/ec: drain in-flight persists before releasing dedicated channels
c3y1huang 044dc77
bdev/ec: enforce the WIB stripe ceiling so bitmap I/O stays within it…
c3y1huang c03e622
bdev/ec: unit-test the unmapped-stripe skip in scrub and rebuild walks
c3y1huang b9f98bf
bdev/ec: widen the WIB and unmapped-bitmap generation counters to u64
c3y1huang a25a1a3
bdev/ec: force a full scrub when a salvage create finds no valid WIB
c3y1huang f0ca313
bdev/ec: define the on-disk commit-record format
c3y1huang 136d9a0
bdev/ec: reserve the commit-record region ahead of the WIB
c3y1huang 14df442
bdev/ec: add the pure commit-selection decision for bitmap load
c3y1huang 7e5fda4
bdev/ec: extract ec_bitmap_persist_issue_round from the persist path
c3y1huang 6c5392d
bdev/ec: write a commit-record stamp after the bitmap blob reaches m+1
c3y1huang 82e69fb
bdev/ec: surface blob_crc from ec_bitmap_validate_buf
c3y1huang d01ab30
bdev/ec: adopt only committed bitmap generations on load
c3y1huang 85a2ecb
bdev/ec: rename ec_persist_in_flight to ec_teardown_must_defer
c3y1huang 17d7517
bdev/ec: defer teardown across the create window to fix create-race UAF
c3y1huang 99c8717
bdev/ec: reject delete while a create is in progress
c3y1huang ed32791
bdev/ec: defer teardown for the scrub and rebuild background walks
c3y1huang ea8d516
bdev/ec: add optional name filter to bdev_ec_get_bdevs
c3y1huang 0188b08
bdev/ec: track crash-loaded WIB regions separately from runtime write…
c3y1huang ac7fa37
bdev/ec: retarget the degraded-RMW guard to the crash-dirty map
c3y1huang 068f87b
bdev/ec: count and warn on data-slot rebuild inside crash-dirty regions
c3y1huang 41f4284
bdev/ec: expose crash-dirty counters in get_bdevs and get_wib_status
c3y1huang b6f28cd
bdev/ec: extract ec_set_unmap_limits from ec_compute_geometry
c3y1huang 8693c02
bdev/ec: extract ec_base_blockcnt_range shared by geometry and resize
c3y1huang 63e13f5
bdev/ec: compute strip_size_bytes once in ec_compute_geometry
c3y1huang 9c6b35e
bdev/ec: extract ec_wib_total_size shared by geometry and WIB fill
c3y1huang 52e691b
bdev/ec: drop the wib_buf scope block in ec_alloc_runtime_arrays
c3y1huang f6e6c30
bdev/ec: extract ec_open_dedicated_channels from ec_bdev_create_async
c3y1huang 45023bd
bdev/ec: extract ec_full_write_persist_and_dispatch from ec_submit_fu…
c3y1huang 5d281ba
bdev/ec: share the WIB region-mark trio via ec_wib_mark_region
c3y1huang d8714d8
bdev/ec: share the stripe-busy claim via ec_stripe_try_claim
c3y1huang 5a7b4c3
bdev/ec: mark region crash-dirty on partial write failure
c3y1huang 444d72a
bdev/ec: extract ec_io_route_complete_to_submitter for owner-route ha…
c3y1huang 9990296
bdev/ec: extract ec_unmap_uctx_complete for inner-UNMAP completion
c3y1huang ec77d80
bdev/ec: extract ec_rmw_apply_payload and ec_rmw_encode_parity from r…
c3y1huang 7640ab3
bdev/ec: extract ec_wib_adopt_best_copy from WIB load merge
c3y1huang 6c76529
bdev/ec: extract stripe-range helpers for the UNMAP claim/stage/publi…
c3y1huang 752b162
bdev/ec: hoist UNMAP split-arg and fan-out-hop decls out of anonymous…
c3y1huang 9602757
bdev/ec: widen failed_count to uint32_t for consistency with k/m/n
c3y1huang f51fd9f
bdev/ec: split ec_alloc_full_stripe into a zeroed primitive and a gat…
c3y1huang 3cd11d5
bdev/ec: route shared statistics counters through ec_counter_inc/add
c3y1huang fe50355
bdev/ec: add bdev_ec_* RPC methods to schema.json
c3y1huang 992aee6
bdev/ec: mark RMW region crash-dirty only after a write was issued
c3y1huang File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # SPDX-License-Identifier: BSD-3-Clause | ||
| # Copyright (C) 2026 Longhorn Authors. | ||
| # All rights reserved. | ||
|
|
||
| SPDK_ROOT_DIR := $(abspath $(CURDIR)/../../..) | ||
| include $(SPDK_ROOT_DIR)/mk/spdk.common.mk | ||
|
|
||
| SO_VER := 1 | ||
| SO_MINOR := 0 | ||
|
|
||
| C_SRCS = bdev_ec.c bdev_ec_bitmap.c bdev_ec_rpc.c bdev_ec_io.c bdev_ec_rebuild.c \ | ||
| bdev_ec_resize.c bdev_ec_rmw.c bdev_ec_wib.c bdev_ec_unmap.c | ||
|
|
||
| LIBNAME = bdev_ec | ||
|
|
||
| # bdev_ec requires ISA-L for Reed-Solomon coding. The isa-l include path and | ||
| # -lisal are provided globally by mk/spdk.common.mk when CONFIG_ISAL=y (the | ||
| # default), so they are not repeated here. | ||
|
|
||
| SPDK_MAP_FILE = $(SPDK_ROOT_DIR)/mk/spdk_blank.map | ||
|
|
||
| include $(SPDK_ROOT_DIR)/mk/spdk.lib.mk | ||
|
c3y1huang marked this conversation as resolved.
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.