Skip to content

Name thread pools to match production: ThriftSrv.IO, SREventBase, RANKER, GlobalCPUThread (#718) - #718

Closed
excelle08 wants to merge 7 commits into
facebookresearch:v2-betafrom
excelle08:export-D103766488-to-v2-beta
Closed

Name thread pools to match production: ThriftSrv.IO, SREventBase, RANKER, GlobalCPUThread (#718)#718
excelle08 wants to merge 7 commits into
facebookresearch:v2-betafrom
excelle08:export-D103766488-to-v2-beta

Conversation

@excelle08

@excelle08 excelle08 commented Jul 2, 2026 •
edited by meta-codesync Bot
Loading

Copy link
Copy Markdown
Contributor

Summary:

LeafNodeRank's existing thread pools were anonymous from Strobelight's point of view, so the prod-vs-bench thread-pool breakdown landed almost entirely in the Unknown / framework-noise bucket on CPL and BGM. Per the multifeed_aggregator prod profile (~/feedsim_v2/profiles/multifeed_aggregator_main_prod/), the four hot pools are ThriftSrv.IO, SREventBase{N}, RANKER-{N}, and GlobalCPUThread (see ~/feedsim_v2/docs/phase4_researcher_notes.md section 1).

This diff is the Programmer-A half of Phase 4 (thread pools only). It renames the four existing pools to match the prod names that Strobelight categorizes, and adds two new pools (SREventBase, RANKER) that Phase 5 will start dispatching outbound RPC fanout onto. Programmer-B's diff (thrift schema + 5 new method registrations) lands as a sibling commit; via() callsites stay on their existing pool aliases so this rename is a no-op behaviorally.

Changes:

  • cpuThreadPool is now backed by folly::getGlobalCPUExecutor() (the folly singleton already exposes its threads as GlobalCPUThread via NamedThreadFactory("GlobalCPUThreadPool") in folly/executors/GlobalExecutor.cpp:65). DO NOT instantiate a second CPU pool — that would double-count the prod GlobalCPUThread category. The ThreadData::cpuThreadPool field type changed from shared_ptr<CPUThreadPoolExecutor> to shared_ptr<folly::Executor> so the same field can hold the global singleton via an aliasing shared_ptr that owns a folly::Executor::KeepAlive<>. All existing folly::via(this_thread.cpuThreadPool.get(), ...) callsites continue to compile because folly::via accepts an Executor*.
  • srvCPUThreadPool is now NamedThreadFactory("RANKER"), sized max(1, nproc/2) by default (matches CPL prod: 26 RANKER threads on a 52-logical-core host). Tunable via the new --ranker_threads flag.
  • ioThreadPool is now NamedThreadFactory("ThriftSrv.IO"), sized --io_threads. Was previously anonymous (the kernel just labeled the threads with the executable name).
  • New srEventBasePool (folly::IOThreadPoolExecutor, NamedThreadFactory("SREventBase")), sized max(1, nproc * 7 / 10) (matches CPL prod: 39 SREventBase threads on a 52-logical-core host). Tunable via the new --sr_event_base_threads flag. Idle in Phase 4 — Phase 5 wires the outbound mock_services fanout onto it. Threads are warmed up at server start so Strobelight sees them in steady state.
  • The legacy srvIOThreadPool (compression dispatch) is preserved for now and retired in Phase 6 once compression callsites move onto GlobalCPUThread per the researcher notes.
  • New CLI flags --ranker_threads and --sr_event_base_threads in LeafNodeRankCmdline.ggo. Default 0 means "auto-compute from folly::available_concurrency() per the formulas above".
  • Includes added: <folly/executors/GlobalExecutor.h> and <folly/system/HardwareConcurrency.h>. Both are already on ${FOLLY_INCLUDE_DIR} in the existing CMake target so no CMakeLists.txt edits were required.

Reviewed By: charles-typ

Differential Revision: D103766488

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 2, 2026
@meta-codesync

meta-codesync Bot commented Jul 2, 2026

Copy link
Copy Markdown

@excelle08 has exported this pull request. If you are a Meta employee, you can view the originating Diff in D103766488.

excelle08 added a commit to excelle08/DCPerf-1 that referenced this pull request Jul 2, 2026
@excelle08 @meta-codesync
Name thread pools to match production: ThriftSrv.IO, SREventBase, RAN…
d067bb1 
…KER, GlobalCPUThread (facebookresearch#718)

Summary:
Pull Request resolved: facebookresearch#718

LeafNodeRank's existing thread pools were anonymous from Strobelight's point of view, so the prod-vs-bench thread-pool breakdown landed almost entirely in the `Unknown` / framework-noise bucket on CPL and BGM. Per the multifeed_aggregator prod profile (~/feedsim_v2/profiles/multifeed_aggregator_main_prod/), the four hot pools are `ThriftSrv.IO`, `SREventBase{N}`, `RANKER-{N}`, and `GlobalCPUThread` (see ~/feedsim_v2/docs/phase4_researcher_notes.md section 1).

This diff is the Programmer-A half of Phase 4 (thread pools only). It renames the four existing pools to match the prod names that Strobelight categorizes, and adds two new pools (`SREventBase`, `RANKER`) that Phase 5 will start dispatching outbound RPC fanout onto. Programmer-B's diff (thrift schema + 5 new method registrations) lands as a sibling commit; via() callsites stay on their existing pool aliases so this rename is a no-op behaviorally.

Changes:
- `cpuThreadPool` is now backed by `folly::getGlobalCPUExecutor()` (the folly singleton already exposes its threads as `GlobalCPUThread` via `NamedThreadFactory("GlobalCPUThreadPool")` in `folly/executors/GlobalExecutor.cpp:65`). DO NOT instantiate a second CPU pool — that would double-count the prod `GlobalCPUThread` category. The `ThreadData::cpuThreadPool` field type changed from `shared_ptr<CPUThreadPoolExecutor>` to `shared_ptr<folly::Executor>` so the same field can hold the global singleton via an aliasing shared_ptr that owns a `folly::Executor::KeepAlive<>`. All existing `folly::via(this_thread.cpuThreadPool.get(), ...)` callsites continue to compile because `folly::via` accepts an `Executor*`.
- `srvCPUThreadPool` is now `NamedThreadFactory("RANKER")`, sized `max(1, nproc/2)` by default (matches CPL prod: 26 RANKER threads on a 52-logical-core host). Tunable via the new `--ranker_threads` flag.
- `ioThreadPool` is now `NamedThreadFactory("ThriftSrv.IO")`, sized `--io_threads`. Was previously anonymous (the kernel just labeled the threads with the executable name).
- New `srEventBasePool` (`folly::IOThreadPoolExecutor`, `NamedThreadFactory("SREventBase")`), sized `max(1, nproc * 7 / 10)` (matches CPL prod: 39 SREventBase threads on a 52-logical-core host). Tunable via the new `--sr_event_base_threads` flag. Idle in Phase 4 — Phase 5 wires the outbound mock_services fanout onto it. Threads are warmed up at server start so Strobelight sees them in steady state.
- The legacy `srvIOThreadPool` (compression dispatch) is preserved for now and retired in Phase 6 once compression callsites move onto `GlobalCPUThread` per the researcher notes.
- New CLI flags `--ranker_threads` and `--sr_event_base_threads` in `LeafNodeRankCmdline.ggo`. Default `0` means "auto-compute from `folly::available_concurrency()` per the formulas above".
- Includes added: `<folly/executors/GlobalExecutor.h>` and `<folly/system/HardwareConcurrency.h>`. Both are already on `${FOLLY_INCLUDE_DIR}` in the existing CMake target so no `CMakeLists.txt` edits were required.

Reviewed By: charles-typ

Differential Revision: D103766488
@meta-codesync meta-codesync Bot changed the title Name thread pools to match production: ThriftSrv.IO, SREventBase, RANKER, GlobalCPUThread Name thread pools to match production: ThriftSrv.IO, SREventBase, RANKER, GlobalCPUThread (#718) Jul 2, 2026
@excelle08
excelle08 force-pushed the export-D103766488-to-v2-beta branch from 4458158 to d067bb1 Compare July 2, 2026 20:13
excelle08 added 7 commits July 3, 2026 11:13
@excelle08 @meta-codesync
Add scaled-up feature extraction pipeline
07d7197 
Summary:
Add mock feature extraction pipeline to FeedSim with large-scale code
generation for I-cache and frontend pressure. 27 genuinely diverse code
patterns (derived from studying 696 production feature extractors) generate
~700 variants × 1000 copies = ~700K unique functions at install time.

Key components:
- 6 hand-written extractors based on production leaf function profiling
- 27 pattern-specific code generators (P01-P27) producing genuinely
  different instruction sequences (different branch topologies, loop
  nesting, data access patterns, code sizes from 10 to 2300 lines)
- Flat shuffled dispatch: all copy function pointers shuffled into one
  vector, iterated sequentially per request for maximum I-cache pressure
- DLRM medium/large model generation on-server during install
- Configurable via --num_stories, --extractors_per_story, --feature_complexity

Results on T1_BGM (Bergamo, 176 cores):
  500K calls/req: L1 I-Cache MPKI 21.34 (prod target 21), IPC 0.69 (prod 0.6-0.8)
  100K calls/req: Frontend Bound 23.5%, IPC 1.22, QPS 242

Results on T11_GRC_ARM (Grace, 72 cores):
  100K calls/req: IPC 0.52, L1 I-Cache MPKI 15.91
  Medium DLRM + 100K calls: IPC 1.03 (prod target 1.05)

Differential Revision: D97022149
@excelle08 @meta-codesync
Refactor feature extractors to match production instruction mix
2ad0723 
Summary:
Replace scalar FP transforms with integer hash operations, add data-dependent
conditional branches, eliminate FP division with integer reciprocal approximation,
deepen MockHashTable::find() call chain from 1 to 5 levels, and increase basic
block sizes with MurmurHash-style computation chains.

Results (5/7 instruction mix targets met on CPL):
- Scalar FP: 8% → 0.46% (target <3.5%) ✓
- Conditional branches: 5.28% → 10.73% (target >15%) ✗
- Near call/return: 3.93% → 0.75% (target <1.5%) ✓
- Memory (ld+st): 51.56% → 40.72% (target 40-46%) ✓
- Divider active: 11.50% → 1.13% (target <2%) ✓
- Avg BB size: 7.3 → 13.7 (LBR, target >18) ✗

QPS impact: CPL -2.2%, BGM -6.3%, Grace -3.8%.

Differential Revision: D99494831
@excelle08 @meta-codesync
Replace oldisim framework with folly-based FeedSimServer/FeedSimDriver
2d162f0 
Summary:
Replace oldisim framework internals (libevent server, pthreads, boost::lockfree) with
folly-based equivalents for LeafNodeRank and DriverNodeRank. ParentNodeRank retains
oldisim dependency.

New files:
- FeedSimProtocol.h: Wire protocol types (binary compatible with oldisim)
- FeedSimServer.{h,cc}: Server using folly::AsyncServerSocket + folly::EventBase
- FeedSimDriver.{h,cc}: Client driver with libevent for timer precision

Modified:
- LeafNodeRank.cc: Use feedsim::FeedSimServer, feedsim::RequestContext
- DriverNodeRank.cc: Use feedsim::FeedSimDriver, feedsim::TestDriver
- CMakeLists.txt: Add FeedSimFramework library, replace OLDISim link dep
- utils.h: Remove oldisim DIE() dependency
- run.sh: Change readiness check from HTTP monitor port to TCP data port

Differential Revision: D99498073
@excelle08 @meta-codesync
Add Silesia corpus client-server story pipeline
a8aea32 
Summary:
Phase 3 of FeedSim v2 refactor. Client loads the Silesia compression corpus
(203MB, 12 files) via mmap, picks random snippets as "stories," and sends them
to the server via thrift. Server uses story content to derive data-dependent
feature extraction inputs and DLRM features instead of random data.

Changes:
- Add StoryContent/StoryBatch thrift types and story_batch field on RankingRequest
- Add SilesiaLoader.h: mmap-based corpus loader with random snippet serving
- Update DriverNodeRank to load Silesia at startup, populate stories per request
- Update LeafNodeRank to extract stories, derive features from content bytes
  (byte frequency histogram -> dense features, rolling bigram hash -> sparse)
- Rewrite DLRMRequestHandler from sync to async with folly futures
  (I/O simulation + compression + pointer chase, pipelined)
- Add storyContent/storyContentLength fields to CopyContext for extractors
- Fix feature_suite missing from ThreadData (lost during rebase)
- Fix $feature_opts not passed to LeafNodeRank in run.sh
- Fix runFeatureExtraction() never called from request handlers
- Fix ICacheBuster SIGSEGV: init moved outside PAGERANK block
- Remove ICacheBuster from DLRMRequestHandler (DLRM inference is own workload)
- Add --silesia-dir, --stories-per-request, --story-size-min/max CLI options
- Download Silesia corpus during install (x86 + aarch64)

Differential Revision: D104076734
@excelle08 @meta-codesync
Match production request size distribution in DriverNodeRank
bee936d 
Summary:
FeedSim's profile shows RPC at 4-5% vs production's 30-34% — partly because the benchmark sends tiny requests with no resemblance to production's payload size distribution. This diff lets the client sample a target serialized request size from a JSON percentile distribution and pad the request to hit that size.

Changes:
- Add `optional binary padding` field to `RankingRequest` thrift struct
- New `RequestSizeSampler` (header-only) loads a JSON file of percentile data (`req_size_min`, `req_size_p05`, ... `req_size_max`) and samples target sizes via inverse-CDF with linear interpolation between percentiles
- Add `--req_size_dist <json>` flag to DriverNodeRank. When present, each request is built normally, then padded with Silesia bytes (or random bytes if Silesia not loaded) to reach the sampled target size
- Plumb `--req-size-dist` through `run.sh` with auto-detection of `feed_aggregator_req_sizes.json` next to `run.sh`
- Bundle `feed_aggregator_req_sizes.json` and `feed_aggregator_resp_sizes.json` (production data from ServiceRouter) and copy them in install scripts

Differential Revision: D102693799
@excelle08 @meta-codesync
Refactor LeafNodeRank: split DLRM functions, async DLRM, Silesia resp…
6b715a1 
…onse generator

Summary:
Several cleanups in LeafNodeRank, all motivated by the leaf-only hot-func breakdown which surfaced ~15% of CPU on server-side response RNG and the misleading-named dlrmInferenceServerSideDataGeneration:

1. Split DLRM inference into two functions, both async (return folly::Future<int>):
   - dlrmInferenceServerSide: inference path where features are generated inside DLRM::infer (server-side)
   - dlrmInferenceClientSide: inference path that uses DLRM::inferWithFeatures with client-provided dense+sparse features

   The old name dlrmInferenceServerSideDataGeneration hid the actual ML inference call (this_thread.dlrm_ranker->infer) inside a function whose name suggested it was just generating feature data. The new names are honest. A shared shardInferences() helper distributes work across cpu_threads_arg.

2. Rewrite DLRMRequestHandler to be fully async with a single future chain (DLRM inference -> I/O sleep -> compression -> pointer chase -> generate+send response). The previous code blocked synchronously on the inference future via .get() before starting the rest of the pipeline. Now the handler thread returns immediately after kicking off the chain.

3. Pick the right inference function based on what the client sent: if request.dlrm_features() is set, use dlrmInferenceClientSide (no server RNG for inputs); otherwise dlrmInferenceServerSide.

4. Remove the dead `else if (g_workload_type == DLRM)` branches from PageRankRequestHandler and AsyncPageRankRequestHandler. DLRM workload requests use kDLRMRequestType, which routes to DLRMRequestHandler (registered separately in main()), so the DLRM branch in PageRank handlers was unreachable.

5. New Silesia-backed server response generator (generators/SilesiaResponseGenerator.h). When the server is started with --silesia_dir, response RankingObjects/RankingStorys are populated by slicing bytes out of the mmap'd Silesia corpus instead of running xor128() RNG. Replaces ~15% of leaf CPU previously spent on RNG (mersenne_twister, generateRandomString, xor128) with cheap memcpy from a hot mmap. The bytes have realistic entropy for downstream ZSTD compression.

6. Added LeafNodeRank --silesia_dir option and plumbed through run.sh. The same --silesia-dir flag now provides bytes both to DriverNodeRank's story_batch and to the server's response generator.

Differential Revision: D103100125
@excelle08 @meta-codesync
Name thread pools to match production: ThriftSrv.IO, SREventBase, RAN…
4f8a2b0 
…KER, GlobalCPUThread (facebookresearch#718)

Summary:
Pull Request resolved: facebookresearch#718

LeafNodeRank's existing thread pools were anonymous from Strobelight's point of view, so the prod-vs-bench thread-pool breakdown landed almost entirely in the `Unknown` / framework-noise bucket on CPL and BGM. Per the multifeed_aggregator prod profile (~/feedsim_v2/profiles/multifeed_aggregator_main_prod/), the four hot pools are `ThriftSrv.IO`, `SREventBase{N}`, `RANKER-{N}`, and `GlobalCPUThread` (see ~/feedsim_v2/docs/phase4_researcher_notes.md section 1).

This diff is the Programmer-A half of Phase 4 (thread pools only). It renames the four existing pools to match the prod names that Strobelight categorizes, and adds two new pools (`SREventBase`, `RANKER`) that Phase 5 will start dispatching outbound RPC fanout onto. Programmer-B's diff (thrift schema + 5 new method registrations) lands as a sibling commit; via() callsites stay on their existing pool aliases so this rename is a no-op behaviorally.

Changes:
- `cpuThreadPool` is now backed by `folly::getGlobalCPUExecutor()` (the folly singleton already exposes its threads as `GlobalCPUThread` via `NamedThreadFactory("GlobalCPUThreadPool")` in `folly/executors/GlobalExecutor.cpp:65`). DO NOT instantiate a second CPU pool — that would double-count the prod `GlobalCPUThread` category. The `ThreadData::cpuThreadPool` field type changed from `shared_ptr<CPUThreadPoolExecutor>` to `shared_ptr<folly::Executor>` so the same field can hold the global singleton via an aliasing shared_ptr that owns a `folly::Executor::KeepAlive<>`. All existing `folly::via(this_thread.cpuThreadPool.get(), ...)` callsites continue to compile because `folly::via` accepts an `Executor*`.
- `srvCPUThreadPool` is now `NamedThreadFactory("RANKER")`, sized `max(1, nproc/2)` by default (matches CPL prod: 26 RANKER threads on a 52-logical-core host). Tunable via the new `--ranker_threads` flag.
- `ioThreadPool` is now `NamedThreadFactory("ThriftSrv.IO")`, sized `--io_threads`. Was previously anonymous (the kernel just labeled the threads with the executable name).
- New `srEventBasePool` (`folly::IOThreadPoolExecutor`, `NamedThreadFactory("SREventBase")`), sized `max(1, nproc * 7 / 10)` (matches CPL prod: 39 SREventBase threads on a 52-logical-core host). Tunable via the new `--sr_event_base_threads` flag. Idle in Phase 4 — Phase 5 wires the outbound mock_services fanout onto it. Threads are warmed up at server start so Strobelight sees them in steady state.
- The legacy `srvIOThreadPool` (compression dispatch) is preserved for now and retired in Phase 6 once compression callsites move onto `GlobalCPUThread` per the researcher notes.
- New CLI flags `--ranker_threads` and `--sr_event_base_threads` in `LeafNodeRankCmdline.ggo`. Default `0` means "auto-compute from `folly::available_concurrency()` per the formulas above".
- Includes added: `<folly/executors/GlobalExecutor.h>` and `<folly/system/HardwareConcurrency.h>`. Both are already on `${FOLLY_INCLUDE_DIR}` in the existing CMake target so no `CMakeLists.txt` edits were required.

Reviewed By: charles-typ

Differential Revision: D103766488
@excelle08
excelle08 force-pushed the export-D103766488-to-v2-beta branch from d067bb1 to 4f8a2b0 Compare July 3, 2026 18:19
meta-codesync Bot pushed a commit that referenced this pull request Jul 4, 2026
@excelle08 @meta-codesync
Name thread pools to match production: ThriftSrv.IO, SREventBase, RAN…
e46510e 
…KER, GlobalCPUThread (#718)

Summary:
Pull Request resolved: #718

LeafNodeRank's existing thread pools were anonymous from Strobelight's point of view, so the prod-vs-bench thread-pool breakdown landed almost entirely in the `Unknown` / framework-noise bucket on CPL and BGM. Per the multifeed_aggregator prod profile (~/feedsim_v2/profiles/multifeed_aggregator_main_prod/), the four hot pools are `ThriftSrv.IO`, `SREventBase{N}`, `RANKER-{N}`, and `GlobalCPUThread` (see ~/feedsim_v2/docs/phase4_researcher_notes.md section 1).

This diff is the Programmer-A half of Phase 4 (thread pools only). It renames the four existing pools to match the prod names that Strobelight categorizes, and adds two new pools (`SREventBase`, `RANKER`) that Phase 5 will start dispatching outbound RPC fanout onto. Programmer-B's diff (thrift schema + 5 new method registrations) lands as a sibling commit; via() callsites stay on their existing pool aliases so this rename is a no-op behaviorally.

Changes:
- `cpuThreadPool` is now backed by `folly::getGlobalCPUExecutor()` (the folly singleton already exposes its threads as `GlobalCPUThread` via `NamedThreadFactory("GlobalCPUThreadPool")` in `folly/executors/GlobalExecutor.cpp:65`). DO NOT instantiate a second CPU pool — that would double-count the prod `GlobalCPUThread` category. The `ThreadData::cpuThreadPool` field type changed from `shared_ptr<CPUThreadPoolExecutor>` to `shared_ptr<folly::Executor>` so the same field can hold the global singleton via an aliasing shared_ptr that owns a `folly::Executor::KeepAlive<>`. All existing `folly::via(this_thread.cpuThreadPool.get(), ...)` callsites continue to compile because `folly::via` accepts an `Executor*`.
- `srvCPUThreadPool` is now `NamedThreadFactory("RANKER")`, sized `max(1, nproc/2)` by default (matches CPL prod: 26 RANKER threads on a 52-logical-core host). Tunable via the new `--ranker_threads` flag.
- `ioThreadPool` is now `NamedThreadFactory("ThriftSrv.IO")`, sized `--io_threads`. Was previously anonymous (the kernel just labeled the threads with the executable name).
- New `srEventBasePool` (`folly::IOThreadPoolExecutor`, `NamedThreadFactory("SREventBase")`), sized `max(1, nproc * 7 / 10)` (matches CPL prod: 39 SREventBase threads on a 52-logical-core host). Tunable via the new `--sr_event_base_threads` flag. Idle in Phase 4 — Phase 5 wires the outbound mock_services fanout onto it. Threads are warmed up at server start so Strobelight sees them in steady state.
- The legacy `srvIOThreadPool` (compression dispatch) is preserved for now and retired in Phase 6 once compression callsites move onto `GlobalCPUThread` per the researcher notes.
- New CLI flags `--ranker_threads` and `--sr_event_base_threads` in `LeafNodeRankCmdline.ggo`. Default `0` means "auto-compute from `folly::available_concurrency()` per the formulas above".
- Includes added: `<folly/executors/GlobalExecutor.h>` and `<folly/system/HardwareConcurrency.h>`. Both are already on `${FOLLY_INCLUDE_DIR}` in the existing CMake target so no `CMakeLists.txt` edits were required.

Reviewed By: charles-typ

Differential Revision: D103766488

fbshipit-source-id: cf7b5482b329d8ed2cb06a3720f8db1b14b02cd8
@excelle08 excelle08 closed this Jul 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. meta-exported

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@excelle08