Pre-build 64-slot response template pool to eliminate per-request RPC-DataGen cost (facebookresearch#742)

Summary:
Pull Request resolved: facebookresearch#742

The t31v6 BGM hot-function breakdown showed RPC-DataGen at 5.9% of total CPU, dominated by `SilesiaResponseGenerator::generateRankingResponse` (~4.65% leaf self-time) plus the matching `~RankingObject` / `~RankingStory` destructor chain (~1.23%) — every request constructs a fresh 2000-story RankingResponse, serializes it, then immediately destroys it. Total chain is ~5.5% of CPU that has no production analog (this is test-only data fabrication).

Switch to a precomputed template pool: build `kResponseTemplatePoolSize=64` RankingResponse instances at server startup (right after `g_silesia_response_gen` is wired), and each request hands a `RankingResponse&` to the serializer by rotating through the pool. Only the `queryID` field is mutated per-request (the one field a real aggregator varies). 64 slots are enough to keep any single template from getting cache-hot enough to skew the Serialization/Compression bucket measurements.

Three callsites updated:
- `LeafNodeRank.cc:1216-1224` — DLRM async request handler Stage-5 (the dominant hot path)
- `LeafNodeRank.cc:1419-1425` — second async response handler (sister to the above)
- `LeafNodeRank.cc:1511-1513` — sync legacy ranking handler

Init point added at `LeafNodeRank.cc:~2346` after the Silesia loader is up, so the templates correctly use Silesia-derived payload bytes when `--silesia_dir` is set (and fall back to the xor128 RNG path otherwise).

Wire-shape is identical to today: same story count, same object count per story, same payload-map sizes, same action counts. The 64-slot rotation prevents any single template from being cache-hot enough to distort the Serialization or Compression CPU buckets.

Reviewed By: YifanYuan3

Differential Revision: D107327607

Author: excelle08

packages/feedsim/third_party/src/workloads/ranking/LeafNodeRank.cc

@@ -283,6 +283,37 @@ static ranking::RankingResponse generateResponse(int num_objects) {
   return ranking::generators::generateRandomRankingResponse(num_objects);
 }
 
+// Pre-built response template pool. Constructed once during server init
+// after --num_objects is parsed. Each request grabs one by index, mutates
+// only queryID (the only field a real aggregator would vary per-request),
+// then serializes. Eliminates ~5% of total CPU previously spent in
+// SilesiaResponseGenerator::generateRankingResponse + ~RankingObject
+// destructor chain (was RPC-DataGen=6.3% in the t31v5 profile; this drops
+// it to <0.5%).
+//
+// The 64-slot rotation prevents a single template from getting cache-hot
+// enough to skew the Serialization/Compression bucket measurements. The
+// queryID mutation is racy across threads but acceptable: we only need
+// wire-shape stability, not value correctness for the benchmark.
+static constexpr size_t kResponseTemplatePoolSize = 64;
+static std::vector<ranking::RankingResponse> g_response_templates;
+static std::atomic<size_t> g_response_template_idx{0};
+
+static void initResponseTemplatePool(int num_objects) {
+  g_response_templates.reserve(kResponseTemplatePoolSize);
+  for (size_t i = 0; i < kResponseTemplatePoolSize; ++i) {
+    g_response_templates.push_back(generateResponse(num_objects));
+  }
+}
+
+static ranking::RankingResponse& pickResponseTemplate() {
+  size_t idx = g_response_template_idx.fetch_add(
+      1, std::memory_order_relaxed) % kResponseTemplatePoolSize;
+  ranking::RankingResponse& resp = g_response_templates[idx];
+  resp.queryID() = static_cast<int64_t>(idx);
+  return resp;
+}
+
 #ifdef FEEDSIM_USE_DLRM
 void ThreadStartup(
     int thread_id,
@@ -1202,13 +1233,14 @@ void AsyncPageRankRequestHandler(
               return total;
             });
       })
-      .thenValue([context_ptr, num_objects](int /*final_result*/) {
-        // Stage 5: Generate and send response.
-        ranking::RankingResponse resp = generateResponse(num_objects);
-
+      .thenValue([context_ptr](int /*final_result*/) {
+        // Stage 5: serialize a pre-built response template (see
+        // initResponseTemplatePool above). Replaces per-request response
+        // construction + destruction (was ~5% of total CPU under the
+        // RPC-DataGen category in the t31v5 profile).
+        ranking::RankingResponse& resp = pickResponseTemplate();
         auto payloadiobufq = serializePayload(resp);
         auto buf = payloadiobufq.move();
-
         context_ptr->sendResponse(buf->data(), buf->length());
       })
       .thenError(folly::tag_t<std::exception>{}, [context_ptr](const std::exception& e) {
@@ -1404,8 +1436,10 @@ void DLRMRequestHandler(
               return total;
             });
       })
-      .thenValue([context_ptr, num_objects](int /*final_result*/) {
-        auto resp = generateResponse(num_objects);
+      .thenValue([context_ptr](int /*final_result*/) {
+        // Use the pre-built template pool to avoid the ~5% RPC-DataGen
+        // construction + destructor cost per request.
+        ranking::RankingResponse& resp = pickResponseTemplate();
         folly::IOBufQueue bufq;
         apache::thrift::CompactSerializer::serialize(resp, &bufq);
         auto buf = bufq.move();
@@ -1495,9 +1529,8 @@ void PageRankRequestHandler(
   auto chaseFs = folly::collect(chaseFutures).get();
   int chaseResult = std::accumulate(chaseFs.begin(), chaseFs.end(), 0);
 
-  // Generate a response
-  auto r = generateResponse(args.num_objects_arg);
-  ranking::RankingResponse resp = r;
+  // Use the pre-built template pool to avoid the ~5% RPC-DataGen cost.
+  ranking::RankingResponse& resp = pickResponseTemplate();
 
   // Serialize into FBThrift
   auto payloadiobufq = serializePayload(resp);
@@ -2331,6 +2364,15 @@ int main(int argc, char** argv) {
               << std::endl;
   }
 
+  // Pre-build the response template pool now that generateResponse is wired
+  // (either Silesia or xor128). Each handler grabs a template by index +
+  // mutates queryID, eliminating the per-request construct+destruct chain
+  // that was 6.3% of total CPU (RPC-DataGen) in the t31v5 profile.
+  initResponseTemplatePool(args.num_objects_arg);
+  std::cout << "Response template pool: " << kResponseTemplatePoolSize
+            << " pre-built " << args.num_objects_arg
+            << "-object RankingResponse instances" << std::endl;
+
   // Phase 5: load rpc_dist.json and instantiate the RpcDistRegistry. When
   // --rpc_dist_path is empty (default) OR --use_legacy_sleep is set, the
   // legacy folly::futures::sleep I/O simulation is used everywhere --