Add 5 prod-shaped thrift structs and register new request type IDs (facebookresearch#721)

Summary:
Pull Request resolved: facebookresearch#721

Phase 4 programmer-B: introduce production-shaped multifeed aggregator thrift schema and dispatch IDs so the FeedSim leaf node can be exercised by per-method driver traffic in Phase 6. Builds on Phase 4-A (`11e9bc9a3431` — pool rename to ThriftSrv.IO/SREventBase/RANKER/GlobalCPUThread) and Phase 5-A (`31948cd0d579` — mock_services binary).

Three changes, all additive (no existing struct/handler is removed — Phase 6 deletes RankingRequest/RankingResponse and the legacy kPageRank/kDLRM type IDs):

1. `if/ranking.thrift` — five new request/response struct pairs sized to the p50 wire targets from `~/feedsim_v2/profiles/rpc_dist.json`:
   - CreateAndPrimeSessionRequest/Response (379 B / 44 B)
   - GetStoriesRequest/Response (2.13 MB / 171 KB) — also adds shared helpers GetStoriesResponseStats and RankedStoryInfo
   - GetAllStoriesRequest/Response (55 B / 1.47 MB)
   - StreamDataRequest/Response (58 KB / 4 B) plus StreamingUseCase enum
   - StreamIfrPriorityRankingRequest/Response (949 KB / 4 B)

   Each request struct mirrors prod field counts and types (including primitive vs container vs binary) per `~/feedsim_v2/docs/phase4_researcher_notes.md` section 3, so CompactProtocol serialization cost is realistic. Bulk wire size lives in named `binary` fields (e.g. `settings_compressed`, `serialized_payload`, `ifr_objects_serialized`) that the Phase 6 driver populates by sampling from the percentile table.

2. `RequestTypes.h` — five new uint32_t constants `0x10..0x14` for the new methods. Existing `kPageRankRequestType` (0x00) and `kDLRMRequestType` (0x01) stay so the in-flight stack keeps working.

3. `LeafNodeRank.cc` — five new shim handler functions and matching `registerQueryCallback` calls:
   - Heavy methods (`getStoriesUncompressed`, `getAllStories`) deserialize the new struct then route to the existing `DLRMRequestHandler`. Phase 4 CPU profile is unchanged for those.
   - Light methods (`createAndPrimeSession`, `streamData`, `streamIfrPriorityRanking`) deserialize, then send a small fixed-size response (44 B / 4 B / 4 B) without invoking `DLRMRequestHandler`. Production p50 latencies for these are 3-13 ms with 4-44 B responses, so attributing DLRM CPU to them in Phase 4 testing would distort the profile. Phase 6 replaces these shims with real per-method handlers (session bookkeeping, ack-only paths, IFR scoring).

Sizing methodology: targets are p50 wire sizes from `rpc_dist.json`. Computed sizes are CompactProtocol overhead (1 byte per short field tag, 2 bytes for tags >15, varint length + N bytes data for binary, ~1 byte stop) plus the binary field contents the driver supplies:

   | Method                        | Target p50 | Size source                                                          |
   |-------------------------------|-----------:|----------------------------------------------------------------------|
   | CreateAndPrimeSessionRequest  |      379 B | ~110 B field overhead + ~270 B `session_init_blob`                    |
   | CreateAndPrimeSessionResponse |       44 B | ~7 B field overhead + 32-char hex `session_id` (~36 B) + 4 B status   |
   | GetStoriesRequest             |   2.13 MB  | ~150 B fixed fields + 5 binary blobs (driver fills to ~2.07 MB total) |
   | GetStoriesResponse            |   171 KB   | ~50 B fixed + ~100 stories x ~1.5 KB story_payload + ~10 KB debug    |
   | GetAllStoriesRequest          |       55 B | 36 B session_id + 8 B query_id + ~10 B caller_id + ~6 B overhead     |
   | GetAllStoriesResponse         |   1.47 MB  | ~50 B fixed + ~500-1000 stories x ~1.5 KB + ~10 KB debug             |
   | StreamDataRequest             |    58 KB   | ~50 B fixed + driver-sampled `serialized_payload` (bimodal in prod)  |
   | StreamDataResponse            |        4 B | 1 B field header + 1 B i32 zigzag + 1 B stop = 3-4 B                 |
   | StreamIfrPriorityRankingReq   |   949 KB   | ~80 B fixed + driver-sampled `ifr_objects_serialized` etc.           |
   | StreamIfrPriorityRankingResp  |        4 B | identical encoding to StreamDataResponse                              |

Because the binary fields are sampled per-request from the percentile table (in Phase 6), every method can hit not just p50 but the entire prod distribution (p05/p25/p75/p95). The structs themselves carry no binary defaults.

Generated `gen-cpp2/ranking_types.h` is regenerated by CMake at fbpkg-install time; the checked-in copy predates `RankingRequest`/`DLRMFeatures`/`StoryBatch` and is also missing those, confirming it is rebuilt out-of-tree.

Reviewed By: charles-typ

Differential Revision: D103767023

Author: excelle08

packages/feedsim/third_party/src/workloads/ranking/LeafNodeRank.cc

@@ -1097,6 +1097,147 @@ void PageRankRequestHandler(
   context.sendResponse(buf->data(), buf->length());
 }
 
+// ============================================================================
+// Phase 4 shim handlers — exercise the new prod-shaped thrift schema and
+// dispatch path. Heavy methods (getStoriesUncompressed, getAllStories) route
+// to the existing DLRMRequestHandler so QPS/CPU profile is unchanged. Light
+// methods (createAndPrimeSession, streamData, streamIfrPriorityRanking) send
+// a small response without invoking DLRMRequestHandler — production p50
+// latency and response size for these are tiny (4-44 B, 3-13 ms) and we don't
+// want Phase 4 to attribute DLRM CPU to them. Phase 6 replaces these shims
+// with real per-method handlers.
+// ============================================================================
+
+namespace {
+
+// Helper: send a small fixed-size response after validating the inbound
+// schema. Used by the three "light" methods. The payload bytes are filled
+// with zeros — Phase 6 will populate real fields.
+template <typename ResponseT>
+void sendTinyResponse(feedsim::RequestContext& context, ResponseT&& response) {
+  folly::IOBufQueue queue;
+  apache::thrift::CompactSerializer::serialize(response, &queue);
+  auto buf = queue.move();
+  if (buf) {
+    // CompactSerializer may chain IOBufs; coalesce so sendResponse sees the
+    // full payload instead of just the head segment.
+    buf->coalesce();
+    context.sendResponse(buf->data(), buf->length());
+  } else {
+    context.sendResponse(nullptr, 0);
+  }
+}
+
+} // namespace
+
+void CreateAndPrimeSessionRequestHandler(
+    int /*thread_id*/,
+    feedsim::RequestContext& context,
+    std::vector<ThreadData>& /*thread_data*/) {
+  ranking::CreateAndPrimeSessionRequest typed_req;
+  try {
+    folly::IOBuf buf(
+        folly::IOBuf::WRAP_BUFFER, context.payload, context.payload_length);
+    apache::thrift::CompactSerializer::deserialize(&buf, typed_req);
+  } catch (const std::exception& e) {
+    std::cerr << "CreateAndPrimeSession: deserialize failed: " << e.what()
+              << std::endl;
+    context.sendResponse(nullptr, 0);
+    return;
+  }
+  ranking::CreateAndPrimeSessionResponse resp;
+  // 32-char hex placeholder; Phase 6 generates a real session_id.
+  resp.session_id() = "00000000000000000000000000000000";
+  resp.status_code() = 0;
+  sendTinyResponse(context, resp);
+}
+
+void GetStoriesUncompressedRequestHandler(
+    int thread_id,
+    feedsim::RequestContext& context,
+    std::vector<ThreadData>& thread_data) {
+  ranking::GetStoriesRequest typed_req;
+  try {
+    folly::IOBuf buf(
+        folly::IOBuf::WRAP_BUFFER, context.payload, context.payload_length);
+    apache::thrift::CompactSerializer::deserialize(&buf, typed_req);
+  } catch (const std::exception& e) {
+    std::cerr << "GetStoriesUncompressed: deserialize failed: " << e.what()
+              << std::endl;
+    context.sendResponse(nullptr, 0);
+    return;
+  }
+#ifdef FEEDSIM_USE_DLRM
+  DLRMRequestHandler(thread_id, context, thread_data);
+#else
+  (void)thread_id;
+  (void)thread_data;
+  context.sendResponse(nullptr, 0);
+#endif
+}
+
+void GetAllStoriesRequestHandler(
+    int thread_id,
+    feedsim::RequestContext& context,
+    std::vector<ThreadData>& thread_data) {
+  ranking::GetAllStoriesRequest typed_req;
+  try {
+    folly::IOBuf buf(
+        folly::IOBuf::WRAP_BUFFER, context.payload, context.payload_length);
+    apache::thrift::CompactSerializer::deserialize(&buf, typed_req);
+  } catch (const std::exception& e) {
+    std::cerr << "GetAllStories: deserialize failed: " << e.what() << std::endl;
+    context.sendResponse(nullptr, 0);
+    return;
+  }
+#ifdef FEEDSIM_USE_DLRM
+  DLRMRequestHandler(thread_id, context, thread_data);
+#else
+  (void)thread_id;
+  (void)thread_data;
+  context.sendResponse(nullptr, 0);
+#endif
+}
+
+void StreamDataRequestHandler(
+    int /*thread_id*/,
+    feedsim::RequestContext& context,
+    std::vector<ThreadData>& /*thread_data*/) {
+  ranking::StreamDataRequest typed_req;
+  try {
+    folly::IOBuf buf(
+        folly::IOBuf::WRAP_BUFFER, context.payload, context.payload_length);
+    apache::thrift::CompactSerializer::deserialize(&buf, typed_req);
+  } catch (const std::exception& e) {
+    std::cerr << "StreamData: deserialize failed: " << e.what() << std::endl;
+    context.sendResponse(nullptr, 0);
+    return;
+  }
+  ranking::StreamDataResponse resp;
+  resp.ack_code() = 0;
+  sendTinyResponse(context, resp);
+}
+
+void StreamIfrPriorityRankingRequestHandler(
+    int /*thread_id*/,
+    feedsim::RequestContext& context,
+    std::vector<ThreadData>& /*thread_data*/) {
+  ranking::StreamIfrPriorityRankingRequest typed_req;
+  try {
+    folly::IOBuf buf(
+        folly::IOBuf::WRAP_BUFFER, context.payload, context.payload_length);
+    apache::thrift::CompactSerializer::deserialize(&buf, typed_req);
+  } catch (const std::exception& e) {
+    std::cerr << "StreamIfrPriorityRanking: deserialize failed: " << e.what()
+              << std::endl;
+    context.sendResponse(nullptr, 0);
+    return;
+  }
+  ranking::StreamIfrPriorityRankingResponse resp;
+  resp.ack_code() = 0;
+  sendTinyResponse(context, resp);
+}
+
 int main(int argc, char** argv) {
   if (cmdline_parser(argc, argv, &args) != 0) {
     std::cerr << "cmdline_parser failed" << std::endl;
@@ -1427,6 +1568,43 @@ int main(int argc, char** argv) {
         return DLRMRequestHandler(thread_id, context, thread_data);
       });
 #endif
+
+  // Phase 4: register the 5 production-shaped inbound methods. Heavy methods
+  // (getStoriesUncompressed, getAllStories) route to DLRMRequestHandler so
+  // CPU profile is unchanged. Light methods (createAndPrimeSession,
+  // streamData, streamIfrPriorityRanking) send a tiny response to match
+  // prod p50 (4-44 B, 3-13 ms latency).
+  std::cout << "Registering Phase 4 prod-shaped inbound method handlers"
+            << std::endl;
+  server.registerQueryCallback(
+      ranking::kCreateAndPrimeSessionRequestType,
+      [&thread_data](int thread_id, feedsim::RequestContext& context) {
+        return CreateAndPrimeSessionRequestHandler(
+            thread_id, context, thread_data);
+      });
+  server.registerQueryCallback(
+      ranking::kGetStoriesUncompressedRequestType,
+      [&thread_data](int thread_id, feedsim::RequestContext& context) {
+        return GetStoriesUncompressedRequestHandler(
+            thread_id, context, thread_data);
+      });
+  server.registerQueryCallback(
+      ranking::kGetAllStoriesRequestType,
+      [&thread_data](int thread_id, feedsim::RequestContext& context) {
+        return GetAllStoriesRequestHandler(thread_id, context, thread_data);
+      });
+  server.registerQueryCallback(
+      ranking::kStreamDataRequestType,
+      [&thread_data](int thread_id, feedsim::RequestContext& context) {
+        return StreamDataRequestHandler(thread_id, context, thread_data);
+      });
+  server.registerQueryCallback(
+      ranking::kStreamIfrPriorityRankingRequestType,
+      [&thread_data](int thread_id, feedsim::RequestContext& context) {
+        return StreamIfrPriorityRankingRequestHandler(
+            thread_id, context, thread_data);
+      });
+
   server.setNumThreads(args.threads_arg);
   server.setThreadPinning(args.noaffinity_given == 0u);
   server.setThreadLoadBalancing(args.noloadbalance_given == 0u);

packages/feedsim/third_party/src/workloads/ranking/RequestTypes.h

@@ -15,14 +15,27 @@
 #ifndef REQUEST_TYPES_H
 #define REQUEST_TYPES_H
 
+#include <cstdint>
+
 namespace ranking {
 
+// Legacy request types — Phase 6 deletes these.
 // Request type for PageRank/DLRM with server-side feature generation
 static const int kPageRankRequestType = 0;
 
 // Request type for DLRM with client-side feature generation (Phase 7)
 static const int kDLRMRequestType = 1;
 
+// Phase 4: production-shaped multifeed aggregator inbound methods. Type
+// IDs 0x10..0x14 are dispatched by FeedSimServer::registerQueryCallback
+// to the per-method shim handlers in LeafNodeRank.cc. See Phase 4
+// researcher notes section 4 for the dispatch design.
+constexpr uint32_t kCreateAndPrimeSessionRequestType = 0x10;
+constexpr uint32_t kGetStoriesUncompressedRequestType = 0x11;
+constexpr uint32_t kGetAllStoriesRequestType = 0x12;
+constexpr uint32_t kStreamDataRequestType = 0x13;
+constexpr uint32_t kStreamIfrPriorityRankingRequestType = 0x14;
+
 } // namespace ranking
 
 #endif // REQUEST_TYPES_H