Skip to content

Commit 57d8b53

Browse files
excelle08meta-codesync[bot]
authored andcommitted
Tune feedsim_autoscale_dlrm_mini toward production CPU profile (1800 stories, 8 inferences) (#715)
Summary: Pull Request resolved: #715 The first sweep with async feature extraction (D...c58bc574d94c stack tip prior) confirmed the runFlatExtractors fanout reaches GlobalCPUThread correctly, but the dlrm_mini defaults left two large gaps vs production multifeed_aggregator's strobelight breakdown: | Category | Bench % non-idle (BGM q150 s05) | Prod target | |-------------------|---------------------------------|-------------| | DLRM-Inference | 61.6% | 7-13% | | FeatureExtraction | 1.9% | 30-35% | Two knob changes get both metrics moving toward prod: 1. dlrm_inferences=64 -> 8. Shrinks the per-request DLRM matmul work 8x. Should drop the DLRM-Inference share from ~55-62% toward prod's 7-13% Ranking-Prediction share. 2. num_stories=100 -> 1800. With extractors_per_story=50 unchanged, that's 90K extractor calls/req (was 5K). Should push FeatureExtraction CPU share from 1.7-1.9% toward prod's 28-32% Ranking-FeatureExtraction share. The system has plenty of CPU headroom (BGM 80% idle past saturation, CPL 60% idle), so adding 90K extractor calls/req shouldn't break throughput. Both knobs only affect feedsim_autoscale_dlrm_mini. The full feedsim_autoscale_dlrm job keeps its existing config. Knobs remain overridable per-run via -i. Reviewed By: YifanYuan3 Differential Revision: D105119226
1 parent 8058693 commit 57d8b53

1 file changed

Lines changed: 7 additions & 2 deletions

File tree

benchpress/config/jobs.yml

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -828,15 +828,20 @@
828828
- 'dlrm_model=models/dlrm_small.pt'
829829
- 'dlrm_batch_size=256'
830830
- 'dlrm_threads=1'
831-
- 'dlrm_inferences=64'
831+
# 8 inferences/req brings DLRM-Inference CPU share from 55-62% down
832+
# toward prod multifeed_aggregator's Ranking-Prediction share of 7-13%.
833+
- 'dlrm_inferences=8'
832834
- 'client_side_features=0'
833835
- 'client_batch_size=256'
834836
- 'client_inferences=64'
835837
- 'client_feature_seed=42'
836838
- 'client_num_dense=13'
837839
- 'client_num_sparse=26'
838840
- 'feature_complexity=5'
839-
- 'num_stories=100'
841+
# 1800 stories x 50 extractors = 90K extractor calls/req. Tuned to
842+
# push FeatureExtraction CPU share into prod's 30-35% band; 5K
843+
# calls/req previously gave only 1.7-1.9%.
844+
- 'num_stories=1800'
840845
- 'extractors_per_story=50'
841846
- 'fixed_qps=100000'
842847
- 'fixed_qps_duration=10'

0 commit comments

Comments
 (0)