Skip to content

fix: catch std::out_of_range when parsing inference header length in HTTP frontends - #8925

Open
akhilraj9 wants to merge 5 commits into
mainfrom
asaraswathi/tri-1636-fix-inference-header-length-out-of-range
Open

fix: catch std::out_of_range when parsing inference header length in HTTP frontends#8925
akhilraj9 wants to merge 5 commits into
mainfrom
asaraswathi/tri-1636-fix-inference-header-length-out-of-range

Conversation

@akhilraj9

Copy link
Copy Markdown
Contributor

What does the PR do?

GetInferenceHeaderLength parses the Inference-Header-Content-Length request
header (and the SageMaker/Vertex json-header-size MIME parameter) with
std::stoi. std::stoi throws std::out_of_range for values outside the
int range, but the three frontend implementations only caught
std::invalid_argument. An uncaught std::out_of_range unwinds out of the
handler into the libevhtp C event loop, which has no C++ landing pad, so the
runtime calls std::terminate() and the server process aborts.

This applies the same catch (const std::out_of_range&) pattern already used by
the sibling GetContentLength in http_server.cc, returning a 400 instead of
terminating. Fixed in all three HTTP frontends:

  • HTTPAPIServer::GetInferenceHeaderLength (src/http_server.cc)
  • SagemakerAPIServer::GetInferenceHeaderLength + its Content-Length parse (src/sagemaker_server.cc)
  • VertexAiAPIServer::GetInferenceHeaderLength (src/vertex_ai_server.cc)

Commit Type:

  • fix

Where should the reviewer start?

src/http_server.cc — compare the new catch (const std::out_of_range& oor)
block in GetInferenceHeaderLength against the existing one in the sibling
GetContentLength (same file); they now match.

Test plan:

Added one regression test per frontend that sends a header value of
99999999999 (exceeds INT_MAX) and asserts the server returns 400 and stays
live:

  • qa/L0_http/http_test.py::test_inference_header_content_length_out_of_range
  • qa/L0_sagemaker/sagemaker_test.py::test_malformed_binary_header_out_of_range
  • qa/L0_vertex_ai/vertex_ai_test.py::test_malformed_binary_header_out_of_range

Expected unit-test counts in the corresponding test.sh harnesses were bumped
(17 / 10 / 9). L0_http, L0_sagemaker and L0_vertex_ai all pass.

  • CI Pipeline ID: 61802272

Caveats:

None. Change is limited to exception handling on an existing error path.

Background

Regression/incomplete-fix residual of CVE-2026-24210: the out_of_range catch
was added to GetContentLength but not to the three GetInferenceHeaderLength
variants when std::atoi was replaced with std::stoi. Reported via internal
security tracking.

Related Issues:

  • Relates to internal tracking DLIS-8702 / NVBug 6522755 / TRI-1636

…erver crash (TRI-1636)

std::stoi throws std::out_of_range for values outside [INT_MIN, INT_MAX] but
GetInferenceHeaderLength in all three HTTP frontends only caught invalid_argument,
letting an oversized header value escape into libevhtp's C event loop and crash the server.
…(TRI-1636)

Add test_inference_header_content_length_out_of_range to L0_http,
test_malformed_binary_header_out_of_range to L0_sagemaker and L0_vertex_ai.
Each test sends a header value of 99999999999 (exceeds INT_MAX) and asserts
the server returns 400 and remains live — verifying the std::out_of_range
catch added to GetInferenceHeaderLength prevents the server crash.
The L0_http, L0_sagemaker and L0_vertex_ai harnesses assert an exact number of
executed unit tests. Adding one out_of_range regression test to each file raised
the counts to 17/10/9, so update EXPECTED_NUM_TESTS / UNIT_TEST_COUNT accordingly.
@greptile-apps

greptile-apps Bot commented Aug 10, 2026

Copy link
Copy Markdown

Greptile Summary

The PR prevents oversized inference-header length values from escaping HTTP handlers as uncaught std::out_of_range exceptions.

  • Adds invalid-argument handling for out-of-range numeric values in the HTTP, SageMaker, and Vertex AI frontends.
  • Adds frontend regression tests that expect HTTP 400 responses and verify server liveness.
  • Updates the corresponding test-count expectations.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
src/http_server.cc Converts out-of-range Inference-Header-Content-Length parsing failures into invalid-argument responses.
src/sagemaker_server.cc Handles out-of-range MIME header-size and Content-Length values without allowing exceptions to escape the request handler.
src/vertex_ai_server.cc Converts out-of-range Vertex AI binary-header size parsing failures into invalid-argument responses.
qa/L0_http/http_test.py Adds regression coverage for HTTP 400 handling and server liveness after an oversized inference-header length.
qa/L0_sagemaker/sagemaker_test.py Adds SageMaker regression coverage for oversized binary-header sizes and subsequent liveness.
qa/L0_vertex_ai/vertex_ai_test.py Adds Vertex AI regression coverage for oversized binary-header sizes and subsequent liveness.

Reviews (2): Last reviewed commit: "ci: satisfy pre-commit hooks (flake8 E40..." | Re-trigger Greptile

@akhilraj9
akhilraj9 requested review from pskiran1, whoisj and yinggeh and removed request for whoisj and yinggeh August 10, 2026 10:39
pskiran1
pskiran1 previously approved these changes Aug 10, 2026

@pskiran1 pskiran1 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@pskiran1

Copy link
Copy Markdown
Member

@akhilraj9, please fix pre-commit errors

- Add file-level "# flake8: noqa: E402" to the three test files (imports
  follow the sys.path.append("../common") adjustment), matching the existing
  convention in L0_lifecycle/lifecycle_test.py.
- clang-format the out_of_range error string in http_server.cc to the 80-col style.
- Bump copyright year to 2026 on sagemaker_test.py and vertex_ai_test.py.
@yinggeh

yinggeh commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

@pskiran1 Do we have a pipeline running?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants