fix(epp): move TokenizedPrompt to request attribute store (#1375)

[yuanhuizang-cloud]

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

Moves TokenizedRequest out of InferenceRequestBody and stores it in the per-request attribute store under TokenizedPromptDataKey.

TokenizedPromptDataKey was already used to declare producer/consumer dependencies, while the actual tokenized prompt was still stored and accessed through InferenceRequestBody.TokenizedRequest. This made the declared dependency graph and the runtime data flow use two different mechanisms for the same data.

This PR aligns them by:

• publishing tokenized prompts from token-producer through the request attribute store;
• migrating request-control, scheduling, flow-control, and observability consumers to read the tokenized prompt from request attributes;
• removing InferenceRequestBody.TokenizedRequest;
• keeping protocol-level pre-tokenized input in the parsed request body and letting token-producer normalize it into the shared TokenizedRequest;
• updating the vLLM gRPC parser to use the existing GenerateRequest representation for pre-tokenized generate input and multimodal metadata;
• using the canonical TokenizedPromptDataKey consistently for production and consumption;
• updating affected tests and fixtures to use the new request-attribute contract.

This removes the previous hidden body mutation path and avoids maintaining multiple sources of truth for tokenized prompt state.

Which issue(s) this PR fixes:

Fixes #1375

Release note (write NONE if no user-facing change):

NONE

[github-actions]

🚨 Unsigned commits detected! Please sign your commits.

For instructions on how to set up GPG/SSH signing and verify your commits, please see GitHub Documentation.

[ahg-g]

I am not confident that this is strictly necessary and the value it adds, the change is significant.

[yuanhuizang-cloud]

Thanks for the feedback. I agree the footprint is broad, so the value should justify the cost.

The main value I see is aligning the framework contract with runtime behavior: TokenizedPrompt is already produced once, consumed by multiple plugins, and declared through TokenizedPromptDataKey, but the actual value still flows through a mutable field on InferenceRequestBody.

The footprint is broad, but the semantic change is relatively narrow: most production changes are consumers switching from Body.TokenizedRequest to the request attribute, while a significant portion of the diff is test fixture migration. Behavior remains the same, including pre-tokenized inputs.

I agree the size of the migration is the main trade-off here. If there is a smaller way to achieve the same alignment, I'm happy to adjust the implementation.