feat: better prometheus buckets for batch_size, isl, and input length

[michaelfeil]

What does this PR do?

I realized the prometheus metrics do some short-cuts where it could be helpful to have more information. Right now, buckets are bit generic and would benefit from some exta buckets. Most of the time batch size, and tokens are small, tokens by default often 8k or 16k, except for qwen models also 40k. Then for input-sequence-length, its in the same bucket. Potentially people release longer models, so the upper buckets are ok. I think this could help getting more visibility in load-distribution.

1. te_batch_next_size: 1, 2, 3, ..., 32, 64, 128, 256, 512, 1024, 2048, 4096
2. te_batch_next_tokens: Half-steps up to 32k, then powers of 2
   • 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 768, 1024, 1536, 2048, 3072, 4096, 6144, 8192, 12288, 16384, 24576, 32768, 65536, 131072, ...
3. te_request_input_length: Same as tokens, but filtered by max_input_length

For batch_size:

• 1, 2, 3, ..., 32 (integers)
• 64, 128, 256, 512, 1024, 2048, 4096 (powers of 2 beyond 32)
  For tokens (half-steps to 32k):
• Powers of 2: 1, 2, 4, 8, 16, 32, 64, 128, 256, 512
• Then half-steps: 768 (512+256), 1024, 1536 (1024+512), 2048, 3072 (2048+1024), 4096, 6144 (4096+2048), 8192, 12288 (8192+4096), 16384, 24576 (16384+8192), 32768
• Then powers of 2 beyond: 65536, 131072, 262144, 524288, 1048576

There is a limit on total number of metrics (times labels and buckets) which is around 750 for Baseten etc. I think we are still far below it.

Fixes # (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the documentation guidelines.
• Did you write any new necessary tests? If applicable, did you include or update the insta snapshots?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.