Search results

Effect of Inflight batching or Continous batching on the Results?

Status: Open.

#8 In Infini-AI-Lab/MagicDec;

· ekagra-ranjan opened on Aug 19, 2025

No Speedup across bsz: 1 --> 48

Status: Open.

#7 In Infini-AI-Lab/MagicDec;

· ZepinLi opened on Feb 26, 2025

Confusion about the results

Status: Open.

#6 In Infini-AI-Lab/MagicDec;

· wutong4012 opened on Nov 28, 2024

Question about the draft/target memory ratio

Status: Open.

#5 In Infini-AI-Lab/MagicDec;

· zhuzhui-2000 opened on Nov 20, 2024

Handling Prefill Lengths Exceeding 2k in TinyLlama_v1.1

Status: Open.

#4 In Infini-AI-Lab/MagicDec;

· bettybaii opened on Oct 24, 2024

Question regarding the termination condition.

Status: Open.

#3 In Infini-AI-Lab/MagicDec;

· saeyoon17 opened on Sep 12, 2024

Hanging on multiple GPU clusters

Status: Open.

#2 In Infini-AI-Lab/MagicDec;

· YJHMITWEB opened on Sep 6, 2024

KV Loading Time

Status: Open.

#1 In Infini-AI-Lab/MagicDec;

· wutong4012 opened on Sep 3, 2024