Skip to content

Don't schedule prefill unless there is enough kv cache left to fulfil decode stage#5467

Open
AleksKnezevic wants to merge 3 commits into
mainfrom
aknezevic/prefill_schedule
Open

Don't schedule prefill unless there is enough kv cache left to fulfil decode stage#5467
AleksKnezevic wants to merge 3 commits into
mainfrom
aknezevic/prefill_schedule

Address feedback

11f9734
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar