feat(candle-nn): QuantizedKvCache — INT8 KV cache with attention sinks (TurboQuant)#3577
Open
aryanputta wants to merge 2 commits into
Open
feat(candle-nn): QuantizedKvCache — INT8 KV cache with attention sinks (TurboQuant)#3577aryanputta wants to merge 2 commits into
aryanputta wants to merge 2 commits into
