Skip to content

Commit fa15196

Browse files
committed
fix: allow known_usecases to override backend heuristic checks for rerank
This fix addresses issue mudler#8658 where users setting known_usecases: [rerank] in their model config still get 'This server does not support reranking' error because GuessUsecases() overrides the explicit known_usecases setting. The issue is that when a model config has known_usecases set (e.g., 'rerank'), the HasUsecases() method first checks if KnownUsecases is set and matches, but if it doesn't match exactly, it falls back to GuessUsecases() which has hardcoded backend name checks (e.g., requires backend == 'rerankers'). This change makes GuessUsecases() return false early if KnownUsecases is already set, allowing the explicit user configuration to take precedence over the heuristic backend name checks.
1 parent 00abf1b commit fa15196

1 file changed

Lines changed: 5 additions & 0 deletions

File tree

core/config/model_config.go

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -609,6 +609,11 @@ func (c *ModelConfig) HasUsecases(u ModelConfigUsecase) bool {
609609
// In its current state, this function should ideally check for properties of the config like templates, rather than the direct backend name checks for the lower half.
610610
// This avoids the maintenance burden of updating this list for each new backend - but unfortunately, that's the best option for some services currently.
611611
func (c *ModelConfig) GuessUsecases(u ModelConfigUsecase) bool {
612+
// If the user has explicitly set known_usecases in the config, trust that instead of using heuristics
613+
if c.KnownUsecases != nil {
614+
return false
615+
}
616+
612617
if (u & FLAG_CHAT) == FLAG_CHAT {
613618
if c.TemplateConfig.Chat == "" && c.TemplateConfig.ChatMessage == "" && !c.TemplateConfig.UseTokenizerTemplate {
614619
return false

0 commit comments

Comments
 (0)