Llama.cpp endpoints showing as offline after recent update

[sniksz1]

Just pulled the latest changes today and noticed my local llama.cpp endpoint was showing up as completely offline in the model selection UI. Everywhere else it was showing as online and working, even in the native llama.ui web interface. Everything worked perfectly before the git pull.

Chatting actually worked fine, mostly, with occasional break ups that caused restarting/looping messages. (caused by triggering endpoint offline error and canceling prompts)

I ended up fixing it by changing a few code lines in model_routes.py. From line 755 specifically.

For anyone having the same issue, here is what I changed:

   try:
        r = httpx.get(base, headers=headers, timeout=timeout, verify=llm_verify())
        return _result_from_response(r)
    except Exception as e:
        last_error = str(e)[:120]

    return {"reachable": False, "status_code": None, "error": last_error}

And this is what I changed it to:

  try:
        r = httpx.get(base.rstrip("/") + "/models", headers=headers, timeout=timeout, verify=llm_verify())
        result = _result_from_response(r)
        if result["reachable"]:
            return result
    except Exception:
        pass

    try:
        r = httpx.get(base, headers=headers, timeout=timeout, verify=llm_verify())
        return _result_from_response(r)
    except Exception as e:
        last_error = str(e)[:120]

    return {"reachable": False, "status_code": None, "error": last_error}

[jaimebarrancos]

This works great. Thanks!