Possible to use remote Ollama install?

[ksolomon75]

Describe your question

I have an Ollama instance running on a remote server (multiple GPUs, lots of memory, etc). I have an exposed URL that connects as it should, and I can use it with other tools. I've added the link in the settings for ClassifAI, but I never get a list of models, so I'm stuck at that point. Am I missing something?

Code of Conduct

• I agree to follow this project's Code of Conduct

[dkotter]

In theory that all should work, noting I've only ever tested with a locally hosted version of Ollama.

When we validate the connection, we make a request to the models URL to see which models are available. This API can be seen here: https://github.com/ollama/ollama/blob/main/docs/api.md#list-local-models

So if you've entered a custom URL, it will use that as the base and make a request to URL/api/tags. If you make a request to that directly, does that work? I'm wondering if that API isn't available if run externally.

[ksolomon75]

It returns the model list as expected...

[ksolomon75]

Interesting. After further playing around with it last night, it appears that upon initial addition if you include the trailing slash, it doesn't work. But any use past the initial setup, the slash needs to be there...

[dkotter]

upon initial addition if you include the trailing slash, it doesn't work. But any use past the initial setup, the slash needs to be there

Interesting, is this in reference to the Endpoint URL added in settings? Or is this the models URL we make a request to? In theory, a trailing slash shouldn't make a difference as we run trailingslashit on the Endpoint URL, which should add in a trailing slash if that doesn't exist.

Also worth noting (and this is a known issue we want to fix) once you configure the Provider (basically just adding in the Endpoint URL), you need to save the settings and then refresh the page to see the models list.

[ksolomon75]

That's in relation to the Endpoint URL in settings. Just to be sure, I added the plugin to a second testing site, and including the trailing slash made it fail each time, but adding the URL without the slash resulted in it working every time. I can verify that the plugin does indeed work well talking to a remote Ollama install though.

And yeah, I assumed that was the case (having to save the settings before the models show up), since it didn't appear that things were happening via AJAX or anything like that.