AI model only responds with "The"

[ariusl]

As the title states and image shows I typed hello and it only responds with the. I dont know how to fix please help.

[CoolJohn-lab]

Need more details. What OS? What's your system spec, how are you running Qwen?

[nhatquangdinh0000-hub]

I have the same problem. I use Window 10 and when I say smt like "hello", the reply would end with just 1 word.

This is my spec.

[CoolJohn-lab]

You don't mention any GPU in your spec so I assume you don't have one?

In which case this simply will not work.

[jorgeporragas]

Ran into the exact same issue with Qwen3.6. I've been checking memory and processing with free -h and ollama ps constantly and it's not a memory issue.

Sometimes it replies a simple "The", but there have been times where it's actually been able to reply appropriately.

I've had better results with smaller models like Qwen3:14B but the problem with those is that they don't understand the functions that come with Odysseus to manage the calendar or create memories. I can ask it something simple like "Check my calendar for events scheduled for tomorrow" and it will go crazy and enter a loop where it just answers itself over and over that it cannot view my calendar.

Qwen3.6:27B seems to be better at understanding what I want but then it abruptly stops answering or just replies "The" like that Spongebob episode.

For reference, I'm running everything on Fedora 44. My specs are R5 5600X, RX 9600XT 16GB and 32 of RAM.

[CoolJohn-lab]

Seems to be common problem when trying to host models on a system with no GPU.

This is possible, but totally impractical. It will just die on its ass the whole time and write the word "The".

You say you have a pretty powerful AMD card. How are you running the model on it?

[jorgeporragas]

I'm running it using Ollama.

Ollama reports the model is mostly running on the GPU:
NAME ID SIZE PROCESSOR CONTEXT
qwen3.6:27b a50eda8ed977 17 GB 10%/90% CPU/GPU 4096
The issue I'm seeing is that responses sometimes stop abruptly. Examples:
Tool task starts correctly, then cuts off mid-answer.
One response was literally:

The
and then stopped.

Maybe it's a Qwen family or Ollama issue?

[CoolJohn-lab]

Your issue is trying to run a 17GB model on a 16GB card. That means 100% of your context is in RAM and none in VRAM. Although it "works", that setup is effectively broken. You've managed to make it work at all by keeping context down to 4K tokens, but that's not enough context to do anything really, you need at least 32K ideally 128K or more.

You need to aim for about 60/40 model and context in VRAM. So in your case a 8-10GB model leaving 6-8GB free for context would work well. Check out some 3B-6B models with Q8, or 27B at Q4 and exotic MoE configurations. You will find something that works.

[jorgeporragas]

Thanks! Another user pointed out that you can raise the context window my modding the ollama modelfile, which I didn't know. And yes, I actually started with the 6B model and also tried a 14B model because I kept having these issues and I thought it was a reasoning issue, not context. I'll run some tests with a smaller model and a larger context window.

[jonas-df]

I had this issue and solved it by setting the model context length using the parameter n_ctx in a custom Modelfile.

Ollama defaults to 4096 in context length for whatever reason and odysseus UI does not allow you to change this. That is not enough context to run the agent mode. If you flip the mode to chat it should work for short replies. But better yet to make a separate model file and set the context there. Why ollama defaults to on all the models from the cookbook must be a bug I guess.

From the terminal

1. ollama show --modelfile [name_of_your_model] > Modelfile
2. Edit model file and add a line at the top that says
   PARAMETER num_ctx 32768 (or whatever you want to set the context to)
3. save file
4. run ollama create [new_model_name] -f Modelfile
5. run ollama [new_model_name]

It should start with more context and you can get the agent mode to work.

[jorgeporragas]

That makes sense! I saw Felix's post about the system prompt using 10K tokens on it's own and it immediately raised a flag, I just couldn't find an option to increase the agent's context other than on the Deep Research settings.

I'll give this a try, thank you!

[CoolJohn-lab]

Yeah that's right, Ollama default context limit is too low for any practical purpose, you need to increase it.

It's not a bug that Ollama consumers aren't allowed to modify the max context window size, it's a feature of Ollama, it's meant to prevent crappy tools and harnesses ramping the context window to max and burning through your token budget.

[jonas-df]

Ollama setting a low context window as default makes a lot of sense then, thanks for pointing that out.

Being able to set it from the cookbook when serving a model would be great.