Does it support Arm machine ?

[mmaaxxii2018]

Thanks for the great work.
May i know if it runs on arm machine such as Jetson orin nano ?
regards,

[Mega4alik]

@mmaaxxii2018 thanks for the question. Right now some loading modules use kvikio that only works with Cuda, but gpt-oss and qwen3 do not. May I suggest trying it by simply setting o.device=(your device). It may work since it's built on transformers

[Mega4alik]

@mmaaxxii2018 kvikio and flash-attn are optional now and ollm should work on any device supported by HF transformers. Kindly update the ollm and try it!

[danieldietrich]

Hi @Mega4alik,in version 1.0.3, "flash-attn" #tested: 2.7.4.post1 is still part of the pyproject.toml dependencies. When I remove that line, ollm installs on my Arm machine (from source via uv pip install --no-build-isolation -e .).

[Mega4alik]

@danieldietrich thanks for sharing. It's very helpful!