Skip to content

Commit c993e8d

Browse files
authored
Update qlora_adapter.py
Added instructions to the header
1 parent d8e7c4b commit c993e8d

1 file changed

Lines changed: 6 additions & 0 deletions

File tree

plug_in_modules/qlora_adapter.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,9 @@
1+
# The bitsandbytes library has historically been Linux/CUDA only — Mac MPS support is improving, but not guaranteed on the M-series chip.
2+
# On a Mac Mini, test with pip install bitsandbytes and check if 4-bit quantization works on MPS.
3+
# If not, the fallback is 8-bit or straight float16 without quantization, which will use more RAM but will run.
4+
# The Ollama bridge at the bottom (export_to_ollama_modelfile, merge_adapter_weights) is the path to getting a fine-tuned adapter running locally — you train with HuggingFace, merge the LoRA weights back into the base model, then import to Ollama.
5+
# The CAIOS.txt system prompt gets baked into the Modelfile, so it loads automatically without needing to pass it every call.
6+
17
#V04082026
28
# =============================================================================
39
# CAIOS — QLoRA Adapter Template

0 commit comments

Comments
 (0)