Skip to content

Support dense Qwen3 generation in EMLXAxon #119

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
hfiguera wants to merge 16 commits into
base: main
Choose a base branch
from
Open

Support dense Qwen3 generation in EMLXAxon #119

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
d730eb2
feat: add dense qwen3 generation to EMLXAxon
hfiguera Jun 20, 2026
49fb914
Update emlx_axon/lib/emlx_axon/qwen3/generate.ex
hfiguera Jun 22, 2026
c2dec0c
Update emlx_axon/lib/emlx_axon/qwen3/generate.ex
hfiguera Jun 22, 2026
b6843a9
Update emlx_axon/lib/emlx_axon/qwen3/generate.ex
hfiguera Jun 22, 2026
e5bfe74
Update emlx_axon/lib/emlx_axon/qwen3/generate.ex
hfiguera Jun 22, 2026
a4c287c
Address text generation style feedback
hfiguera Jun 22, 2026
de701ea
Use microsecond timing for Qwen3 profiling
hfiguera Jun 22, 2026
6935cc7
Update emlx_axon/lib/emlx_axon/qwen3/dense_loader.ex
hfiguera Jun 23, 2026
597753e
Simplify Qwen3 dense parameter lookup
hfiguera Jun 23, 2026
ef75529
Rename Qwen3 quantized attention path
hfiguera Jun 23, 2026
b7ddac5
Leave dense Qwen3 state evaluation lazy
hfiguera Jun 23, 2026
685ec29
Keep Qwen3 greedy sampler generic
hfiguera Jun 23, 2026
92c279a
Isolate Qwen3 native accelerators
hfiguera Jun 23, 2026
3dba672
Inline Qwen3 native error returns
hfiguera Jun 23, 2026
08f66e7
Move Qwen3 native code under emlx_fast
hfiguera Jun 23, 2026
51c2393
Move memory include to Qwen3 source
hfiguera Jun 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions emlx/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,8 @@ endif
MAKE_JOBS ?= $(MAKE_DEFAULT_JOBS)

# Source files
SOURCES = c_src/emlx_nif.cpp c_src/emlx_fast.cpp
HEADERS = c_src/nx_nif_utils.hpp c_src/emlx_worker.hpp c_src/emlx_async.hpp c_src/emlx_nif_shared.hpp
SOURCES = c_src/emlx_nif.cpp c_src/emlx_fast.cpp c_src/emlx_fast/qwen3.cpp
HEADERS = c_src/nx_nif_utils.hpp c_src/emlx_worker.hpp c_src/emlx_async.hpp c_src/emlx_nif_shared.hpp c_src/emlx_fast/qwen3.hpp
OBJECTS = $(patsubst c_src/%.cpp,$(call esc,$(BUILD_DIR))/%.o,$(SOURCES))

# Main targets
Expand All @@ -59,7 +59,7 @@ $(call esc,$(PRIV_DIR)):
@ mkdir -p "$(PRIV_DIR)"

$(call esc,$(BUILD_DIR))/%.o: c_src/%.cpp $(HEADERS)
@ mkdir -p "$(BUILD_DIR)"
@ mkdir -p "$(dir $@)"
$(CXX) $(CFLAGS) -c "$<" -o "$@"

$(call esc,$(MLX_SRC_ARCHIVE)):
Expand Down
1 change: 1 addition & 0 deletions emlx/c_src/emlx_fast.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -640,3 +640,4 @@ NIF(kv_cache_sdpa_update) {
CATCH()
}
ASYNC_NIF(kv_cache_sdpa_update)

Loading
Loading