feat(studio): add vLLM 4-bit export via Auto-Round (AWQ/GPTQ)#4837
feat(studio): add vLLM 4-bit export via Auto-Round (AWQ/GPTQ)#4837OnePunchMonk wants to merge 4 commits intounslothai:mainfrom
Conversation
- Add save_to_vllm_4bit() in unsloth/save.py using Intel Auto-Round - Add ExportVllm4bitRequest Pydantic schema with format/bits/group_size - Add export_vllm_4bit() to ExportBackend, Orchestrator, and Worker - Add POST /api/export/export/vllm4bit route - Add exportVllm4bit() frontend API client - Add vllm4bit export method + VLLM_QUANT_OPTIONS constants - Add 4-bit format picker UI in export-page.tsx - Fix missing ExportVllm4bitRequest import in routes/export.py - Fix missing VLLM_QUANT_OPTIONS import in export-page.tsx
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
OnePunchMonk
requested review from
Manan17,
Shine1i,
danielhanchen and
rolandtannous
as code owners
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dcf58747b6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| import type { TrainingMethod } from "@/types/training"; | ||
|
|
||
| export type ExportMethod = "merged" | "lora" | "gguf"; | ||
| export type ExportMethod = "merged" | "lora" | "gguf" | "vllm4bit"; |
There was a problem hiding this comment.
Q: Isn't this a subset of "merged" case?
|
@OnePunchMonk please rename methods, classes, etc to be more autoround specific, so for example ExportAutoRound4bit instead of ExportVllm4bit and so on |
OnePunchMonk
force-pushed
the
feat/vllm-4bit-export
branch
from
dcf5874 to
c6997bc
Compare
for more information, see https://pre-commit.ci
3 participants
Closes #4761