For 4 bit quantization, cogvlm-chat-v1.1, how much RAM and num of CPU do I need? #389
|
Tried with: |
Answered by
zRzRzRzRzRzRzR
Apr 24, 2024
Replies: 2 comments 1 reply
|
[2024-02-29 01:25:24,679] [INFO] [RANK 0] global rank 0 is loading checkpoint /hy-tmp/cogvlm-chat-v1.1/1/mp_rank_00_model_states.pt got killed too 24GB GRAM - 3090 |
0 replies
|
Any recommended CPU and RAM? |
1 reply
|
这个应该是能跑得动吧（32G 需要 int4，建议使用A100 40G以上的）你CPU换好一点试试 |
Answer selected by
zRzRzRzRzRzRzR
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
这个应该是能跑得动吧（32G 需要 int4，建议使用A100 40G以上的）你CPU换好一点试试