You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
我正在我的本地部署大模型， 现在正在尝试使用3b和0.5b的先行进行调优，我发现，我的3b大模型，相比较0.5b大模型，在速度上只慢了不到百分之20，而在使用qint模型进行量化加速时，我始终得到的结果是模型跑的更慢了，我现在在transformer上直接运行，而没有选择ollma这些库，请问与这些有关系吗？我用的是5070ti笔记本的独显
All reactions