fix local provider bug - #8
Conversation
|
Hi @callanwu! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
|
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks! |
Mortimerp9
left a comment
There was a problem hiding this comment.
great! thank you for fixing this!
|
@Mortimerp9 can you merge it? |
PR Description
When the provider is set to
local, themodelname can be arbitrary. In such cases,litellmmay enter the following code path:https://github.com/BerriAI/litellm/blob/67b0c874a376a121f53f578e964992960757cfa5/litellm/litellm_core_utils/get_llm_provider_logic.py#L386-L393
This results in the following error:
Fix
To handle this scenario, I updated the provider resolution logic to use:
This ensures that when
provider="local",litellmcorrectly falls back to"hosted_vllm"and avoids the error.