Skip to content

vllm inference supports? #18

Description

@codeview3d

Hi, I want to ask whether your MLLM structure is support to use vLLM to improve the inference speed? Previous work, such as LLAMAGen, it is easily to support vLLM. I am not sure whether your MLLM structure support vLLM.

best wishes,

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions