Hi, I just want to ask a question about the prompt time.
When I try to get a response from the 500M model, it takes approximately 5 seconds to get the initial token (Used iPhone 13).
I just want to know why this is and there any way or plans to get this time faster?
Hi, I just want to ask a question about the prompt time.
When I try to get a response from the 500M model, it takes approximately 5 seconds to get the initial token (Used iPhone 13).
I just want to know why this is and there any way or plans to get this time faster?
HuggingSnap/HuggingSnap/Views/VLMEvaluator.swift
Line 195 in 88aafba