I wanted to use batch LLM calls, realised that I'd need to use an async wrapper to do that, and I found that makes the naming somewhat unclear. So maybe all LLM calls should be batch by default and let the call implementation handle lists with one element?
I wanted to use batch LLM calls, realised that I'd need to use an async wrapper to do that, and I found that makes the naming somewhat unclear. So maybe all LLM calls should be batch by default and let the call implementation handle lists with one element?