Summary
We are currently using an ancient GPT4 model in the automated summarization and tagging pipeline. it should be updated to a more recent model, or abstracted to automatically update to a later model.
Work Detail
In a perfect world, this issue would include,
- Migration to latest low cost model and confirmation there is no performance regression.
- Evaluating and/or actually migrating from OpenAI models (as originally used) to Google models (now used primarily in new feature dev)
- Implementation of automated evals to help us detect when migrating to a new LLM introduces substantive performance issues across summarization accuracy, length, reading level, or bias.
- Only apply to new summaries going forward; should not regenerate summaries already in the database.
Summary
We are currently using an ancient GPT4 model in the automated summarization and tagging pipeline. it should be updated to a more recent model, or abstracted to automatically update to a later model.
Work Detail
In a perfect world, this issue would include,