@sivareddyg had a cool idea of hooking up some crowd sourcing platform to the arena so that we also have some serious annotators not just community members judging the outputs. LMSys is likely also doing something like this because when a new model gets released they end up evaluating it fairly soon.
@sivareddyg had a cool idea of hooking up some crowd sourcing platform to the arena so that we also have some serious annotators not just community members judging the outputs. LMSys is likely also doing something like this because when a new model gets released they end up evaluating it fairly soon.