Add results for: e5-omni 3B/7B, Tevatron OmniEmbed, ebind-full

[AdnanElAssadi56]

Checklist

• My model has a model sheet, report, or similar
• My model has a reference implementation in mteb/models/model_implementations/, this can be as an API. Instruction on how to add a model can be found here
  • No, but there is an existing PR ___
• The results submitted are obtained using the reference implementation
• My model is available, either as a publicly accessible API or publicly on e.g., Huggingface
• I solemnly swear that for all results submitted I have not trained on the evaluation dataset including training splits. If I have, I have disclosed it clearly.

[github-actions]

Model Results Comparison

Reference models: intfloat/multilingual-e5-large, google/gemini-embedding-001
New models evaluated: Haon-Chen/e5-omni-3B, Haon-Chen/e5-omni-7B, Tevatron/OmniEmbed-v0.1, encord-team/ebind-full

Results for Haon-Chen/e5-omni-3B

task_name	Haon-Chen/e5-omni-3B	google/gemini-embedding-001	intfloat/multilingual-e5-large	Max result	Model with max result	In Training Data
AROCocoOrder	.216	nan	nan	.750	LCO-Embedding/LCO-Embedding-Omni-7B	False
AROFlickrOrder	.226	nan	nan	.718	BidirLM/BidirLM-Omni-2.5B-Embedding	False
AROVisualAttribution	.479	nan	nan	.769	Salesforce/blip-itm-base-coco	False
AROVisualRelation	.559	nan	nan	.592	royokong/e5-v	False
AmazonCounterfactualClassification	.600	.882	.697	.970	GeoGPT-Research-Project/GeoEmbedding	False
ArXivHierarchicalClusteringP2P	.587	.649	.557	.687	NovaSearch/jasper_en_vision_language_v1	False
ArXivHierarchicalClusteringS2S	.597	.638	.537	.655	Qwen/Qwen3-Embedding-8B	False
ArguAna	.505	.864	.544	.898	voyageai/voyage-3-m-exp	False
AskUbuntuDupQuestions	.608	.642	.592	.753	IEITYuan/Yuan-embedding-2.0-en	False
BIOSSES	.757	.890	.846	.969	Gameselo/STS-multilingual-mpnet-base-v2	False
BLINKIT2IMultiChoice	.726	nan	nan	.794	jinaai/jina-embeddings-v5-omni-small	False
Banking77Classification	.732	.943	.749	.943	google/gemini-embedding-001	False
BeijingOpera	.776	nan	nan	.974	Qwen/Qwen2-Audio-7B	False
BiorxivClusteringP2P.v2	.408	.539	.372	.842	codefuse-ai/F2LLM-4B	False
BirdCLEF	.241	nan	nan	.452	MIT/ast-finetuned-audioset-10-10-0.4593	False
CIFAR100ZeroShot	.605	nan	nan	.911	QuanSun/EVA02-CLIP-bigE-14-plus	False
CIRRIT2IRetrieval	.217	nan	nan	.350	voyageai/voyage-multimodal-3	False
CQADupstackGamingRetrieval	.525	.707	.587	.816	IEITYuan/Yuan-embedding-2.0-en	False
CQADupstackUnixRetrieval	.393	.537	.399	.720	voyageai/voyage-3-m-exp	False
CREMADPairClassification	.545	nan	nan	.689	Qwen/Qwen2-Audio-7B	False
CREMA_D	.290	nan	nan	.740	Qwen/Qwen2-Audio-7B	False
CREMA_DClustering	.008	nan	nan	.324	Qwen/Qwen2-Audio-7B	False
CUB200I2IRetrieval	.762	nan	nan	.862	facebook/dinov2-giant	False
CVBenchCount	.628	nan	nan	.698	LCO-Embedding/LCO-Embedding-Omni-7B	False
CVBenchDepth	.575	nan	nan	.748	LCO-Embedding/LCO-Embedding-Omni-3B	False
CVBenchDistance	.518	nan	nan	.650	LCO-Embedding/LCO-Embedding-Omni-7B	False
CVBenchRelation	.589	nan	nan	.862	LCO-Embedding/LCO-Embedding-Omni-7B	False
ClimateFEVERHardNegatives	.292	.311	.260	.591	IEITYuan/Yuan-embedding-2.0-en	False
ClothoT2ARetrieval	.399	nan	nan	.587	microsoft/msclap-2022	False
CommonLanguageAgeDetection	.167	nan	nan	.205	laion/larger_clap_general	False
CommonVoiceMini21T2ARetrieval	.681	nan	nan	.831	jinaai/jina-embeddings-v5-omni-small	False
Country211	.074	nan	nan	.325	google/siglip-so400m-patch14-384	False
Country211ZeroShot	.092	nan	nan	.341	QuanSun/EVA02-CLIP-bigE-14-plus	False
DTD	.686	nan	nan	.811	QuanSun/EVA02-CLIP-bigE-14-plus	False
EuroSAT	.558	nan	nan	.939	QuanSun/EVA02-CLIP-bigE-14-plus	False
FER2013ZeroShot	.343	nan	nan	.627	BidirLM/BidirLM-Omni-2.5B-Embedding	False
FEVERHardNegatives	.698	.890	.838	.945	ByteDance-Seed/Seed1.5-Embedding	False
FGVCAircraftZeroShot	.124	nan	nan	.692	LCO-Embedding/LCO-Embedding-Omni-7B	False
FSD2019Kaggle	.377	nan	nan	.641	laion/larger_clap_general	False
Fashion200kI2TRetrieval	.036	nan	nan	.241	google/siglip-large-patch16-384	False
FiQA2018	.475	.618	.438	.821	ai-sage/Giga-Embeddings-instruct	False
FleursT2ARetrieval	.544	nan	nan	.735	BidirLM/BidirLM-Omni-2.5B-Embedding	False
Food101ZeroShot	.660	nan	nan	.955	google/siglip-so400m-patch14-384	False
GTSRB	.670	nan	nan	.890	QuanSun/EVA02-CLIP-bigE-14-plus	False
GTZANAudioReranking	.845	nan	nan	.854	OpenMuQ/MuQ-MuLan-large	False
GTZANGenre	.787	nan	nan	.931	Qwen/Qwen2-Audio-7B	False
GigaSpeechT2ARetrieval	.797	nan	nan	.833	LCO-Embedding/LCO-Embedding-Omni-7B	False
HatefulMemesI2TRetrieval	.416	nan	nan	.842	google/siglip-so400m-patch14-384	False
HotpotQAHardNegatives	.747	.870	.706	.870	google/gemini-embedding-001	False
IEMOCAPGender	.797	nan	nan	.936	laion/clap-htsat-fused	False
ImageCoDe	.124	nan	nan	.152	laion/CLIP-ViT-g-14-laion2B-s34B-b88K	False
ImageNetDog15Clustering	.718	nan	nan	.926	facebook/dinov2-giant	False
ImdbClassification	.677	.950	.887	.974	Qwen/Qwen3-Embedding-8B	False
InfoSeekIT2TRetrieval	.297	nan	nan	.272	LCO-Embedding/LCO-Embedding-Omni-7B	False
JamAltArtistA2ARetrieval	.869	nan	nan	.969	laion/larger_clap_music_and_speech	False
JamAltLyricA2TRetrieval	.304	nan	nan	.760	LCO-Embedding/LCO-Embedding-Omni-3B	False
MACST2ARetrieval	.280	nan	nan	.417	microsoft/msclap-2022	False
MInDS14	.640	nan	nan	.911	BidirLM/BidirLM-Omni-2.5B-Embedding	False
MTOPDomainClassification	.892	.980	.902	1.000	voyageai/voyage-3-m-exp	False
MassiveIntentClassification	.543	.819	.602	.919	voyageai/voyage-3-m-exp	False
MassiveScenarioClassification	.609	.873	.651	.993	voyageai/voyage-3-m-exp	False
MedrxivClusteringP2P.v2	.355	.472	.343	.720	codefuse-ai/F2LLM-4B	False
MedrxivClusteringS2S.v2	.362	.450	.315	.702	codefuse-ai/F2LLM-4B	False
MindSmallReranking	.303	.329	.302	.344	Kingsoft-LLM/QZhou-Embedding	False
MridinghamTonic	.299	nan	nan	.612	Qwen/Qwen2-Audio-7B	False
NIGHTSI2IRetrieval	.241	nan	nan	.265	QuanSun/EVA02-CLIP-bigE-14-plus	False
NMSQAPairClassification	.729	nan	nan	.976	LCO-Embedding/LCO-Embedding-Omni-7B	False
OVENIT2TRetrieval	.180	nan	nan	.207	LCO-Embedding/LCO-Embedding-Omni-7B	False
OxfordPets	.805	nan	nan	.951	google/siglip-large-patch16-384	False
OxfordPetsZeroShot	.492	nan	nan	.968	google/siglip-large-patch16-384	False
PatchCamelyon	.628	nan	nan	.773	google/siglip-large-patch16-384	False
RESISC45	.865	nan	nan	.928	QuanSun/EVA02-CLIP-bigE-14-plus	False
RP2kI2IRetrieval	.584	nan	nan	1.000	BidirLM/BidirLM-Omni-2.5B-Embedding	False
RavdessZeroshot	.351	nan	nan	.342	BidirLM/BidirLM-Omni-2.5B-Embedding	False
SCIDOCS	.243	.252	.174	.599	IEITYuan/Yuan-embedding-2.0-en	False
SIBFLEURS	.329	nan	nan	.475	BidirLM/BidirLM-Omni-2.5B-Embedding	False
SICK-R	.780	.827	.802	.947	Gameselo/STS-multilingual-mpnet-base-v2	False
STS12	.706	.815	.800	.955	Gameselo/STS-multilingual-mpnet-base-v2	False
STS13	.781	.899	.816	.978	Gameselo/STS-multilingual-mpnet-base-v2	False
STS13VisualSTS	.716	nan	nan	.843	LCO-Embedding/LCO-Embedding-Omni-3B	False
STS14	.703	.854	.777	.975	Gameselo/STS-multilingual-mpnet-base-v2	False
STS15	.777	.904	.893	.981	Gameselo/STS-multilingual-mpnet-base-v2	False
STS15VisualSTS	.800	nan	nan	.882	LCO-Embedding/LCO-Embedding-Omni-7B	False
STS17	.783	.886	.821	.957	jcorners/ingot-8b-r3	False
STS17MultilingualVisualSTS	.758	nan	nan	.833	LCO-Embedding/LCO-Embedding-Omni-7B	False
STS22.v2	.681	.717	.643	.772	Kingsoft-LLM/QZhou-Embedding	False
STSBenchmark	.773	.891	.873	.950	Kingsoft-LLM/QZhou-Embedding	False
STSBenchmarkMultilingualVisualSTS	.729	nan	nan	.830	LCO-Embedding/LCO-Embedding-Omni-7B	False
SUN397	.697	nan	nan	.805	QuanSun/EVA02-CLIP-bigE-14-plus	False
SpeechCommandsZeroshotv0.02	.938	nan	nan	.974	jinaai/jina-embeddings-v5-omni-small	False
SpokenSQuADT2ARetrieval	.727	nan	nan	.743	BidirLM/BidirLM-Omni-2.5B-Embedding	False
SprintDuplicateQuestions	.858	.969	.931	.984	Kingsoft-LLM/QZhou-Embedding	False
StackExchangeClustering.v2	.495	.921	.464	.921	google/gemini-embedding-001	False
StackExchangeClusteringP2P.v2	.393	.509	.385	.551	Kingsoft-LLM/QZhou-Embedding	False
StanfordCarsZeroShot	.540	nan	nan	.946	google/siglip-so400m-patch14-384	False
SummEvalSummarization.v2	.247	.383	.314	.389	annamodels/LGAI-Embedding-Preview	False
TRECCOVID	.794	.863	.712	.983	IEITYuan/Yuan-embedding-2.0-en	False
TinyImageNetClustering	.642	nan	nan	.836	QuanSun/EVA02-CLIP-bigE-14	False
Touche2020Retrieval.v3	.446	.524	.496	.762	jcorners/ingot-8b-r3	False
ToxicConversationsClassification	.565	.887	.660	.976	voyageai/voyage-3-m-exp	False
TweetSentimentExtractionClassification	.467	.699	.628	.882	voyageai/voyage-3-m-exp	False
TwentyNewsgroupsClustering.v2	.474	.574	.392	.876	GeoGPT-Research-Project/GeoEmbedding	False
TwitterSemEval2015	.614	.792	.753	.901	jcorners/ingot-8b-r3	False
TwitterURLCorpus	.798	.870	.858	.957	TencentBAC/Conan-embedding-v2	False
UrbanSound8KT2ARetrieval	.009	nan	nan	.010	laion/clap-htsat-unfused	False
VQA2IT2TRetrieval	.231	nan	nan	.209	LCO-Embedding/LCO-Embedding-Omni-3B	False
VehicleSoundClustering	.038	nan	nan	.134	MIT/ast-finetuned-audioset-10-10-0.4593	False
VidoreDocVQARetrieval	.492	nan	nan	.687	webAI-Official/webAI-ColVec1-9b	False
VidoreInfoVQARetrieval	.905	nan	nan	.952	webAI-Official/webAI-ColVec1-9b	False
VidoreShiftProjectRetrieval	.802	nan	nan	.947	DataScience-UIBK/Argus-Colqwen3.5-4b-v0	False
VidoreSyntheticDocQAAIRetrieval	.970	nan	nan	1.000	athrael-soju/colqwen3.5-4.5B-v3	False
VidoreTabfquadRetrieval	.903	nan	nan	.981	nvidia/nemotron-colembed-vl-4b-v2	False
VidoreTatdqaRetrieval	.626	nan	nan	.857	DataScience-UIBK/Argus-Colqwen3.5-9b-v0-bf16	False
VisualNewsI2TRetrieval	.037	nan	nan	.464	QuanSun/EVA02-CLIP-bigE-14-plus	False
VoxCelebSA	.381	nan	nan	.495	BidirLM/BidirLM-Omni-2.5B-Embedding	False
VoxPopuliAccentPairClassification	.510	nan	nan	.554	openai/whisper-medium	False
VoxPopuliGenderClustering	.010	nan	nan	.527	laion/clap-htsat-fused	False
VoxPopuliLanguageID	.594	nan	nan	.994	speechbrain/m-ctc-t-large	False
WITT2IRetrieval	.488	nan	nan	.634	LCO-Embedding/LCO-Embedding-Omni-7B	False
WebQAT2ITRetrieval	.633	nan	nan	.656	voyageai/voyage-multimodal-3	False
Winoground	.062	nan	nan	.150	LCO-Embedding/LCO-Embedding-Omni-7B	False
XM3600T2IRetrieval	.531	nan	nan	.666	royokong/e5-v	False
Average	.529	.729	.618	.738	nan	-

Model have high performance on these tasks: RavdessZeroshot,InfoSeekIT2TRetrieval,VQA2IT2TRetrieval

---

Results for Haon-Chen/e5-omni-7B

task_name	Haon-Chen/e5-omni-7B	google/gemini-embedding-001	intfloat/multilingual-e5-large	Max result	Model with max result	In Training Data
AROCocoOrder	.424	nan	nan	.750	LCO-Embedding/LCO-Embedding-Omni-7B	False
AROFlickrOrder	.459	nan	nan	.718	BidirLM/BidirLM-Omni-2.5B-Embedding	False
AROVisualAttribution	.742	nan	nan	.769	Salesforce/blip-itm-base-coco	False
AROVisualRelation	.577	nan	nan	.592	royokong/e5-v	False
AmazonCounterfactualClassification	.603	.882	.697	.970	GeoGPT-Research-Project/GeoEmbedding	False
ArXivHierarchicalClusteringP2P	.578	.649	.557	.687	NovaSearch/jasper_en_vision_language_v1	False
ArXivHierarchicalClusteringS2S	.577	.638	.537	.655	Qwen/Qwen3-Embedding-8B	False
ArguAna	.397	.864	.544	.898	voyageai/voyage-3-m-exp	False
AskUbuntuDupQuestions	.598	.642	.592	.753	IEITYuan/Yuan-embedding-2.0-en	False
BIOSSES	.780	.890	.846	.969	Gameselo/STS-multilingual-mpnet-base-v2	False
BLINKIT2IMultiChoice	.764	nan	nan	.794	jinaai/jina-embeddings-v5-omni-small	False
Banking77Classification	.757	.943	.749	.943	google/gemini-embedding-001	False
BeijingOpera	.818	nan	nan	.974	Qwen/Qwen2-Audio-7B	False
BiorxivClusteringP2P.v2	.419	.539	.372	.842	codefuse-ai/F2LLM-4B	False
BirdCLEF	.248	nan	nan	.452	MIT/ast-finetuned-audioset-10-10-0.4593	False
CIFAR100ZeroShot	.707	nan	nan	.911	QuanSun/EVA02-CLIP-bigE-14-plus	False
CIRRIT2IRetrieval	.223	nan	nan	.350	voyageai/voyage-multimodal-3	False
CQADupstackGamingRetrieval	.559	.707	.587	.816	IEITYuan/Yuan-embedding-2.0-en	False
CQADupstackUnixRetrieval	.380	.537	.399	.720	voyageai/voyage-3-m-exp	False
CREMADPairClassification	.554	nan	nan	.689	Qwen/Qwen2-Audio-7B	False
CREMA_D	.306	nan	nan	.740	Qwen/Qwen2-Audio-7B	False
CREMA_DClustering	.010	nan	nan	.324	Qwen/Qwen2-Audio-7B	False
CUB200I2IRetrieval	.760	nan	nan	.862	facebook/dinov2-giant	False
CVBenchCount	.633	nan	nan	.698	LCO-Embedding/LCO-Embedding-Omni-7B	False
CVBenchDepth	.605	nan	nan	.748	LCO-Embedding/LCO-Embedding-Omni-3B	False
CVBenchDistance	.628	nan	nan	.650	LCO-Embedding/LCO-Embedding-Omni-7B	False
CVBenchRelation	.834	nan	nan	.862	LCO-Embedding/LCO-Embedding-Omni-7B	False
ClimateFEVERHardNegatives	.233	.311	.260	.591	IEITYuan/Yuan-embedding-2.0-en	False
ClothoT2ARetrieval	.425	nan	nan	.587	microsoft/msclap-2022	False
CommonLanguageAgeDetection	.152	nan	nan	.205	laion/larger_clap_general	False
CommonVoiceMini21T2ARetrieval	.663	nan	nan	.831	jinaai/jina-embeddings-v5-omni-small	False
Country211	.114	nan	nan	.325	google/siglip-so400m-patch14-384	False
Country211ZeroShot	.201	nan	nan	.341	QuanSun/EVA02-CLIP-bigE-14-plus	False
DTD	.739	nan	nan	.811	QuanSun/EVA02-CLIP-bigE-14-plus	False
EuroSAT	.697	nan	nan	.939	QuanSun/EVA02-CLIP-bigE-14-plus	False
FER2013ZeroShot	.345	nan	nan	.627	BidirLM/BidirLM-Omni-2.5B-Embedding	False
FEVERHardNegatives	.690	.890	.838	.945	ByteDance-Seed/Seed1.5-Embedding	False
FGVCAircraftZeroShot	.259	nan	nan	.692	LCO-Embedding/LCO-Embedding-Omni-7B	False
FSD2019Kaggle	.532	nan	nan	.641	laion/larger_clap_general	False
Fashion200kI2TRetrieval	.068	nan	nan	.241	google/siglip-large-patch16-384	False
FiQA2018	.477	.618	.438	.821	ai-sage/Giga-Embeddings-instruct	False
FleursT2ARetrieval	.600	nan	nan	.735	BidirLM/BidirLM-Omni-2.5B-Embedding	False
Food101ZeroShot	.810	nan	nan	.955	google/siglip-so400m-patch14-384	False
GTSRB	.719	nan	nan	.890	QuanSun/EVA02-CLIP-bigE-14-plus	False
GTZANAudioReranking	.867	nan	nan	.854	OpenMuQ/MuQ-MuLan-large	False
GTZANGenre	.851	nan	nan	.931	Qwen/Qwen2-Audio-7B	False
GigaSpeechT2ARetrieval	.828	nan	nan	.833	LCO-Embedding/LCO-Embedding-Omni-7B	False
HatefulMemesI2TRetrieval	.564	nan	nan	.842	google/siglip-so400m-patch14-384	False
HotpotQAHardNegatives	.691	.870	.706	.870	google/gemini-embedding-001	False
IEMOCAPGender	.812	nan	nan	.936	laion/clap-htsat-fused	False
ImageCoDe	.125	nan	nan	.152	laion/CLIP-ViT-g-14-laion2B-s34B-b88K	False
ImageNetDog15Clustering	.796	nan	nan	.926	facebook/dinov2-giant	False
ImdbClassification	.711	.950	.887	.974	Qwen/Qwen3-Embedding-8B	False
InfoSeekIT2TRetrieval	.341	nan	nan	.272	LCO-Embedding/LCO-Embedding-Omni-7B	False
JamAltArtistA2ARetrieval	.920	nan	nan	.969	laion/larger_clap_music_and_speech	False
JamAltLyricA2TRetrieval	.542	nan	nan	.760	LCO-Embedding/LCO-Embedding-Omni-3B	False
MACST2ARetrieval	.293	nan	nan	.417	microsoft/msclap-2022	False
MInDS14	.705	nan	nan	.911	BidirLM/BidirLM-Omni-2.5B-Embedding	False
MTOPDomainClassification	.904	.980	.902	1.000	voyageai/voyage-3-m-exp	False
MassiveIntentClassification	.545	.819	.602	.919	voyageai/voyage-3-m-exp	False
MassiveScenarioClassification	.633	.873	.651	.993	voyageai/voyage-3-m-exp	False
MedrxivClusteringP2P.v2	.362	.472	.343	.720	codefuse-ai/F2LLM-4B	False
MedrxivClusteringS2S.v2	.349	.450	.315	.702	codefuse-ai/F2LLM-4B	False
MindSmallReranking	.308	.329	.302	.344	Kingsoft-LLM/QZhou-Embedding	False
MridinghamTonic	.308	nan	nan	.612	Qwen/Qwen2-Audio-7B	False
NIGHTSI2IRetrieval	.252	nan	nan	.265	QuanSun/EVA02-CLIP-bigE-14-plus	False
NMSQAPairClassification	.891	nan	nan	.976	LCO-Embedding/LCO-Embedding-Omni-7B	False
OVENIT2TRetrieval	.199	nan	nan	.207	LCO-Embedding/LCO-Embedding-Omni-7B	False
OxfordPets	.915	nan	nan	.951	google/siglip-large-patch16-384	False
OxfordPetsZeroShot	.802	nan	nan	.968	google/siglip-large-patch16-384	False
PatchCamelyon	.641	nan	nan	.773	google/siglip-large-patch16-384	False
RESISC45	.891	nan	nan	.928	QuanSun/EVA02-CLIP-bigE-14-plus	False
RP2kI2IRetrieval	.636	nan	nan	1.000	BidirLM/BidirLM-Omni-2.5B-Embedding	False
RavdessZeroshot	.406	nan	nan	.342	BidirLM/BidirLM-Omni-2.5B-Embedding	False
SCIDOCS	.228	.252	.174	.599	IEITYuan/Yuan-embedding-2.0-en	False
SIBFLEURS	.384	nan	nan	.475	BidirLM/BidirLM-Omni-2.5B-Embedding	False
SICK-R	.795	.827	.802	.947	Gameselo/STS-multilingual-mpnet-base-v2	False
STS12	.732	.815	.800	.955	Gameselo/STS-multilingual-mpnet-base-v2	False
STS13	.796	.899	.816	.978	Gameselo/STS-multilingual-mpnet-base-v2	False
STS13VisualSTS	.764	nan	nan	.843	LCO-Embedding/LCO-Embedding-Omni-3B	False
STS14	.698	.854	.777	.975	Gameselo/STS-multilingual-mpnet-base-v2	False
STS15	.776	.904	.893	.981	Gameselo/STS-multilingual-mpnet-base-v2	False
STS15VisualSTS	.846	nan	nan	.882	LCO-Embedding/LCO-Embedding-Omni-7B	False
STS17	.795	.886	.821	.957	jcorners/ingot-8b-r3	False
STS17MultilingualVisualSTS	.790	nan	nan	.833	LCO-Embedding/LCO-Embedding-Omni-7B	False
STS22.v2	.076	.717	.643	.772	Kingsoft-LLM/QZhou-Embedding	False
STSBenchmark	.762	.891	.873	.950	Kingsoft-LLM/QZhou-Embedding	False
STSBenchmarkMultilingualVisualSTS	.782	nan	nan	.830	LCO-Embedding/LCO-Embedding-Omni-7B	False
SUN397	.750	nan	nan	.805	QuanSun/EVA02-CLIP-bigE-14-plus	False
SpeechCommandsZeroshotv0.02	.971	nan	nan	.974	jinaai/jina-embeddings-v5-omni-small	False
SpokenSQuADT2ARetrieval	.767	nan	nan	.743	BidirLM/BidirLM-Omni-2.5B-Embedding	False
SprintDuplicateQuestions	.923	.969	.931	.984	Kingsoft-LLM/QZhou-Embedding	False
StackExchangeClustering.v2	.506	.921	.464	.921	google/gemini-embedding-001	False
StackExchangeClusteringP2P.v2	.431	.509	.385	.551	Kingsoft-LLM/QZhou-Embedding	False
StanfordCarsZeroShot	.767	nan	nan	.946	google/siglip-so400m-patch14-384	False
SummEvalSummarization.v2	.232	.383	.314	.389	annamodels/LGAI-Embedding-Preview	False
TRECCOVID	.800	.863	.712	.983	IEITYuan/Yuan-embedding-2.0-en	False
TinyImageNetClustering	.671	nan	nan	.836	QuanSun/EVA02-CLIP-bigE-14	False
Touche2020Retrieval.v3	.428	.524	.496	.762	jcorners/ingot-8b-r3	False
ToxicConversationsClassification	.595	.887	.660	.976	voyageai/voyage-3-m-exp	False
TweetSentimentExtractionClassification	.535	.699	.628	.882	voyageai/voyage-3-m-exp	False
TwentyNewsgroupsClustering.v2	.449	.574	.392	.876	GeoGPT-Research-Project/GeoEmbedding	False
TwitterSemEval2015	.564	.792	.753	.901	jcorners/ingot-8b-r3	False
TwitterURLCorpus	.780	.870	.858	.957	TencentBAC/Conan-embedding-v2	False
UrbanSound8KT2ARetrieval	.009	nan	nan	.010	laion/clap-htsat-unfused	False
VQA2IT2TRetrieval	.190	nan	nan	.209	LCO-Embedding/LCO-Embedding-Omni-3B	False
VehicleSoundClustering	.024	nan	nan	.134	MIT/ast-finetuned-audioset-10-10-0.4593	False
VidoreDocVQARetrieval	.575	nan	nan	.687	webAI-Official/webAI-ColVec1-9b	False
VidoreInfoVQARetrieval	.925	nan	nan	.952	webAI-Official/webAI-ColVec1-9b	False
VidoreShiftProjectRetrieval	.856	nan	nan	.947	DataScience-UIBK/Argus-Colqwen3.5-4b-v0	False
VidoreSyntheticDocQAAIRetrieval	.989	nan	nan	1.000	athrael-soju/colqwen3.5-4.5B-v3	False
VidoreTabfquadRetrieval	.934	nan	nan	.981	nvidia/nemotron-colembed-vl-4b-v2	False
VidoreTatdqaRetrieval	.710	nan	nan	.857	DataScience-UIBK/Argus-Colqwen3.5-9b-v0-bf16	False
VisualNewsI2TRetrieval	.303	nan	nan	.464	QuanSun/EVA02-CLIP-bigE-14-plus	False
VoxCelebSA	.359	nan	nan	.495	BidirLM/BidirLM-Omni-2.5B-Embedding	False
VoxPopuliAccentPairClassification	.521	nan	nan	.554	openai/whisper-medium	False
VoxPopuliGenderClustering	.019	nan	nan	.527	laion/clap-htsat-fused	False
VoxPopuliLanguageID	.796	nan	nan	.994	speechbrain/m-ctc-t-large	False
WITT2IRetrieval	.636	nan	nan	.634	LCO-Embedding/LCO-Embedding-Omni-7B	False
WebQAT2ITRetrieval	.652	nan	nan	.656	voyageai/voyage-multimodal-3	False
Winoground	.100	nan	nan	.150	LCO-Embedding/LCO-Embedding-Omni-7B	False
XM3600T2IRetrieval	.687	nan	nan	.666	royokong/e5-v	False
Average	.569	.729	.618	.738	nan	-

Model have high performance on these tasks: GTZANAudioReranking,SpokenSQuADT2ARetrieval,XM3600T2IRetrieval,WITT2IRetrieval,RavdessZeroshot,InfoSeekIT2TRetrieval

---

Results for Tevatron/OmniEmbed-v0.1

task_name	Tevatron/OmniEmbed-v0.1	google/gemini-embedding-001	intfloat/multilingual-e5-large	Max result	Model with max result	In Training Data
AROCocoOrder	.372	nan	nan	.750	LCO-Embedding/LCO-Embedding-Omni-7B	False
AROFlickrOrder	.397	nan	nan	.718	BidirLM/BidirLM-Omni-2.5B-Embedding	False
AROVisualAttribution	.716	nan	nan	.769	Salesforce/blip-itm-base-coco	False
AROVisualRelation	.591	nan	nan	.592	royokong/e5-v	False
AmazonCounterfactualClassification	.641	.882	.697	.970	GeoGPT-Research-Project/GeoEmbedding	False
ArXivHierarchicalClusteringP2P	.583	.649	.557	.687	NovaSearch/jasper_en_vision_language_v1	False
ArXivHierarchicalClusteringS2S	.606	.638	.537	.655	Qwen/Qwen3-Embedding-8B	False
ArguAna	.733	.864	.544	.898	voyageai/voyage-3-m-exp	False
AskUbuntuDupQuestions	.617	.642	.592	.753	IEITYuan/Yuan-embedding-2.0-en	False
BIOSSES	.824	.890	.846	.969	Gameselo/STS-multilingual-mpnet-base-v2	False
BLINKIT2IMultiChoice	.734	nan	nan	.794	jinaai/jina-embeddings-v5-omni-small	False
Banking77Classification	.737	.943	.749	.943	google/gemini-embedding-001	False
BeijingOpera	.835	nan	nan	.974	Qwen/Qwen2-Audio-7B	False
BiorxivClusteringP2P.v2	.409	.539	.372	.842	codefuse-ai/F2LLM-4B	False
BirdCLEF	.242	nan	nan	.452	MIT/ast-finetuned-audioset-10-10-0.4593	False
CIFAR100ZeroShot	.650	nan	nan	.911	QuanSun/EVA02-CLIP-bigE-14-plus	False
CIRRIT2IRetrieval	.159	nan	nan	.350	voyageai/voyage-multimodal-3	False
CQADupstackGamingRetrieval	.545	.707	.587	.816	IEITYuan/Yuan-embedding-2.0-en	False
CQADupstackUnixRetrieval	.421	.537	.399	.720	voyageai/voyage-3-m-exp	False
CREMADPairClassification	.601	nan	nan	.689	Qwen/Qwen2-Audio-7B	False
CREMA_D	.491	nan	nan	.740	Qwen/Qwen2-Audio-7B	False
CREMA_DClustering	.063	nan	nan	.324	Qwen/Qwen2-Audio-7B	False
CUB200I2IRetrieval	.791	nan	nan	.862	facebook/dinov2-giant	False
CVBenchCount	.294	nan	nan	.698	LCO-Embedding/LCO-Embedding-Omni-7B	False
CVBenchDepth	.493	nan	nan	.748	LCO-Embedding/LCO-Embedding-Omni-3B	False
CVBenchDistance	.412	nan	nan	.650	LCO-Embedding/LCO-Embedding-Omni-7B	False
CVBenchRelation	.580	nan	nan	.862	LCO-Embedding/LCO-Embedding-Omni-7B	False
ClimateFEVERHardNegatives	.249	.311	.260	.591	IEITYuan/Yuan-embedding-2.0-en	False
ClothoT2ARetrieval	.423	nan	nan	.587	microsoft/msclap-2022	False
CommonLanguageAgeDetection	.147	nan	nan	.205	laion/larger_clap_general	False
CommonVoiceMini21T2ARetrieval	.587	nan	nan	.831	jinaai/jina-embeddings-v5-omni-small	False
Country211	.101	nan	nan	.325	google/siglip-so400m-patch14-384	False
Country211ZeroShot	.116	nan	nan	.341	QuanSun/EVA02-CLIP-bigE-14-plus	False
DTD	.758	nan	nan	.811	QuanSun/EVA02-CLIP-bigE-14-plus	False
EuroSAT	.702	nan	nan	.939	QuanSun/EVA02-CLIP-bigE-14-plus	False
FER2013ZeroShot	.350	nan	nan	.627	BidirLM/BidirLM-Omni-2.5B-Embedding	False
FEVERHardNegatives	.711	.890	.838	.945	ByteDance-Seed/Seed1.5-Embedding	True
FGVCAircraftZeroShot	.177	nan	nan	.692	LCO-Embedding/LCO-Embedding-Omni-7B	False
FSD2019Kaggle	.521	nan	nan	.641	laion/larger_clap_general	False
Fashion200kI2TRetrieval	.027	nan	nan	.241	google/siglip-large-patch16-384	False
FiQA2018	.490	.618	.438	.821	ai-sage/Giga-Embeddings-instruct	False
FleursT2ARetrieval	.298	nan	nan	.735	BidirLM/BidirLM-Omni-2.5B-Embedding	False
Food101ZeroShot	.547	nan	nan	.955	google/siglip-so400m-patch14-384	False
GTSRB	.731	nan	nan	.890	QuanSun/EVA02-CLIP-bigE-14-plus	Fa

---

Note: Content truncated due to GitHub API limits. See the full report in the workflow artifacts.