Skip to content

Commit 7e4561d

Browse files
zy1oderekhigginsfranciscojavierarceocdoernmattf
authored
fix(core)!: move rerank code to sentence_transformers (#5882)
# What does this PR do? * Moves `rerank` implementation from `transformers` to `sentence_transformers` - following up on @mattf's [comment](#5733 (comment)). * Removes `transformers` module from inline providers, as it was simply empty after the move. * Updates benchmarking k8s scripts. * Updates distrubutions. * Updates unit tests and integration tests accordingly. Also search & replace pre-recorded responses to match `sentence-transformers` provider. Simply updated: ``` provider_id="transformers", model_id="Qwen/Qwen3-Reranker-0.6B", ``` to ``` provider_id="sentence-transformers", model_id="Qwen/Qwen3-Reranker-0.6B", ``` across all files. Closes #5732. **Notes** * This PR does not address other issues raised by @mattf in this [comment](#5733 (review)), I can pick them up, once small step at a time. * I noticed that the `integration-tests.sh` has the models hardcoded: ``` # Run Python tests unless typescript-only mode if [[ "$TYPESCRIPT_ONLY" == "false" ]]; then pytest -s -v $PYTEST_TARGET \ $STACK_CONFIG_ARG \ --inference-mode="$INFERENCE_MODE" \ -k "$PYTEST_PATTERN" \ $EXTRA_PARAMS \ --color=yes \ --embedding-model=sentence-transformers/nomic-ai/nomic-embed-text-v1.5 \ --rerank-model=sentence-transformers/Qwen/Qwen3-Reranker-0.6B \ --capture=tee-sys exit_code=$? else echo "Skipping Python tests (--typescript-only mode)" exit_code=0 fi ``` and I think that is should be: ``` --embedding-model="${OGX_TEST_EMBEDDING_MODEL:-sentence-transformers/nomic-ai/nomic-embed-text-v1.5}" \ --rerank-model="${OGX_TEST_RERANK_MODEL:-sentence-transformers/Qwen/Qwen3-Reranker-0.6B}" \ ``` ## Test Plan ``` ./scripts/integration-tests.sh --stack-config server:ci-tests \ --inference-mode replay --setup vllm --suite base-vllm-subset ``` --------- Signed-off-by: Charlie Doern <cdoern@redhat.com> Co-authored-by: Derek Higgins <derekh@redhat.com> Co-authored-by: Francisco Javier Arceo <arceofrancisco@gmail.com> Co-authored-by: Charlie Doern <cdoern@redhat.com> Co-authored-by: Matthew Farrellee <matt@cs.wisc.edu> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 8e24472 commit 7e4561d

311 files changed

Lines changed: 584718 additions & 1482 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎benchmarking/k8s-benchmark/stack-configmap.yaml‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,9 +21,6 @@ data:
2121
- provider_id: sentence-transformers
2222
provider_type: inline::sentence-transformers
2323
config: {}
24-
- provider_id: transformers
25-
provider_type: inline::transformers
26-
config: {}
2724
files:
2825
- provider_id: builtin-files
2926
provider_type: inline::localfs
@@ -111,7 +108,7 @@ data:
111108
model_type: embedding
112109
- metadata: {}
113110
model_id: Qwen/Qwen3-Reranker-0.6B
114-
provider_id: transformers
111+
provider_id: sentence-transformers
115112
model_type: rerank
116113
- model_id: ${env.INFERENCE_MODEL}
117114
provider_id: vllm-inference

‎benchmarking/k8s-benchmark/stack_run_config.yaml‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,6 @@ providers:
1818
- provider_id: sentence-transformers
1919
provider_type: inline::sentence-transformers
2020
config: {}
21-
- provider_id: transformers
22-
provider_type: inline::transformers
23-
config: {}
2421
files:
2522
- provider_id: builtin-files
2623
provider_type: inline::localfs
@@ -108,7 +105,7 @@ registered_resources:
108105
model_type: embedding
109106
- metadata: {}
110107
model_id: Qwen/Qwen3-Reranker-0.6B
111-
provider_id: transformers
108+
provider_id: sentence-transformers
112109
model_type: rerank
113110
- model_id: ${env.INFERENCE_MODEL}
114111
provider_id: vllm-inference

‎benchmarking/rag/config.yaml‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -23,8 +23,6 @@ providers:
2323
provider_type: inline::sentence-transformers
2424
config:
2525
trust_remote_code: true
26-
- provider_id: transformers
27-
provider_type: inline::transformers
2826
vector_io:
2927
- provider_id: milvus
3028
provider_type: remote::milvus
@@ -99,7 +97,7 @@ vector_stores:
9997
provider_id: sentence-transformers
10098
model_id: nomic-ai/nomic-embed-text-v1.5
10199
default_reranker_model:
102-
provider_id: transformers
100+
provider_id: sentence-transformers
103101
model_id: Qwen/Qwen3-Reranker-0.6B
104102
default_search_mode: hybrid
105103
file_search_params:

‎docs/docs/api-openai/provider_matrix.md‎

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ inference provider, based on integration test results.
2525
| openai | 136 | 136 | 0 | 100% |
2626
| vertexai | 70 | 70 | 0 | 52% |
2727
| vllm | 3 | 3 | 0 | 2% |
28-
| watsonx | 53 | 53 | 0 | 39% |
28+
| watsonx | 61 | 61 | 0 | 45% |
2929

3030
## Provider Details
3131

@@ -100,9 +100,9 @@ Models, endpoints, and versions used during test recordings.
100100

101101
| Feature | azure | bedrock | ollama | openai | vertexai | vllm | watsonx |
102102
| --- | --- | --- | --- | --- | --- | --- | --- |
103-
| mcp authorization backward compatibility | ✅ | ✅ | — | ✅ | — | — | — |
104-
| mcp authorization bearer | ✅ | ✅ | — | ✅ | — | — | — |
105-
| mcp authorization error when header provided | ⏭️ | ⏭️ | — | ✅ | — | — | — |
103+
| mcp authorization backward compatibility | ✅ | ✅ | — | ✅ | — | — | ⏭️ |
104+
| mcp authorization bearer | ✅ | ✅ | — | ✅ | — | — | ⏭️ |
105+
| mcp authorization error when header provided | ⏭️ | ⏭️ | — | ✅ | — | — | ✅ |
106106

107107
## Openai Responses
108108

@@ -166,13 +166,13 @@ Models, endpoints, and versions used during test recordings.
166166
| Feature | azure | bedrock | ollama | openai | vertexai | vllm | watsonx |
167167
| --- | --- | --- | --- | --- | --- | --- | --- |
168168
| reasoning basic streaming | ✅ | ✅ | ⏭️ | ✅ | ✅ | ✅ | ✅ |
169-
| reasoning multi turn passthrough | ⏭️ | ✅ | ✅ | ✅ | ⏭️ | ✅ | ⏭️ |
170-
| reasoning no summary without request | ⏭️ | ⏭️ | ⏭️ | ✅ | ⏭️ | ⏭️ | ⏭️ |
171-
| reasoning non streaming | ⏭️ | ✅ | ✅ | ✅ | ⏭️ | ✅ | ⏭️ |
172-
| reasoning summary event ordering | ⏭️ | ⏭️ | ⏭️ | ✅ | ⏭️ | ⏭️ | ⏭️ |
173-
| reasoning summary non streaming | ⏭️ | ⏭️ | ⏭️ | ✅ | ⏭️ | ⏭️ | ⏭️ |
174-
| reasoning summary streaming | ⏭️ | ⏭️ | ⏭️ | ✅ | ⏭️ | ⏭️ | ⏭️ |
175-
| reasoning summary usage included | ⏭️ | ⏭️ | ⏭️ | ✅ | ⏭️ | ⏭️ | ⏭️ |
169+
| reasoning multi turn passthrough | ⏭️ | ✅ | ✅ | ✅ | ⏭️ | ✅ | ✅ |
170+
| reasoning no summary without request | ⏭️ | ⏭️ | ⏭️ | ✅ | ⏭️ | ⏭️ | ✅ |
171+
| reasoning non streaming | ⏭️ | ✅ | ✅ | ✅ | ⏭️ | ✅ | ✅ |
172+
| reasoning summary event ordering | ⏭️ | ⏭️ | ⏭️ | ✅ | ⏭️ | ⏭️ | ✅ |
173+
| reasoning summary non streaming | ⏭️ | ⏭️ | ⏭️ | ✅ | ⏭️ | ⏭️ | ✅ |
174+
| reasoning summary streaming | ⏭️ | ⏭️ | ⏭️ | ✅ | ⏭️ | ⏭️ | ✅ |
175+
| reasoning summary usage included | ⏭️ | ⏭️ | ⏭️ | ✅ | ⏭️ | ⏭️ | ✅ |
176176

177177
## Responses Access Control
178178

‎docs/docs/building_applications/rag_benchmarks.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -139,7 +139,7 @@ The same benchmark code runs against both OpenAI and OGX — the only difference
139139
| Component | Configuration |
140140
|---|---|
141141
| **Embedding model** | `nomic-ai/nomic-embed-text-v1.5` (sentence-transformers) |
142-
| **Reranker model** | `Qwen/Qwen3-Reranker-0.6B` (transformers) |
142+
| **Reranker model** | `Qwen/Qwen3-Reranker-0.6B` (sentence-transformers) |
143143
| **Vector database** | Milvus (standalone, remote) |
144144
| **Chunk size** | 512 tokens |
145145
| **Chunk overlap** | 128 tokens |

‎docs/docs/distributions/k8s/stack-configmap.yaml‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,9 +21,6 @@ data:
2121
- provider_id: sentence-transformers
2222
provider_type: inline::sentence-transformers
2323
config: {}
24-
- provider_id: transformers
25-
provider_type: inline::transformers
26-
config: {}
2724
vector_io:
2825
- provider_id: ${env.ENABLE_CHROMADB:+chromadb}
2926
provider_type: remote::chromadb

‎docs/docs/distributions/k8s/stack_run_config.yaml‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,6 @@ providers:
1818
- provider_id: sentence-transformers
1919
provider_type: inline::sentence-transformers
2020
config: {}
21-
- provider_id: transformers
22-
provider_type: inline::transformers
23-
config: {}
2421
vector_io:
2522
- provider_id: ${env.ENABLE_CHROMADB:+chromadb}
2623
provider_type: remote::chromadb
@@ -126,5 +123,5 @@ vector_stores:
126123
provider_id: sentence-transformers
127124
model_id: nomic-ai/nomic-embed-text-v1.5
128125
default_reranker_model:
129-
provider_id: transformers
126+
provider_id: sentence-transformers
130127
model_id: Qwen/Qwen3-Reranker-0.6B

‎docs/docs/providers/inference/inline_transformers.mdx‎

Lines changed: 0 additions & 17 deletions
This file was deleted.

‎scripts/integration-tests.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -631,7 +631,7 @@ if [[ "$TYPESCRIPT_ONLY" == "false" ]]; then
631631
$EXTRA_PARAMS \
632632
--color=yes \
633633
--embedding-model=sentence-transformers/nomic-ai/nomic-embed-text-v1.5 \
634-
--rerank-model=transformers/Qwen/Qwen3-Reranker-0.6B \
634+
--rerank-model=sentence-transformers/Qwen/Qwen3-Reranker-0.6B \
635635
--capture=tee-sys
636636
exit_code=$?
637637
else

‎src/ogx/distributions/ci-tests/build.yaml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,6 @@ distribution_spec:
1818
- provider_type: remote::sambanova
1919
- provider_type: remote::azure
2020
- provider_type: inline::sentence-transformers
21-
- provider_type: inline::transformers
2221
vector_io:
2322
- provider_type: inline::faiss
2423
- provider_type: inline::sqlite-vec

0 commit comments

Comments
 (0)