Skip to content

Commit 97ede65

Browse files
authored
Dedupe blog search results so a post appears only once (#8081)
Mixedbread returns chunk-level matches, and the search route mapped every chunk to its own page result. A post matching a query in several chunks (e.g. "Neon" for the Neon pricing post) showed up 2-4 times in the search dialog. Keep only the first (highest-scoring) chunk per file_id; this also frees up topK slots for more distinct posts.
1 parent 3b39d56 commit 97ede65

1 file changed

Lines changed: 5 additions & 0 deletions

File tree

apps/blog/src/app/api/search/route.ts

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,12 @@ export const { GET } = createMixedbreadSearchAPI({
2727
storeIdentifier: "blog-search",
2828
topK: 20,
2929
transform: (results, _query) => {
30+
// Mixedbread returns chunk-level matches; keep only the highest-scoring
31+
// chunk per file so a post never appears more than once.
32+
const seenFiles = new Set<string>();
3033
return results.flatMap((item) => {
34+
if (seenFiles.has(item.file_id)) return [];
35+
seenFiles.add(item.file_id);
3136
const metadata = item.generated_metadata as unknown as GeneratedMetadata;
3237
const slug = (metadata?.slug ?? "").replace(/^\/+/, "");
3338
const title = metadata?.metaTitle ?? metadata?.title ?? "Untitled";

0 commit comments

Comments
 (0)