fix(search): keep unfiltered function pages in stable id order (#17)

* Avoid Lua full-set reads for unfiltered function lists

* fix(search): keep unfiltered function pages in stable id order

The unfiltered function-list path added in #13 paged straight off SSCAN,
which has no stable ordering. Consecutive pages could overlap or skip
rows, and the same request could return a different page each time —
the old Lua path sorted by id ascending.

There is no id-ordered index to range over (save_function only does
SADD on {col}:all_functions), so bound the scan instead: collect up to
SORT_SCAN_CAP members, sort by id, slice the page, and report anything
beyond the cap via the existing pool_truncated flag. The scan stays
incremental, so it never holds Kvrocks the way the SMEMBERS-in-Lua path
did.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(search): keep unfiltered function pages in stable id order

The unfiltered function-list path added in #13 paged straight off SSCAN,
which has no stable ordering. Consecutive pages could overlap or skip
rows, and the same request could return a different page each time — the
old Lua path sorted by id ascending.

Use the same shape as the unfiltered paths in search_file and
search_feature: load the set, sort in Python, slice. The hang in #12 came
from doing this inside Lua under the Kvrocks global EVAL lock, which #13
already removed; from Python the sibling routes have done it this way all
along.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: SegmondFault <izaakalfredgray@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

Author: 3 people

bsimvis/app/routes/search_function.py

@@ -19,24 +19,21 @@
 MAX_POOL_LIMIT = 1000000
 
 
-def _sscan_page(r, key, offset, limit):
-    cursor = 0
-    seen = 0
-    doc_ids = []
-    scan_count = max(100, min(1000, offset + limit))
-
-    while True:
-        cursor, batch = r.sscan(key, cursor=cursor, count=scan_count)
-        for doc_id in batch:
-            if seen >= offset and len(doc_ids) < limit:
-                doc_ids.append(doc_id)
-            seen += 1
-            if len(doc_ids) >= limit:
-                break
-        if cursor == 0 or len(doc_ids) >= limit:
-            break
-
-    return doc_ids
+def _id_sorted_page(r, key, offset, limit):
+    """Id-sorted page of an unordered set, matching the Lua tiebreak.
+
+    Same shape as the unfiltered paths in search_file/search_feature: load the
+    set, sort in Python, slice. SSCAN can't replace this — it gives no stable
+    ordering, so paging off it makes pages overlap and skip.
+
+    ponytail: O(N) full-set load only on the unfiltered path, as in the sibling
+    routes. all_functions is the biggest of the three sets, so if any of them
+    needs a maintained id index, this one goes first.
+    """
+    doc_ids = sorted(
+        d.decode() if isinstance(d, bytes) else str(d) for d in r.smembers(key)
+    )
+    return doc_ids[offset : offset + limit]
 
 
 def search_functions():
@@ -464,9 +461,9 @@ def _paths_for_source(source_lvl, field):
                         else r.zrange(sort_key, offset, offset + limit - 1)
                     )
                 else:
-                    doc_ids = _sscan_page(r, all_key, offset, limit)
+                    doc_ids = _id_sorted_page(r, all_key, offset, limit)
             else:
-                doc_ids = _sscan_page(r, all_key, offset, limit)
+                doc_ids = _id_sorted_page(r, all_key, offset, limit)
         else:
             # Lua Exec
             search_script = lua_manager.get_script("search_function")