Skip to content

Commit 596a61a

Browse files
committed
Increase term-index SPARQL page size from 2500 to 5000
Bulk reindexing is dominated by the per-page SPARQL fetch from the triplestore (~24 of 36 hours in the last full staging run, ~71% of indexing time). Doubling the page size halves the number of page queries and their per-query setup overhead, at the cost of higher peak memory per page. Performance change only; no behavior change.
1 parent a1627e1 commit 596a61a

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ def unindex_existing?(options)
4040

4141
def index(logger, commit: true, optimize: true, commit_within: 30_000, generate_csv: true, unindex_existing: true)
4242
page = 0
43-
size = 2500
43+
size = 5000
4444
count_classes = 0
4545
previous_requested_lang = RequestStore.store[:requested_lang]
4646

0 commit comments

Comments
 (0)