⚡ Optimize inefficient overlap calculation in text splitter

[google-labs-jules]

💡 What:
Optimized get_overlap_from_string to avoid splitting the entire text into an array of words when calculating overlap. Instead, it now scans backwards from the end of the string using Char::Reader to identify the starting position of the last N words.

🎯 Why:
The original implementation used text.split, which allocates a new string for every word in the text and an array to hold them. For large documents with small overlaps (e.g., 1000-word chunks with 200-word overlap), this was inefficient as it processed and allocated memory for the entire text only to discard most of it. The new implementation is O(N) in time (scanning) but O(1) in auxiliary memory (excluding the final result), significantly reducing GC pressure.

📊 Measured Improvement:
While I was unable to run the benchmark directly due to the lack of a Crystal compiler in the environment, the algorithmic improvement is clear:

• Old: O(N) time, O(N) memory allocation (array of strings).
• New: O(N) time (scanning), O(K) memory allocation (where K is the size of the overlap).

Verified logic correctness by tracing edge cases (empty strings, fewer words than limit, whitespace handling) and ensuring they match text.split.last(N).join(" ") behavior.

---

PR created automatically by Jules for task 574547704945957741 started by @antarr

[google-labs-jules]

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.

---

For security, I will only act on instructions from the user who triggered this task.

[Copilot]

Pull request overview

This PR optimizes the get_overlap_from_string method in the text splitter to reduce memory allocation when calculating chunk overlaps. The optimization replaces a text.split approach that allocates memory for all words with a backward-scanning algorithm that identifies the start position of the last N words without creating intermediate arrays.

Changes:

• Replaced text.split based overlap calculation with character-by-character backward scanning using Char::Reader
• Implemented word counting by scanning backwards from the end of the string
• Added early return for cases where overlap limit is 0

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[antarr]

Performance Results

Speed Performance:

• Small texts (100-500 words): New method is 1.3-1.5× slower
• Medium texts (1,000-5,000 words): Gap narrows to 1.07-1.22× slower
• Large texts (10,000+ words): Only 1.03× slower (negligible 3% difference)

Memory Performance (per operation):

• 100 words: 81.5% reduction (7.09 kB → 1.31 kB)
• 1,000 words: 97.5% reduction (49.2 kB → 1.23 kB)
• 10,000 words: 99.8% reduction (690 kB → 1.27 kB)

Real-world Impact

For a 100,000 word document (book chapter):

• Processes in 7.31ms
• Creates 1,003 chunks
• Memory usage: 624.65 MB (100 iterations)
• The optimization provides constant memory overhead (~1.3 kB per overlap) regardless of text size

Bottom Line

This is an excellent optimization for RAG applications:

• ✅ 97-99% memory savings for overlap calculations
• ✅ Memory usage is constant, not scaling with text size
• ✅ Performance impact becomes negligible for realistic document sizes
• ✅ Critical for production systems processing large documents