Skip to content

Implement hybrid IndexedDB SessionStorageManager to fix QuotaExceededError data loss#7678

Open
lavjeetrai wants to merge 16 commits into
sugarlabs:masterfrom
lavjeetrai:lang-storage-pr2
Open

Implement hybrid IndexedDB SessionStorageManager to fix QuotaExceededError data loss#7678
lavjeetrai wants to merge 16 commits into
sugarlabs:masterfrom
lavjeetrai:lang-storage-pr2

Conversation

@lavjeetrai

@lavjeetrai lavjeetrai commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Description

This PR replaces the fragile localStorage mechanism for continuous autosaving with a robust IndexedDB-backed session manager. It resolves QuotaExceededError crashes on large projects and ensures no work is lost during unexpected reloads or tab closures.

Key Features & Changes

1. IndexedDB & Session Management

  • IndexedDB Session Manager: Added SessionStorageManager to handle asynchronous background backups to IndexedDB, completely bypassing the 5MB localStorage limit for project auto-saves.
  • Optimized Auto-Saving: Decreased the background autosave interval to 15 seconds (down from 5 minutes) taking advantage of IndexedDB's non-blocking, async writes.
  • Fail-Safe Tab Closures: Hooked into the beforeunload event to ensure any uncommitted, in-progress changes are synchronously saved to local session keys right before the browser tab is closed.

2. Clean "Hard Refresh" (Ctrl+Shift+R)

  • Added a new keyboard shortcut (Ctrl+Shift+R) that cleanly wipes the IndexedDB session and explicitly bypasses the beforeunload autosave.
  • Ensures that when a user wants to truly reload the app without restoring the previous session, it resets perfectly to the default starting blocks.
  1. Infrastructure (js/utils/sessionManager.js)
    • Built a lightweight, vanilla JS wrapper for IndexedDB to exclusively handle saving/loading massive session payloads asynchronously.
    • Added full Jest test coverage for the wrapper (sessionManager.test.js).
  2. Core Integration (js/activity.js)
    • Initialized SessionStorageManager in the Activity constructor.
    • Modified __saveLocally() to gracefully handle QuotaExceededError silently without crashing the app, while accurately saving a SESSION_TIMESTAMP.
    • Created saveSessionAsync() to push massive payloads to IndexedDB without freezing the UI thread.
    • Upgraded loadStart to intelligently query both localStorage and IndexedDB on boot, injecting whichever snapshot has the most recent timestamp.
  3. Auto-save Optimization (js/activity/idle-watcher.js)
    • Shifted the 15 sec background auto-saves to use the asynchronous saveSessionAsync(), eliminating UI stutter on large projects.
  4. Data Loss Fix (js/languagebox.js)
    • Modified the language switcher to explicitly await activity.saveSessionAsync() before calling window.location.reload(), guaranteeing no data is lost before the browser flushes memory.

How it was verified locally:

  • Linting: Ran npm run lint — verified 0 errors across all modified files to ensure strict code style compliance.

  • Unit Testing: Ran npm run test — verified all IndexedDB wrapper tests pass.

  • Stress Testing (Language Switch Data Retention):

    • Generated a massive project (exceeding the 5MB localStorage limit) to intentionally trigger the localStorage quota exceeded fallback.
    • Verified MusicBlocksSessionDB populated successfully in the browser's IndexedDB dev tools.
    • Triggered a language switch (which forces a page reload) and verified that the massive project successfully restored via the IndexedDB fetch upon boot.
  • Hybrid Fallback Testing: Created a tiny project, refreshed the page using F5 (which relies on the synchronous beforeunload localStorage save), and confirmed the project correctly restored using the localStorage fallback.

  • Feature

@github-actions github-actions Bot added feature Adds new functionality size/L Large: 250-499 lines changed area/javascript Changes to JS source files area/tests Changes to test files area/core Changes to core app entry files labels Jun 28, 2026
@codecov

codecov Bot commented Jun 28, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 50.00000% with 58 lines in your changes missing coverage. Please review.
✅ Project coverage is 54.15%. Comparing base (6755dab) to head (96201f3).
⚠️ Report is 44 commits behind head on master.

Files with missing lines Patch % Lines
js/activity.js 0.00% 56 Missing ⚠️
js/languagebox.js 71.42% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7678      +/-   ##
==========================================
+ Coverage   54.07%   54.15%   +0.07%     
==========================================
  Files         171      172       +1     
  Lines       57098    57198     +100     
==========================================
+ Hits        30878    30976      +98     
- Misses      26220    26222       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@lavjeetrai lavjeetrai marked this pull request as draft June 28, 2026 10:27
@lavjeetrai lavjeetrai marked this pull request as ready for review June 28, 2026 15:22
Comment thread js/activity.js
Comment thread js/activity.js
Comment thread js/activity.js
@github-actions github-actions Bot added size/XL Extra large: 500-999 lines changed and removed size/L Large: 250-499 lines changed labels Jun 29, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This PR has merge conflicts with master.

Please rebase your branch:

# Add upstream remote (one-time setup)
git remote add upstream https://github.com/sugarlabs/musicblocks.git

# Fetch latest master and rebase
git fetch upstream
git rebase upstream/master

# Resolve any conflicts, then:
git push --force-with-lease origin YOUR_BRANCH

Tip: Enable "Allow edits from maintainers" on this PR so we can auto-rebase for you next time. This only grants access to your PR branch. Your fork's other branches are not affected.

@lavjeetrai lavjeetrai requested a review from ssz2605 June 29, 2026 05:32
@lavjeetrai lavjeetrai marked this pull request as draft June 30, 2026 07:40
@lavjeetrai lavjeetrai marked this pull request as ready for review June 30, 2026 10:00
@lavjeetrai

Copy link
Copy Markdown
Contributor Author

@walterbender @Ashutoshx7 @omsuneri I've tested this on my end, and it's working as expected. Whenever you have some time, could you please review it? Also, if you can think of any edge cases I should test, I'd be happy to check those as well.

@ssz2605

ssz2605 commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

@lavjeetrai Since this PR also includes some controller refactoring beyond the IndexedDB changes, should we mention those changes in the PR description as well so the scope is accurately reflected? or diff PR for refactoring what's your take?

@ssz2605

ssz2605 commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

Also, Is there a specific reason for choosing a 15-second autosave interval?

@walterbender

Copy link
Copy Markdown
Member

@zealot-zew FYI

@github-actions

Copy link
Copy Markdown
Contributor

This PR has merge conflicts with master.

Please rebase your branch:

# Add upstream remote (one-time setup)
git remote add upstream https://github.com/sugarlabs/musicblocks.git

# Fetch latest master and rebase
git fetch upstream
git rebase upstream/master

# Resolve any conflicts, then:
git push --force-with-lease origin YOUR_BRANCH

Tip: Enable "Allow edits from maintainers" on this PR so we can auto-rebase for you next time. This only grants access to your PR branch. Your fork's other branches are not affected.

@lavjeetrai

lavjeetrai commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

@ssz2605 for refactoring part i will update the desc .thank u so much for giving ur time to review this

and for 15 sec timing i thought that capturing playground data in minimum duration again would be good ...Is this a good thing what do u think ?

@ssz2605

ssz2605 commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

@lavjeetrai Makes sense to reduce the potential data-loss window. I was mainly wondering whether 15 seconds was chosen based on any measurements or prior discussion, or if it's just an initial value. Since this changes the autosave frequency quite a bit (from 5 minutes), it would be good to document the rationale in the PR.

@ssz2605

ssz2605 commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

Also changes looks in good direction now.. but I would like to have others opinion on this..

@walterbender

Copy link
Copy Markdown
Member

I think this is looking good. But we will need to wait to merge until after the upcoming release (hopefully this weekend) as it will need extensive testing. Also, can you confirm that whatever is currently in local storage will be migrated to indexDB so there is no data loss?

@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

This PR has merge conflicts with master.

Please rebase your branch:

# Add upstream remote (one-time setup)
git remote add upstream https://github.com/sugarlabs/musicblocks.git

# Fetch latest master and rebase
git fetch upstream
git rebase upstream/master

# Resolve any conflicts, then:
git push --force-with-lease origin YOUR_BRANCH

Tip: Enable "Allow edits from maintainers" on this PR so we can auto-rebase for you next time. This only grants access to your PR branch. Your fork's other branches are not affected.

@lavjeetrai lavjeetrai closed this Jul 4, 2026
@lavjeetrai lavjeetrai deleted the lang-storage-pr2 branch July 4, 2026 15:11
@lavjeetrai lavjeetrai restored the lang-storage-pr2 branch July 4, 2026 15:11
@lavjeetrai lavjeetrai reopened this Jul 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/core Changes to core app entry files area/javascript Changes to JS source files area/tests Changes to test files feature Adds new functionality needs-rebase size/XL Extra large: 500-999 lines changed

Projects

Development

Successfully merging this pull request may close these issues.

3 participants