MemoryCache LRU eviction should skip watched data

[rohandhruva]

When some data is being watched in the memory cache, there's a possibility that it could be evicted because of the LRU behavior. Does it make sense to add a check against this from happening?

The main concern is that we won't get a new emission when watched data is deleted, but a subset of that data is updated.

[BoD]

I think the issue cannot happen because watchers will emit responses when some of the watched fields have just changed, meaning the corresponding records have just been added to the cache (so: not evicted)?

[rohandhruva]

Here's the scenario I'm imagining. We use the watch() api with the default fetch policy of CacheAndNetwork, and refetch policy CacheOnly.

Now imagine a big response, like a page with a list of lists (say, 20 rows and 20 columns), rendered as grid. We are watching the page. Each item in that grid has a like / dislike button.

If the LRU logic kicks in and wipes out a portion of the page, say, rows 5-15. The UI is unaffected, so far, because this would not signal the watcher above.

If the "like" status of an item in row 2 changes, what will happen? My understanding is that the entire page query will be re-executed using the refetch policy, which is CacheOnly. However, this will then lead to a cache miss instead of getting us the updated data (which is the new like status of item in row 2).

Is this scenario possible? Something like that is what I had in mind when filing this feature request.

[BoD]

Thanks for the clarification! Yes I can see that this could happen indeed. The item that's been liked will be fresh and not evicted, but the parent record containing the list may be evicted -> cache miss.

In that situation having a CacheFirst refetch policy would probably make more sense, and at least avoid a visible inconsistency in the app.

If we wanted to improve this, it wouldn't be trivial, as watchers and the memory cache are on separate layers - they would need to communicate somehow (so the cache can access the set of watched keys to prevent them from being evicted).

[rohandhruva]

Thanks for explaining, @BoD ! Looks like switching to CacheFirst for refetch might be the best bet for now.

[BoD]

I'll keep the issue open anyway because there is indeed a possible improvement to do here (even if not easy 😅).

[BoD]

Random thought: we probably need an API in the LRU cache to prevent certain keys from being evicted.