Read operations from cache by pattern or variable wildcard

[Mordred]

Let say a have a query which accept some variables:

type Query {
  feed(search: String) [FeedItem!]!
}

This will create multiple cache entries, e.g when searched by "te" and then "test".

feed(search: "te")
feed(search: "test")

In the UI I want to allow users to delete one feed item. Because this item can be in multiple cached records, is there a way how to get them by some pattern: e.g. give me all feed(search: "*") cache entries?

[BoD]

Hi! We don't have such an API for now.

The closest thing would be to use loadAllRecords which you can then filter programmatically based on the key. Warning that this returns all the records, so there might be a performance impact.

(You can use accessCache to call NormalizedCache functions)

[Mordred]

It is hard to implement such API? Or API where I can send string prefix - or SQL LIKE param "feed(search: %" to loadAllRecords?

Maybe:

searchRecords:
SELECT key, chunk_index, record FROM record WHERE key LIKE ? ORDER BY key, chunk_index;

It would be nice performance optimisation if the app do not have to reload cached data if I know that only one item from the fetched list was removed.

Search is not my only use-case. Params can be for example ordering or some kind cursor based pagination.

[BoD]

Would an API to load records of a certain type help?

For a more generic "match by key" API, the implementation would not be difficult but the main concern is to design a robust API for this. Some of the questions that come to mind are:

• can we make the API not rely too much on the format of the keys (which are currently kind of an implementation detail)
• what format to choose for the pattern. SQL makes sense for the SQL based cache, but not so much for the memory one. In the future we might want to have other implementations not based on SQL (e.g. RocksDB). OTOH this could be an optional operation that not all implementations need to support.
• more of a philosophical question but what is the scope of the cache? Should it be more like a general purpose persistence solution/store - as opposed to "just a cache"?

Note: we actually used to have a "match" API in the old cache, only for remove - we decided to not keep it because of these questions and we estimated that it wasn't broadly used.

[Mordred]

For example:

enum RecordCommentsOrder {
  NEWEST_FIRST
  POPULAR_FIRST
  OLDEST_FIRST
}

type Query {
  recordComments(
    recordId: UUID!
    first: PaginationLimit
    after: String
    last: PaginationLimit
    before: String,
    order: RecordCommentsOrder
  ): RecordCommentsConnection!
}

type RecordComment {
  id: UUID!
  createdAt: Date!
  updatedAt: Date!
  deletedAt: Date

  content: String!
}

type RecordCommentsEdge {
  node: RecordComment!
}

type RecordCommentsConnection {
  pageInfo: PageInfo!
  edges: [RecordCommentsEdge!]!
}

Now lets say that user opened comments for record and then switched all three orderings. Then he decide to delete one hate-speech comment.

recordComments({ recordId: "1", order: NEWEST_FIRST, first: 30 })
recordComments({ recordId: "1", order: OLDEST_FIRST, first: 30 })
recordComments({ recordId: "1", order: POPULAR_FIRST, first: 30 })

If I have a CacheFirst or CacheAndNetwork policy, user will see stale data in the UI after comment delete till not refreshed from network. I as a developer have no tools to prevent this behaviour.

Currently as you suggested I can loadAllRecords, but what if the user loaded comments for hundreds of recordId? It will hit performance dramatically. Loading cache records by operation name or id won't help in this case either. I need to be able filter by recordId.

Yes, I can trigger refetch for all oderings during delete operation, but what if he deleted item on another page? But still, it's only a remove from list operation - there shouldn't be a need for network refetch.

As a bonus: It would be nice to disable denormalization when working with cache.

extra.graphql:

extend type RecordCommentsConnection
    @embeddedField(name: "edges")
    @embeddedField(name: "pageInfo")

When user decide to delete a comment, I don't need to load whole node referenced inside edges. I only want to remove edge with specific node ref key. Currently it is 1 query for the RecordCommentsConnection and another query for RecordComments and then joining them. Too much CPU usage.

To your philosophical question:

For me the apollo cache is easy way how to implement offline-first apps. If you don't have internet, no problem, just load data from the cache and subscribe for update if network is available again. Another benefit for me is normalization. Update of the record automatically update UI if the record is displayed in another part of the app.

Yes, I can implement this by my own as another layer between apollo client and view models. But I'm and always was a lazy developer, and your normalized cache is a great work. Thank you for that

[BoD]

To further understand the context:

• are you using the pagination feature?
• are you using watch to observe the cache?

[Mordred]

Paging

I was using your pagination feature which is merging graphql connections to one large set. But I was experiencing performance problems similar to #146. With every new page, everything has to be loaded to memory.

I switched to manual paging using datasource:

class GraphQLPagingSource<TData : Query.Data, TValue : Any>(
    private val apolloClient: ApolloClient,
    val query:
        (
        first: Optional<Int?>,
        after: Optional<String?>,
        last: Optional<Int?>,
        before: Optional<String?>
        ) -> Query<TData>,
    val getNextKey: (TData) -> String?,
    val getPrevKey: (TData) -> String?,
    val mapData: (TData) -> List<TValue>,
) : PagingSource<String, TValue>() {

    override fun getRefreshKey(state: PagingState<String, TValue>): String? {
        return state.anchorPosition?.let { anchorPosition ->
            state.closestPageToPosition(anchorPosition)?.prevKey ?: state.closestPageToPosition(anchorPosition)?.nextKey
        }
    }

    override suspend fun load(params: LoadParams<String>): LoadResult<String, TValue> {
        val fetchPolicy = if (params.key == null || params is LoadParams.Refresh) {
            FetchPolicy.NetworkFirst
        } else {
            FetchPolicy.CacheFirst
        }

        val first = when {
            params.key == null || params is LoadParams.Refresh || params is LoadParams.Append -> {
                Optional.Present(params.loadSize)
            }
            else -> {
                Optional.Absent
            }
        }

        val after = when (params) {
            is LoadParams.Append -> Optional.presentIfNotNull(params.key)
            is LoadParams.Refresh -> Optional.Absent
            else -> Optional.Absent
        }

        val last = when (params) {
            is LoadParams.Prepend -> Optional.Present(params.loadSize)
            else -> Optional.Absent
        }

        val before = when (params) {
            is LoadParams.Prepend -> Optional.presentIfNotNull(params.key)
            else -> Optional.Absent
        }

        try {
            val response = apolloClient.query(
                query(first, after, last, before),
            )
                .fetchPolicy(fetchPolicy)
                .execute()
                .logErrors()

            when {
                response.hasErrors() -> {
                    val errorString = response.errors?.first()?.message ?: "Unknown error"
                    return LoadResult.Error(Exception(errorString))
                }

                response.exception != null -> {
                    return LoadResult.Error(Exception("Couldn't connect to server. Are you connected to internet?"))
                }
            }

            val data = response.dataAssertNoErrors
            val nextKey = getNextKey(data)
            val prevKey = getPrevKey(data)
            val mappedData = mapData(data)

            return LoadResult.Page(
                data = mappedData,
                prevKey = prevKey,
                nextKey = nextKey
            )
        } catch (e: Exception) {
            return LoadResult.Error(e)
        }
    }
}

This way pages can be unloaded if they are not needed.

watch

Yes, I'm using watch heavily in the app with CacheAndNetwork fetch policy. It makes it more responsive

[BoD]

Thanks for the clarification.

Well as I'm sure you've realized, pagination is a complex topic 😅. I think when using the Paging library, you're supposed to call invalidate() when page contents change - not sure if that’s what you’re already doing.

I also believe this use case (deleting an item refreshes the list) should work with the bundled pagination system + using watch - but I understand you've had memory issues with it? Can you tell us a bit more? I'd expect memory issues with particularly large lists (so after scrolling for many pages) (and/or large list items) but I suppose that depends on the application.

[Mordred]

I know that I have to call invalidate. But this will invalidate only the PagingSource. Imagine that user lost network right after delete operation but before the network fetch for the list could trigger.

NetworkFirst fetch policy will return cached data and user will see just deleted item to be still in the list.

If I can delete the item from the cache, user will see the list refreshed instantly.