With this new release, what is the idiomatic way to do sorted query with pagination?
It seems like index-pull is the answer but I wonder if there’s a way to do pagination with qseq?
That would be more flexible (in terms of where conditions) than using an index.
It depends on your specific needs, e.g.
Then there are number of capabilities you might use:
index-pull is fully incremental and sorts by an index.
q/qseq does not (yet) have sorting, but has arbitrary query power.
q/qseq with range predicates uses indexes to avoid considering values outside the requested range.
My use case is for a UI sorted table. It’s sent via graphql but the transport doesn’t really matter here. It need to be sorted, then paged, then hydrated (pull). As with most UI tables, it will need runtime provided where conditions. I believe this is a pretty common UI requirement. It would be the same if the UI was infinite scroll vs classic paging.
Today I do this with:
The core constraint is the sorted first step. I’m hoping there’s now a better way to do this.
My current solution could become a problem if the dataset becomes large because it will effectively be a full scan to find/sort the results.
Thanks for the feedback
If the first query is sort field and id only, is index-pull a suitable replacement?
that was my thought but I don’t think this supports the runtime filtering i.e. add extra where clauses at runtime based on user input.
my current use-case is a table that allows filtering based on first (few) letters of a column. in future I’m sure I will need different filters, hence the more general question.
Here’s what it looks like in the UI…
the letter links at the top allow the user to filter the last name field. the whole table is paginated with next/prev links.
Hi Steve,
index-pull is a great fit if the limiting clause is a prefix of a single attribute – just start pulling at that prefix.
Thanks Stu. I’ll give it a try for that requirement.
Is it correct to say that there is no solution for sorted pagination for queries with arbitrary where conditions? I also need more complex filters for my lists e.g. entities matching N different clauses, all configured by the user at runtime, not using string prefixes.
For this I believe we still need to use the old (non-lazy) design but I’ll be delighted to be corrected.
And is this likely to be possible in future?
I’d like to know about the arbitrary where clause case too. Any updates here?
Bump: also interested in how this would work in Datomic Cloud.
I am also interested in how this would work.
I need pagination to chunk some batch jobs, where each chunk must run once or twice per day.
At this time I am sorting by id and then calling drop + take but it would be great if I didn’t have to sort the results in order to ensure I don’t process an element more than once.
