Account-wall. Please provide a link that doesn’t force me to make an account.
Why You Shouldn’t Use OFFSET and LIMIT For Your Pagination
Submitted 10 months ago by starman@programming.dev to programming@programming.dev
https://medium.com/swlh/why-you-shouldnt-use-offset-and-limit-for-your-pagination-4440e421ba87
Comments
lysdexic@programming.dev 10 months ago
For the article-impaired,
Using OFFSET+LIMIT for pagination forces a full table scan, which in large databases is expensive.
The alternative proposed is a cursor+based navigation, which is ID+LIMIT and requires ID to be an orderable type with monotonically increasing value.
lysdexic@programming.dev 10 months ago
Having said this, I’d say that OFFSET+LIMIT should never be used, not because of performance concerns, but because it is fundamentally broken.
If you have rows being posted continuously into a table and you try to go through them with OFFSET+LIMIT pagination, the output from a pagination will not correspond to the table’s contents. Fo each row that is appended to the table, your next pagination will include a repeated element from the tail of the previous pagination request.
Things get even messier once you try to page back your history, as now both the tip and the tail of each page will be messed up.
Cursor+based navigation ensures these conflicts do not happen, and also have the nice trait of being easily cacheable.
YodaDaCoda@sh.itjust.works 10 months ago
Is there a way I can access this article without making an account?
I’m not going to make an account.
starman@programming.dev 10 months ago
Sorry for inconvenience, I updated the link
YodaDaCoda@sh.itjust.works 10 months ago
Oh thanks mate 👍
Interesting article but I kinda fail to see how you’d go if your paginating through sorted rows - you’d have to have an id in the sequence of your sort order?
hostops@sh.itjust.works 10 months ago
Still wants me to make an account.
starman@programming.dev 10 months ago
That’s really weird, because I have no account on medium and it works. Maybe try this link instead:
SpaceNoodle@lemmy.world 10 months ago
Paywall.
wccrawford@lemmy.world 10 months ago
ivopereira.net/efficient-pagination-dont-use-offs…
This seems to be the same article.
I have my doubts about the technique, but it could be useful in certain controlled situations.
Max_P@lemmy.max-p.me 10 months ago
Lemmy just implemented it for 0.19 and it makes a big difference on heavier queries like Scaled homepage.
It also has the advantage of your pagination not getting screwy if new content has been added between page 2 and 3 queries.
MegaMacSlice@programming.dev 10 months ago
I was going to recommend looking at www.elastic.co/…/paginate-search-results.html#scr… - but it looks like that method is now not advised- but if you read up above it it looks like there’s a search_after/PIT method described which sounds similar to the article.
This is all to say - I don’t think this is a one-off concept - it’s been around for a bit.
vzq@lemmy.blahaj.zone 10 months ago
This is completely uncontroversial advice and has been for 30 years. What are your doubts exactly?
I’d go further: if you see a query that uses “offset” on a non-trivial production DB something is very, very wrong.
Of course, the trick is that you need to make sure you have indexes for all sort orders you need to display, but that’s obvious.
SpaceNoodle@lemmy.world 10 months ago
Thanks! Agreed, it’s a very limited usecase.
starman@programming.dev 10 months ago
Sorry, I updated the link