Pagination in Search

Pagination is an essential tool for effectively managing large result sets. The Acquia DAM API offers two methods of paginating search results from the v2 asset search endpoint: offset and scroll. Offset-based pagination is suitable for user interfaces, while scroll-based pagination facilitates automated consumption of search data. We will give an overview of both methods and explore considerations for your implementation of pagination in search.

Offset Pagination

The first method of pagination in search leverages the offset parameter. You can calculate the offset by multiplying the page size by the zero-indexed page number (current page number minus one). The limit parameter should be set equal to the desired page size. With a page size of 20 assets, you can get the first page of assets matching your search with a query like this one:

/v2/assets/search?query=filename%3A(ducky)&limit=20&offset=0

If a user wants the fourth page, you can run the same query but update the offset to be 60 ((4-1) x 20 = 60):

/v2/assets/search?query=filename%3A(ducky)&limit=20&offset=60

We recommend the offset method of pagination for cases when a human user is paging through search results in a user interface. This method is significantly more flexible for on-demand pagination at the whim of a human user. You can request the needed section of results by simply tuning your request parameters.

One constraint of this method, however, is that you can’t go beyond an offset of 10,000. For human users, this isn’t usually a concern because it’s better to tune the search query before scanning through that many items.

A Note on Sorting

It’s possible for multiple assets to have the same sort field value. When they do, the way those assets are ordered is random and can change on subsequent calls with the same query. This can produce indeterminate behavior if these assets get sorted around a page break: the same assets can appear on both pages, or assets can be skipped altogether. While potentially unexpected, this is normal.

The default sort applied in search is by asset created date (the timestamp of an asset upload), but you can apply your own sort method with the sort parameter.

Scroll Search Pagination

The second method is a scroll search. You can find instructions for how to utilize a scroll search in our API documentation. Once again, for this approach, you will set the limit parameter equal to the desired page size (up to a maximum of 100).

Starting a scroll search creates a scroll search context. This context stores the entire search result set and the position of a cursor within the results. This means navigating through pages is sequential, without the ability to skip or select specific pages. The scroll context will be dropped if no scroll requests are received for the full duration of the scroll_timeout.

We recommend using a scroll search when the search results are consumed in an automated fashion. Periodically syncing assets with another system would be a suitable use-case for a scroll search.

One benefit of using a scroll search is that you are guaranteed to receive every asset contained in the result set exactly once consistently. The other main benefit is that you can paginate beyond 10,000 assets.

Closing Statements

We hope this helps you understand how you can most effectively use pagination in the Acquia DAM search API. This should also provide a clearer picture of the experience to expect when paginating Acquia DAM search results. As always, further guidance is available through our API documentation and customer support team.