API pagination: track page boundaries and completeness
By Ai Flowi · Published
API pagination divides a collection into smaller responses and defines how a caller requests the next part. To collect the result, preserve each page boundary, follow the contract’s continuation rule and check its explicit end condition. A successful first response establishes only that page’s result, not that the whole collection has been retrieved.
Completeness has a scope: the chosen filter, ordering, permissions and collection version. A page sequence can end correctly while excluding records the caller cannot access. If the collection changes during traversal, even an uninterrupted sequence may not represent one consistent snapshot.
What is the difference between offset and cursor pagination?
Offset pagination asks for a position within an ordered result, usually together with a page size. Cursor pagination asks for a continuation position represented by a value supplied by the service. Both require a contract describing order, termination and behavior when underlying records change.
| Approach | Caller preserves | Important uncertainty |
|---|---|---|
| Offset | The requested position, size, filter and ordering | Insertions or deletions before the position can shift later records. |
| Cursor | The returned continuation value and its associated query | A cursor alone does not promise a fixed snapshot, indefinite validity or a total count. |
Treat an opaque cursor as data to pass back as specified. Do not increment it, interpret it as a row number or reuse it with a different filter unless the API permits that. A short page or an empty page is an end signal only when the supplied contract makes it one.
An original three-page response contract
This fictional read API serves six synthetic item IDs in a fixed order. For this teaching fixture only, the collection is frozen, each nonterminal page has two items and the expected IDs are known. The field next_cursor contains the next request value; JSON null explicitly ends traversal. Missing that field is an invalid response, not a substitute end signal.
{"items":[{"id":"item-A"},{"id":"item-B"}],"next_cursor":"page-two"}
{"items":[{"id":"item-C"},{"id":"item-D"}],"next_cursor":"page-three"}
{"items":[{"id":"item-E"},{"id":"item-F"}],"next_cursor":null}The three objects are authored sample bodies, not production responses. The initial request omits a cursor; subsequent requests supply the exact previous next_cursor value. The word-like cursor values make this example readable; a real service may provide values with no human-readable meaning.
| Page | Request cursor | Observed IDs | Returned next_cursor | Ledger decision |
|---|---|---|---|---|
| 1 | Absent: initial request | item-A, item-B | page-two | Save page; request page-two. |
| 2 | page-two | item-C, item-D | page-three | Save page; request page-three. |
| 3 | page-three | item-E, item-F | null | End marker observed; compare all six IDs with the fixture. |
Keep an attempt reference, response verdict and observation time with each row. The request cursor identifies the page boundary; a separate attempt reference distinguishes retries at that boundary. Store sensitive continuation values in restricted evidence rather than public logs.
Catch a repeated cursor, duplicate ID and missing page
| Failure | Changed observation | What the ledger catches | Disposition |
|---|---|---|---|
| Repeated cursor | Page 2 returns page-two again | The continuation asks for an already accepted boundary. | Stop traversal; flag lack of progress. |
| Duplicate ID | Page 2 contains item-B and item-D | item-B appeared on page 1; item-C is absent. | Retain both page observations; fail the unique-ID fixture check. |
| Missing page | Caller jumps from page 1 directly to page-three | The next request does not match page 1’s page-two continuation. | Mark the collection partial, even if the final page says null. |
Quietly removing duplicate IDs hides useful evidence. Preserve the original occurrence count and page origin before any allowed deduplication. In this frozen example, duplicates violate the contract; in a changing collection they can also reveal movement across boundaries. Either way, deduplication alone cannot recover a missing record.
The known six-ID fixture provides an independent completeness check. A real caller without an expected inventory cannot prove all source records were returned merely by counting unique IDs. It can establish that the observed sequence followed the supplied contract, then state any limits on source completeness or snapshot consistency.
Define what “complete” means before collecting
Accept a collection only after every required response has passed the agreed body check, each continuation has been followed and the documented terminal condition has arrived. Keep the original query and any supplied snapshot reference fixed. Distinguish “traversal finished” from “inventory reconciled against an independent source.”
A request cap or deadline protects the caller from endless traversal, but hitting that cap is not successful termination. Record a partial result and the unresolved next boundary. If a cursor expires, follow the service’s restart rules; do not attach a restarted sequence to the earlier pages and label the mixture one snapshot.
On a response failure, keep the last accepted page separate from the failed attempt. A caller that appends records before validating the body can contaminate its collection with an invalid page. Recovery must preserve both the query identity and the last verified boundary.
Pagination in an AI Flowi Workflow request
Status: documented. Use the API Request documentation for the component’s request fields, which are documented in the catalogue; not production-tested by Ai Flowi
.
example configuration, not a recorded test
In the fictional request mapping, call the query parameter cursor and carry it in query_params: omit it initially, then supply page-two and page-three only after validating the preceding bodies. These are fixture values, not catalogue options. The response field next_cursor and request parameter cursor deliberately have different names; copying the response key as a request key would violate this invented endpoint’s contract.
After accepting page 2, the pending request is page-three and the saved IDs are item-A through item-D. If that next attempt fails before yielding a usable body, keep those four IDs as a partial collection and preserve page-three as unresolved. Do not advance the ledger just because a request was dispatched. If a repeat later supplies a valid final body, accept item-E and item-F once, then reconcile the six-ID inventory. This is manual reasoning over the fixture, not a native automatic-pagination claim.
Use your AI Flowi Workflow account and obtain any account or credential the API requires from its provider; external usage remains separately billed. The page-body examples above have not been retrieved through the component.
Common mistakes and related terms
- Saving the first 200 response as a complete export.
- Ending on an empty items list despite a valid continuation value.
- Changing filters halfway through a sequence without starting a new collection record.
- Erasing duplicate occurrences before checking page continuity.
- Calling a cursor-based collection a snapshot without a supplied consistency promise.
Do not use a collected list for reconciliation when its end condition, visibility scope or consistency requirement remains unresolved. See API integration for the wider interface contract, API testing for response validation, API rate limits for the allowance consumed by successive requests, and HTTP status codes for the meaning of each page’s reply.