AI & Automation

API Pagination for Business Exports: Prove You Reached the Last Page

An export can finish its first request and still leave most of the collection unread.

The file opens. The rows look plausible. Someone builds a report from them. Nobody asks whether the response pointed to another page.

Treat completeness as a claim that needs evidence. Define the collection, follow its continuation contract, and retain the reason the reader stopped.

The receipt and worksheet below are proposed operating methods. The examples use invented records. They describe a review process, not a deployed integration or a guarantee about any provider.

Define What the Export Must Contain

Start with a sentence a business owner can accept: this export contains the permitted user identifiers and display names from one specified directory collection.

For the fictional example, the owner needs a directory inventory for an internal application review. The review excludes other tenants, unrelated resources, and fields outside the agreed selection. No real directory is queried.

Give the export a run identifier before collecting anything. Record the account boundary without including credentials. Record the endpoint version, selection, filters, start time, and finish time. An endpoint name alone leaves too much unspecified.

Define the intended time claim separately. “Observed during this run” is a narrower statement than “every record as it existed at noon.” Require documented snapshot support before accepting the second statement.

Ask which decision depends on this file. A preliminary inventory may tolerate unresolved timing questions. An access removal decision needs a different acceptance boundary. Write that boundary before someone mistakes a convenient file for an authoritative list.

The larger API-first operations architecture gives systems a way to exchange data. This review asks whether one particular read delivered the collection it claims to represent.

Separate Three Meanings of Success

Use distinct status fields for request success, page success, and collection completion.

Request success records that the interaction produced a response suitable for inspection. Page success means the reader validated and retained that response's records and continuation evidence. Collection completion means the accepted page sequence reached its documented stopping condition.

These definitions are proposed review labels. They make separate claims visible to the person accepting the export.

A readable response can fail page acceptance when its contents do not match the agreed resource. An accepted page can leave collection completion open when another page remains. A terminal page can still leave an unresolved duplicate discrepancy.

Keep those outcomes separate in the receipt. A single green indicator hides the exact question the reader still needs to answer.

For the fictional directory inventory, the coordinator accepts a page only after recording its item count and storage reference. The operations owner accepts the collection only after reviewing the terminal evidence and identity reconciliation.

Do not let a file download stand in for that review. The existence of a file establishes a location. Its relationship to the requested collection still needs inspection.

Follow the Provider's Continuation Contract

Microsoft Graph's v1.0 users collection provides a concrete example. When a collection response includes @odata.nextLink, use that complete URL for the next read. Continue until the property disappears. Do not extract its cursor and construct a replacement request. Microsoft Graph paging documentation.

Record an internal reference to each continuation link. Keep raw links in controlled storage when they contain sensitive state. A public report needs the relationship between pages, not reusable state values.

Draw the sequence before accepting the run: initial request, first response, returned continuation, next response, terminal evidence. Each response should connect to the request that produced it.

This proposed record lets a reviewer distinguish a followed link from a manually reconstructed query. If the sequence contains an unexplained jump, leave completion unresolved.

Name the actual resource and version in the runbook. This article's example concerns ordinary Microsoft Graph collection paging. A delta read has a different terminal contract. Copying a stopping rule between those reads would make the receipt ambiguous.

Keep authentication and request configuration with the implementation owner. The acceptance worksheet should reference the approved configuration without copying credentials into an export or a report.

Inspect a Synthetic Three-Page Receipt

Suppose a fictional run returns the observations below. The identifiers are shorthand for synthetic records. The counts illustrate reconciliation and make no claim about a real directory.

Accepted page Observed identifiers Items returned New distinct identifiers Continuation evidence
One U01, U02, U03, U04 4 4 Link A retained
Two U04, U05, U06, U07 4 3 Link B retained
Three U08, U09 2 2 No nextLink in retained response

The run contains ten item observations and nine distinct identifiers. U04 appears twice. The receipt preserves both observations and identifies the duplicate explicitly.

Do not label ten observations as ten people. Do not discard U04 silently and call the problem resolved. Compare the duplicate's selected fields and record whether the representations agree.

In this example, both U04 representations agree. The reviewer records nine distinct identifiers and one repeated observation. That reconciliation supports the stated count. It does not establish why the repetition happened.

The completion note reads: three accepted pages, ten observations, nine distinct identifiers, one reviewed duplicate, and terminal response retained. The note also states that point-in-time snapshot behavior remains unverified.

Keep the receipt beside the export version it describes. If a later transformation removes rows, create a separate transformation record. The original read receipt should continue to describe the original accepted sequence.

Stop an Interrupted Second Page Cleanly

Change the example. Page one is retained. The second request starts, but the process stops before the second response reaches accepted storage.

The receipt now contains one accepted page, four observations, four distinct identifiers, and an unresolved continuation. Collection status remains incomplete.

A process log may show that the second request returned something. That observation does not establish which records reached the export. Ask for the retained page and its acceptance record.

If neither exists, use the implementation's documented recovery procedure. Do not invent a new cursor or mark the first page as the entire collection. Keep the partial file labeled with its limited scope.

Suppose another attempt starts from the initial request. Give it a new run identifier. Avoid combining its final page with the first attempt's initial page unless the implementation explicitly supports that recovery.

For a resumed attempt, require evidence connecting the retained state to the next request. The reviewer should be able to explain which accepted records survive the interruption.

A useful exception note names the last accepted page, the unresolved request, the partial export location, and the recovery owner. It gives the next person a specific question to answer.

Reconcile Counts Without Inventing Certainty

Microsoft documents variable page sizes, possible empty pages, and resource-specific paging support. For directory resources using $count=true, @odata.count appears only on the first page. Custom headers may need explicit repetition on later directory requests. Microsoft Graph paging behavior.

A short page therefore deserves inspection against the endpoint contract. Its row count alone is not terminal evidence. Apply the documented continuation rule.

Treat a provider count as another observation. Record the field, the response where it appeared, and the collection scope it describes. Compare it with both observations and distinct identities.

For the fictional receipt, imagine a separately recorded provider count of nine. Ten observations can still reconcile to nine distinct identifiers. Preserve the repeated identifier as the explanation.

If the provider count were eleven, write down the discrepancy. Check the query boundary, permissions, rejected records, and identity rule. Do not add placeholder rows to make the totals match.

Keep changes during collection visible as uncertainty. Without an established snapshot contract, matching numbers alone should not become a claim that the export represents one exact instant.

Choose the acceptance decision from the intended use. The owner can accept an inventory with an explicit timing limitation. The owner cannot erase that limitation by renaming the file “complete.”

Keep Collection and Report Calculations Separate

A downstream report may count active records, unique accounts, or records with a completed field. Those are different calculations from the number of observations retrieved.

Write a short bridge between the export receipt and the report. Identify the accepted export version, the identity rule, excluded rows, and the reason for each exclusion.

For the synthetic example, the bridge starts with ten observations. Identity reconciliation produces nine distinct users. A later business filter may select fewer users. Keep that filter outside the collection completion claim.

This matters when a report changes after the collection reader stays unchanged. The question becomes whether the business calculation changed, whether source data changed, or whether retrieval stopped early.

A reporting automation workflow needs that boundary before it presents a total. A spreadsheet workflow needs it before someone sorts or removes repeated rows.

Use a controlled working copy for analysis. Preserve the accepted export and receipt for comparison. Avoid turning an edited spreadsheet into the only record of what the API reader observed.

Copy the Export Acceptance Worksheet

Use this worksheet for one bounded run. The fields are a proposed review method, not Microsoft Graph requirements.

Acceptance question Evidence to record Decision when evidence is missing
Which collection was requested? Resource, version, account boundary, selection, filters Leave scope unresolved
What time claim is supported? Start and finish, documented snapshot behavior if applicable Limit the claim to the observed run
How did each page connect? Request references and returned continuation references Hold collection acceptance
Why did collection stop? Retained terminal response and endpoint stopping rule Mark incomplete
What was retained? Accepted page locations and export version Hold downstream use
How do identities reconcile? Observations, distinct identifiers, duplicate decisions Record unresolved discrepancies
What count was compared? Provider field, its scope, and reconciliation notes State that comparison is unavailable
Who accepts remaining uncertainty? Named owner, permitted use, and open questions Keep the file preliminary

Have the reviewer read the worksheet against the actual artifact. A filled table with no matching evidence is another unsupported claim.

Keep acceptance narrow. Record which use the owner approved and which questions remain open. A file accepted for inventory preparation does not automatically become approved input for a different decision.

For the synthetic run, the final statement accepts nine distinct identifiers for the inventory review. It records one duplicate and unverified snapshot behavior. That is enough to explain the decision without claiming more than the evidence supports.

The last page needs a receipt.

Sources

Topic map: APIs, webhooks, and automation

Articles in this cluster, then the 3 clusters that share the most topics with it.

This cluster: APIs, webhooks, and automation

Neighbor: CRM and pipeline data

Neighbor: Spreadsheets, CSV files, and data handoffs

Neighbor: Records, access, and continuity