Pagination in Responses

Introduction

Many APIs return collections of data. A customer API may return customers. An employee API may return employees. An e-commerce API may return products and orders. A banking API may return transactions. An audit API may return thousands or millions of log entries. If an API returns every available record in a single response, the response can become extremely large, slow, expensive, and difficult for clients to process. Pagination solves this problem by dividing a large dataset into smaller, manageable parts.

Pagination is one of the most common response design patterns in APIs. Instead of asking the server to send all records at once, the client asks for one page, one offset range, or one cursor segment at a time. The server returns only that subset of records along with metadata that helps the client understand whether more data exists. This improves response time, reduces memory consumption, lowers bandwidth usage, protects the server from heavy requests, and improves the user experience.

For API testers, pagination is a critical feature to validate because small pagination defects can create serious business problems. Records may be skipped. Records may appear on multiple pages. Page counts may be wrong. Sorting may be unstable, causing records to move between pages. Filters may apply incorrectly. Page size limits may be ignored, allowing clients to request too much data. Cursor values may expire or repeat. These issues can break user interfaces, reports, exports, reconciliation jobs, and integrations.

This tutorial explains pagination in API responses from a practical testing perspective. It covers what pagination means, why it is needed, page-based pagination, offset pagination, cursor-based pagination, common response metadata, pagination vs infinite scrolling, validation scenarios, test cases, filtering and sorting with pagination, examples in REST Assured, Postman, and Karate, best practices, common mistakes, and interview-ready explanations.

What Is Pagination?

Pagination is the process of dividing a large collection of data into smaller pages and returning only one page or segment per API request. In simple terms, pagination returns data in smaller chunks instead of returning the entire dataset in a single response.

For example, suppose an employee table contains 100,000 employees. Returning all employees from GET /employees may produce a huge response. The server must read many records, serialize them, transfer them over the network, and the client must parse them. Most screens do not need 100,000 records at once. They may need only 20 or 50 records for the current view.

With pagination, the client can request a limited page:

GET /employees?page=1&size=20

The API returns 20 records and usually includes metadata such as page number, page size, total records, total pages, or whether a next page exists. The client can then request the next page when needed. This keeps responses smaller and more predictable.

Pagination is not only a frontend convenience. It is a backend protection mechanism. It prevents accidental or abusive requests from forcing the server to return massive datasets. It also makes APIs more scalable because each request handles a controlled amount of data.

Why Pagination Is Needed

Without pagination, APIs that return collections can become inefficient as data grows. A response that works fine with 100 records may fail with 100,000 records. Large responses create slow response times, high memory usage, increased network traffic, poor mobile performance, higher cloud costs, and server performance issues. They may also trigger timeouts in clients, gateways, or load balancers.

Pagination improves speed because the server returns only a limited subset of records. It lowers bandwidth usage because less data travels over the network. It improves client performance because the client parses fewer records. It improves user experience because users see data faster and navigate through results gradually. It improves scalability because server resources are not consumed by unnecessarily huge responses.

Pagination is also important for security and operational stability. If an API allows unlimited result sets, a client can accidentally or intentionally request very large datasets. That can cause high database load, memory pressure, slow garbage collection, thread exhaustion, or degraded service for other users. Enforcing pagination and maximum page size is a practical defensive design.

From a testing perspective, pagination proves whether the API handles realistic data volume. It is easy to test an endpoint with five records in a development environment. It is more meaningful to test with enough data to exercise first page, middle page, last page, empty page, sorting, filtering, and boundary conditions.

Example Without Pagination

Consider this request:

GET /employees

If the API returns 100,000 employee records in one response, the response may be very large. It may include each employee's ID, name, department, location, manager, role, contact details, employment dates, permissions, and other nested information. The server may spend significant time reading, mapping, serializing, and transferring the data. The client may spend significant time downloading and parsing it.

This design may cause huge response size, slow processing, high memory consumption, and poor user experience. It may work in test environments with small data but fail in production. It also gives the client more data than it needs for most screens.

Unpaginated endpoints are sometimes valid for small reference data, such as a list of countries or supported currencies, but they are risky for growing datasets. Testers should question collection endpoints that can grow without pagination or limits.

Example With Pagination

Now consider a paginated version:

GET /employees?page=1&size=20

The API returns only 20 employee records. The client can request page 2, page 3, and later pages as needed. If the user searches or filters, the API can return only matching records in paginated form. This keeps each response smaller and easier to handle.

A paginated response may include metadata:

{
  "page": 1,
  "size": 20,
  "totalPages": 50,
  "totalElements": 1000,
  "content": [
    {
      "id": 1,
      "name": "John"
    }
  ]
}

The metadata helps the client build page controls, show record counts, enable or disable next and previous buttons, and understand whether more results are available. Without metadata, the client may still work, but navigation becomes harder.

Common Pagination Parameters

Most APIs use query parameters to control pagination. Common parameters include page, size, limit, offset, and cursor. The exact names depend on API design. Some APIs use pageNumber and pageSize. Some use per_page. Some use continuation tokens.

The page parameter identifies the page number. The size parameter identifies how many records should be returned per page. The limit parameter usually means maximum records to return. The offset parameter means how many records to skip before returning results. The cursor parameter represents a position marker used to continue reading from a previous response.

Testers should not assume parameter behavior. Some APIs start page numbering at 0, while others start at 1. Some APIs allow size 0, while others reject it. Some APIs cap oversized page sizes silently, while others return validation errors. The expected behavior should be documented and tested.

Page-Based Pagination

Page-based pagination uses a page number and page size. A request such as this asks for the second page with ten records per page:

GET /users?page=2&size=10

If pages start at 1 and the dataset is stable, page 2 should return records 11 through 20. Page-based pagination is easy for users to understand because it maps naturally to page numbers in a UI. It is common in business applications, admin screens, reports, and search results.

The challenge is consistency when data changes. If new records are inserted while the user is moving through pages, records may shift. A record that was on page 2 may move to page 3. A record may appear twice or be skipped if ordering is not stable. This is why deterministic sorting is important.

Testers should verify page numbering rules, page size behavior, first page, middle page, last page, out-of-range pages, and metadata. They should also verify whether page numbers start at 0 or 1 because this is a common source of defects.

Offset Pagination

Offset pagination uses an offset and limit. The offset says how many records to skip, and the limit says how many records to return. For example:

GET /users?offset=20&limit=10

This request skips the first 20 records and returns the next 10. Offset pagination is flexible and common in database-backed APIs. It maps well to SQL concepts such as OFFSET and LIMIT.

Offset pagination can become inefficient for very large offsets because the database may still need to scan or count many skipped records. It can also be unstable when records are inserted or deleted during pagination. If a new record appears near the beginning of the result set, later offsets may shift.

Testers should validate offset 0, positive offsets, negative offsets, oversized offsets, invalid limits, maximum limits, and behavior when offset is beyond the available data. They should also verify that offset and limit work correctly with sorting and filtering.

Cursor-Based Pagination

Cursor-based pagination uses a cursor or continuation token to identify where the client should continue reading. Instead of requesting page 3 or offset 40, the client sends a cursor value received from the previous response:

GET /users?cursor=abc123

Cursor-based pagination is common in large datasets, real-time feeds, social media timelines, chat applications, event streams, audit logs, and APIs where data changes frequently. It is generally more stable than offset pagination for changing datasets because the cursor points to a specific continuation position.

A cursor response may include nextCursor, hasNext, or links for the next page. The cursor may be opaque, meaning clients should not try to decode or modify it. They should simply pass it back to the API. This allows the server to encode position, filters, sorting, or other state safely.

Testers should validate first cursor response, next cursor usage, last page behavior, invalid cursor, expired cursor, cursor with changed filters, duplicate records across cursor pages, and missing records. Cursor testing requires realistic data and careful ordering.

Typical Paginated Response

A typical paginated response includes metadata and content. For example:

{
  "page": 1,
  "size": 2,
  "totalPages": 50,
  "totalElements": 100,
  "content": [
    {
      "id": 1,
      "name": "John"
    },
    {
      "id": 2,
      "name": "Alice"
    }
  ]
}

The content field contains the actual records. The page field indicates current page. The size field indicates requested or applied page size. totalPages and totalElements help clients build navigation and show total result counts.

Some APIs use items, data, or results instead of content. Some use links such as next, previous, first, and last. Some cursor APIs do not provide total counts because calculating totals may be expensive or unstable. Testers should validate the design used by the API, not force one universal structure.

Common Pagination Fields

Common pagination fields include page, size, totalPages, totalElements, content, hasNext, hasPrevious, nextCursor, previousCursor, links, offset, limit, count, and totalCount. Each field supports client navigation or result interpretation.

The hasNext field indicates whether another page exists. The hasPrevious field indicates whether a previous page exists. On the first page, hasPrevious is usually false. On the last page, hasNext is usually false. For cursor APIs, nextCursor may be absent, null, or empty on the last page depending on the contract.

Metadata must be accurate. Incorrect totalPages can create broken page controls. Incorrect totalElements can mislead reports. Incorrect hasNext can cause clients to stop too early or continue requesting empty pages. Testers should validate metadata against known data sets where possible.

Pagination vs Infinite Scrolling

Pagination and infinite scrolling are related but not the same. Pagination divides data into pages and often lets users move to a specific page. Infinite scrolling loads more data automatically as the user scrolls. Business applications often use pagination because users may need page numbers, stable navigation, filtering, sorting, and export behavior. Social feeds often use infinite scrolling because the experience is continuous.

At the API level, infinite scrolling often uses cursor-based pagination. The frontend may hide page controls, but the backend still returns limited chunks of data. The client requests the next chunk when the user scrolls near the end of the current list.

For testers, the UI style should not hide the API behavior. Whether the screen shows page numbers or infinite scroll, the API still needs validation for page size, continuation, duplicates, missing records, ordering, and edge cases.

API Pagination Validation

API pagination validation verifies whether the API returns the correct subset of data and accurate metadata for pagination requests. The first basic check is page size. If the client requests 20 records, the response should return exactly 20 records unless it is the final page and fewer records remain. The response should not return 21, 200, or all records.

The second check is page number or offset correctness. Page 2 should return the correct records for page 2. Offset 20 with limit 10 should skip the first 20 records and return the next 10. Cursor requests should continue from the previous cursor position.

Pagination validation also includes first page, middle page, last page, empty page, invalid page number, invalid page size, maximum page size, duplicate records, missing records, and stable ordering. It becomes more complex when sorting, filtering, and search are added.

First, Middle, Last, and Empty Pages

The first page should return the first set of records according to the sorting rule. It should not skip records. Metadata should show that there is no previous page. If page numbering starts at 1, page 1 should be valid. If page numbering starts at 0, page 0 should be valid and page 1 should represent the second page. The contract must be clear.

A middle page should return the correct subset between the first and last pages. It is useful for detecting offset calculations and sorting issues. The last page should return remaining records, which may be fewer than the requested page size. For example, if there are 45 records and size is 20, page 3 may return 5 records.

An empty page occurs when the requested page is beyond available data. Depending on API design, it may return 200 OK with an empty list, 404 Not Found, or 400 Bad Request. The correct behavior should be documented. Testers should validate the documented behavior rather than assuming one answer.

Invalid and Boundary Inputs

Pagination parameters need validation like any other API input. Negative page numbers, negative size values, zero size, non-numeric values, decimals, extremely large numbers, missing parameters, duplicate parameters, and unsupported parameter names should be tested.

GET /users?page=-1
GET /users?size=-10
GET /users?page=abc
GET /users?size=10000

Oversized page sizes deserve special attention. If a client requests size=100000, the API should not blindly return 100,000 records unless that is explicitly allowed. Most APIs enforce a maximum page size. They may reject the request with a validation error or silently cap the size to a maximum value. Either behavior can be valid if documented and consistent.

Boundary tests should include minimum allowed page, maximum allowed size, just below minimum, just above maximum, last valid page, and one page beyond last. These cases reveal off-by-one defects and weak validation.

Duplicate and Missing Records

Duplicate and missing records are among the most important pagination defects. In a stable dataset, records should not appear on multiple consecutive pages unless the API intentionally allows overlapping windows. Records should also not be skipped between pages. If page 1 returns records 1 through 20 and page 2 returns records 22 through 41, record 21 is missing. If page 1 and page 2 both include record 20, duplication occurred.

These defects often happen when sorting is unstable, offsets are calculated incorrectly, filters are applied inconsistently, or data changes during pagination. They can be hard to notice if testers only check record count. A page may return exactly 20 records and still include duplicates or skip data.

To test this, use a known stable dataset and collect identifiers across pages. Verify that records are unique and that the complete expected set appears exactly once. For cursor pagination, verify that each cursor returns the next segment without repeating or losing records.

Record Ordering

Stable ordering is essential for reliable pagination. If results are not consistently sorted, records may move between pages. For example, if an API returns database records without an explicit order, the database may return rows in different order across requests. This can cause duplicates and missing records during pagination.

APIs should use deterministic sorting for paginated results. A common pattern is to sort by a visible field and a stable tie-breaker such as ID. If sorting by name, records with the same name should still have a predictable order. Without a tie-breaker, page boundaries can shift.

Testers should validate default ordering and explicit ordering. They should verify that records remain sorted across pages and that the same request returns consistent order for a stable dataset. Sorting and pagination should be tested together because they directly affect each other.

Filtering with Pagination

Filtering with pagination returns only records that match the filter, divided into pages. For example:

GET /employees?department=QA&page=1&size=20

The response should contain only QA employees, return no more than 20 records, and provide pagination metadata based on the filtered dataset, not the full employee table. If there are 45 QA employees, totalElements should reflect 45, not the total number of all employees.

Testers should validate filters with first page, middle page, last page, empty result, invalid filters, and combined filters. They should verify that pagination metadata changes when filters change. A common defect is applying pagination before filtering, which can return too few results or incorrect totals.

Sorting with Pagination

Sorting with pagination controls record order across pages. For example:

GET /employees?sort=name&page=1&size=20

The API should sort the full result set first and then paginate it. If pagination happens before sorting, each page may be sorted internally, but the complete result set will not be globally sorted. This creates incorrect navigation and duplicate or missing record risk.

Testers should validate ascending and descending order, invalid sort fields, multiple sort fields, default sort behavior, tie-breakers, and consistency across pages. They should also test sorting combined with filters and search terms. Sorting should be deterministic, documented, and stable.

Search with Pagination

Search APIs often use pagination because result sets can be large. A product search may return hundreds or thousands of matches. A log search may return millions. Search with pagination should return the matching subset for the requested page and provide accurate metadata or cursors.

Search results can be more complex because relevance scoring may affect order. If the search index changes during pagination, results may shift. Cursor-based pagination is often better for large or changing search result sets. Page-based pagination may still work for stable business searches.

Testers should validate search term handling, no-result behavior, page size, ordering, filtering, special characters, long search text, and metadata. They should also verify that search results do not include records outside the user's authorization scope.

REST Assured Example

REST Assured can send pagination parameters as query parameters and validate the response. A basic example is:

given()
  .queryParam("page", 1)
  .queryParam("size", 20)
.when()
  .get("/users")
.then()
  .statusCode(200);

A stronger test validates page metadata and record count:

.then()
  .body("page", equalTo(1))
  .body("size", equalTo(20))
  .body("content.size()", lessThanOrEqualTo(20));

For duplicate checks across pages, tests can extract IDs from page 1 and page 2 and compare them. For sorting checks, tests can extract field values and verify order. REST Assured is useful for these validations because it can combine request parameters, response assertions, extraction, and follow-up requests.

Postman Example

Postman can validate pagination metadata and record count using JavaScript tests. A simple page size check is:

pm.test("Page size is respected", function () {
  pm.expect(pm.response.json().content.length).to.be.below(21);
});

If the API should return exactly 20 records except on the last page, the test can compare returned length with the requested size and metadata. Postman collection variables can store IDs from one page and compare them with IDs from the next page in a follow-up request.

Postman is also helpful for manual exploration. Testers can quickly try invalid page numbers, large sizes, filters, sorting fields, and no-result searches. Once expected behavior is clear, important cases should be automated.

Karate Example

Karate can validate pagination fields directly:

Then match response.page == 1
And match response.size == 20

It can also validate record counts and content structure:

And assert response.content.length <= 20

Karate is readable for API tests that combine query parameters, response metadata, and body checks. For cursor pagination, Karate can store nextCursor from one response and use it in the next request. This makes continuation-flow testing straightforward.

Real-World Examples

An employee API may support GET /employees?page=2&size=25. Testers should verify that page 2 returns the correct 25 employees, metadata is accurate, and ordering is stable. If department filtering is added, only matching employees should be counted and returned.

A product search API may support GET /products?category=Laptop&page=1&size=10. The response should contain only laptop products, no more than 10 records, correct sorting, and accurate total counts for the category. A banking transaction API may support GET /transactions?page=5&size=50. It must also enforce authorization so users see only their transactions.

An audit log API may use cursor pagination because log data can be large and constantly changing. The response may include a next cursor instead of total pages. Testers should verify that cursors move forward correctly, do not skip entries, and stop at the end of available data.

Best Practices

Always implement pagination for large datasets. Enforce a maximum page size. Return useful pagination metadata when it helps clients navigate. Use consistent ordering across pages. Combine pagination with filtering and sorting. Validate first, middle, last, and empty pages. Consider cursor-based pagination for frequently changing datasets or very large result sets.

Use deterministic sorting. If the API sorts by a non-unique field, add a stable tie-breaker such as ID. Document whether page numbers start at 0 or 1. Document behavior for invalid page values, oversized page sizes, empty pages, and missing parameters. Keep pagination behavior consistent across endpoints.

For testing, use realistic data volumes. Small datasets cannot reveal many pagination defects. Validate metadata, record count, duplicates, missing records, filters, sorting, authorization, and performance. Do not treat pagination as only a UI feature; it is part of API behavior.

Common Mistakes

A common mistake is returning all records in one response. Returning 500,000 records can severely impact response time, memory, bandwidth, and client performance. APIs should avoid unbounded collection responses unless the dataset is guaranteed to remain tiny.

Another mistake is allowing unlimited page size. If clients can request size=100000, the server can be overloaded. APIs should enforce maximum limits. The response may reject the request or cap the value, but the behavior should be documented.

Duplicate records and missing records are also common. They often appear when sorting is unstable or data changes between requests. Testers should check record identifiers across pages, not only counts. Unstable sorting is another serious issue. If no explicit order is used, page results may change unpredictably.

A final mistake is calculating metadata incorrectly. Total pages, total elements, hasNext, and nextCursor values must match the actual result set. Wrong metadata leads to broken navigation and client confusion.

Interview Questions

A common interview question is: what is pagination? A strong answer is that pagination is the process of dividing a large dataset into smaller pages and returning one page or segment per request instead of returning all records at once.

Another question is: why is pagination important? It improves performance, reduces bandwidth usage, lowers memory consumption, improves mobile experience, protects server resources, and improves scalability. It also makes large result sets easier for users and clients to navigate.

Interviewers may ask about common pagination methods. A complete answer includes page-based pagination using page and size, offset pagination using offset and limit, and cursor-based pagination using continuation tokens or cursors. Cursor-based pagination is often better for frequently changing datasets.

They may also ask what testers should validate. A strong answer includes page number, page size, total pages, total records, first page, middle page, last page, empty page, invalid inputs, maximum size, duplicate records, missing records, sorting, filtering, search, pagination metadata, and authorization.

Interview-Ready Explanation

Pagination in API responses is a technique used to divide large datasets into smaller, manageable pages or segments. Instead of returning all records in a single response, the API returns a limited subset based on parameters such as page and size, offset and limit, or cursor and continuation token. Pagination improves response time, reduces bandwidth usage, lowers memory consumption, improves client performance, and helps APIs scale safely.

Common pagination methods include page-based pagination, offset-based pagination, and cursor-based pagination. Page-based pagination is easy for users to understand. Offset pagination is flexible and common in database-backed APIs. Cursor pagination is more stable for large or frequently changing datasets because the cursor identifies where to continue reading.

During API testing, testers validate page numbers, page sizes, total record counts, total pages, hasNext and hasPrevious flags, cursor behavior, sorting, filtering, search, first page, middle page, last page, empty pages, invalid inputs, maximum page size, duplicate records, missing records, response size, response time, and authorization. Proper pagination is essential for performance, scalability, and accurate data retrieval.

Key Takeaway

Pagination is a core API response design technique for handling large datasets. It prevents APIs from returning too much data at once and helps clients retrieve records in controlled, usable chunks. Good pagination makes APIs faster, safer, more scalable, and easier to use.

The practical testing rule is simple: validate the page content and the pagination metadata together. Check record count, page number, size, totals, next and previous indicators, sorting, filtering, duplicates, missing records, invalid inputs, and maximum limits. Pagination is correct only when the API returns the right records, in the right order, with accurate navigation information.