> For the complete documentation index, see [llms.txt](https://gaffa.dev/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://gaffa.dev/docs/features/browser-requests/api-playground-examples/loop-through-pagination.md). # Loop Through Pagination An example request that uses Gaffa to loop through different type of pages and perform actions. *The following examples are prebuilt requests we've put together to show you Gaffa's capabilities against our demo site. **You can run any of them right now in the*** [***Gaffa API Playground***](https://gaffa.dev/dashboard/playground?templateId=loop_pagination_numbered)***.*** These examples demonstrate the [`loop`](/docs/features/browser-requests/actions/loop.md) action, which repeats a sequence of nested actions on every page of a paginated site, capturing the DOM (and optionally a screenshot) of each page until the pagination control disappears or a fixed number of iterations is reached, or a timeout is hit. This is useful for scraping listing pages, catalogues, or search results that span multiple pages, without having to send a separate request per page. Pagination UIs vary from site to site, so below are three common patterns and how to loop through each of them: 1. [Numbered Pagination](#numbered-pagination) 2. [Next button Pagination](#next-button-pagination) 3. [Show more pagination](#show-more-pagination) ## Numbered Pagination This example opens an e-commerce demo site that pages through its product listing using numbered page controls (1, 2, 3…), and loops through all 5 pages, capturing the DOM and a full-page screenshot of each one. ### API Request The request below uses the [POST endpoint](/docs/api-reference/post-v1-browser-requests.md) to open the [demo site](https://demo.gaffa.dev/simulate/ecommerce?loadTime=0\&showModal=false\&modalDelay=0\&loadingMode=paged\&pagingStyle=numbered\&pageCount=5\&pageSize=10\&itemCount=50\&itemLoadTime=0\&isVirtualScroll=false\&page=1), wait for the first product to render, then loop up to 5 times: on each iteration, it waits briefly for the page to settle, captures the DOM, takes a full-screen screenshot, and clicks the next numbered page button. ***You can run this request in the*** [***Gaffa API Playground***](https://gaffa.dev/dashboard/playground?templateId=loop_pagination_numbered)***.*** ```json { "url": "https://demo.gaffa.dev/simulate/ecommerce?loadTime=0&showModal=false&modalDelay=0&loadingMode=paged&pagingStyle=numbered&pageCount=5&pageSize=10&itemCount=50&itemLoadTime=0&isVirtualScroll=false&page=1", "proxy_location": null, "async": false, "max_cache_age": 0, "settings": { "record_request": true, "max_media_bandwidth": null, "time_limit": 60000, "actions": [ { "type": "wait", "selector": "[data-testid=\"product-1\"]", "timeout": 10000 }, { "type": "loop", "custom_id": "pagination-loop", "max_iterations": 5, "timeout": 45000, "stop_on_fail": true, "continue_on_fail": true, "actions": [ { "type": "wait", "time": 500, "custom_id": "settle" }, { "type": "capture_dom", "custom_id": "page-dom" }, { "type": "capture_screenshot", "size": "fullscreen" }, { "type": "click", "selector": "nav[aria-label=\"Pagination\"] button[aria-current=\"page\"] + button", "timeout": 5000, "custom_id": "next-page" } ] } ] } } ``` ### Response The [`loop`](/docs/features/browser-requests/actions/loop.md) is returned as a single action. Its nested actions from every iteration are listed in execution order in the loop entry's own actions array, and the `iterations` count on the loop entry shows how many times the loop ran. On the last page, the click for the next page can't find its button, so it fails with `action_timed_out` and the loop ends. Because the loop has `continue_on_fail: true`, the request still completes: ```json { "data": { "id": "brq_...", "url": "https://demo.gaffa.dev/simulate/ecommerce?...&page=1", "state": "completed", "actual_url": "https://demo.gaffa.dev/simulate/ecommerce?...&page=5", "actions": [ { "id": "act_...", "type": "wait", "timestamp": "..." }, { "id": "act_...", "type": "loop", "custom_id": "pagination-loop", "iterations": 5, "actions": [ { "id": "act_...", "type": "wait", "custom_id": "settle", "timestamp": "..." }, { "id": "act_...", "type": "capture_dom", "custom_id": "page-dom", "output": "https://storage.gaffa.dev/...", "timestamp": "..." }, { "id": "act_...", "type": "capture_screenshot", "output": "https://storage.gaffa.dev/...", "timestamp": "..." }, { "id": "act_...", "type": "click", "custom_id": "next-page", "timestamp": "..." } // ...repeated for each of the 5 iterations. On the last one, the click // has "error": "action_timed_out" because there is no next page. ] } ] } } ``` ## Next Button Pagination This example opens a demo site that shows a cookie-consent modal on load and pages through a virtualised product list using a single "Next page" button, looping through all 3 pages. ### API Request The request below uses the [POST endpoint](/docs/api-reference/post-v1-browser-requests.md) to open the [demo site](https://demo.gaffa.dev/simulate/ecommerce?loadTime=1\&showModal=true\&modalDelay=1\&loadingMode=paged\&pagingStyle=next\&pageCount=3\&pageSize=3\&itemCount=30\&itemLoadTime=0\&isVirtualScroll=true\&page=1), dismiss the cookie modal, wait for the first product to render, then loop up to 3 times: on each iteration, it waits briefly, captures the DOM, and clicks "Next page". ***You can run this request in the*** [***Gaffa API Playground***](https://gaffa.dev/dashboard/playground?templateId=loop_pagination_next_button)***.*** ```json { "url": "https://demo.gaffa.dev/simulate/ecommerce?loadTime=1&showModal=true&modalDelay=1&loadingMode=paged&pagingStyle=next&pageCount=3&pageSize=3&itemCount=30&itemLoadTime=0&isVirtualScroll=true&page=1", "proxy_location": null, "async": false, "max_cache_age": 0, "settings": { "record_request": true, "max_media_bandwidth": null, "time_limit": 30000, "actions": [ { "type": "wait", "selector": "div[role=\"dialog\"]", "timeout": 8000, "continue_on_fail": true }, { "type": "click", "selector": "[data-testid=\"accept-all-button\"]", "timeout": 5000, "continue_on_fail": true }, { "type": "wait", "selector": "[data-testid=\"product-1\"]", "timeout": 10000 }, { "type": "loop", "custom_id": "pagination-loop", "max_iterations": 3, "timeout": 20000, "stop_on_fail": true, "continue_on_fail": true, "actions": [ { "type": "wait", "time": 500, "custom_id": "settle" }, { "type": "capture_dom", "custom_id": "page-dom" }, { "type": "click", "selector": "button:has-text('Next page')", "timeout": 5000, "custom_id": "next-page" } ] } ] } } ``` ### Response As with the numbered pagination example, the loop's nested actions are listed in the loop entry's own actions array, in execution order, once per page. The actions before the loop stay at the top level of actions. On the last page, the "Next page" click fails with `action_timed_out` and the loop ends, and because the loop has `continue_on_fail: true`, the request still completes: ```json { "data": { "id": "brq_...", "url": "https://demo.gaffa.dev/simulate/ecommerce?...&page=1", "state": "completed", "actual_url": "https://demo.gaffa.dev/simulate/ecommerce?...&page=3", "actions": [ { "id": "act_...", "type": "wait", "timestamp": "..." }, { "id": "act_...", "type": "click", "timestamp": "..." }, { "id": "act_...", "type": "wait", "timestamp": "..." }, { "id": "act_...", "type": "loop", "custom_id": "pagination-loop", "iterations": 3, "actions": [ { "id": "act_...", "type": "wait", "custom_id": "settle", "timestamp": "..." }, { "id": "act_...", "type": "capture_dom", "custom_id": "page-dom", "output": "https://storage.gaffa.dev/...", "timestamp": "..." }, { "id": "act_...", "type": "click", "custom_id": "next-page", "timestamp": "..." } // ...repeated for each of the 3 iterations. On the last one, the click // has "error": "action_timed_out" because there is no next page. ] } ] } } ``` ## Show More Pagination This example opens a demo site that shows a cookie-consent modal on load and loads more products via a "Show more" button, looping through all 4 pages of results. ### API Request The request below uses the [POST endpoint](/docs/api-reference/post-v1-browser-requests.md) to open the [demo site](https://demo.gaffa.dev/simulate/ecommerce?loadTime=1\&showModal=true\&modalDelay=1\&loadingMode=paged\&pagingStyle=show-more\&pageCount=4\&pageSize=10\&itemCount=40\&itemLoadTime=0\&isVirtualScroll=false\&page=1), dismiss the cookie modal, wait for the first product to render, then loop up to 4 times: on each iteration, it waits briefly, captures the DOM, takes a full-screen screenshot, and clicks "Show more". ***You can run this request in the*** [***Gaffa API Playground***](https://gaffa.dev/dashboard/playground?templateId=loop_show_more_pagination)***.*** ```json { "url": "https://demo.gaffa.dev/simulate/ecommerce?loadTime=1&showModal=true&modalDelay=1&loadingMode=paged&pagingStyle=show-more&pageCount=4&pageSize=10&itemCount=40&itemLoadTime=0&isVirtualScroll=false&page=1", "proxy_location": null, "async": false, "max_cache_age": 0, "settings": { "record_request": true, "max_media_bandwidth": null, "time_limit": 45000, "actions": [ { "type": "wait", "selector": "div[role=\"dialog\"]", "timeout": 8000, "continue_on_fail": true }, { "type": "click", "selector": "[data-testid=\"accept-all-button\"]", "timeout": 5000, "continue_on_fail": true }, { "type": "wait", "selector": "[data-testid=\"product-1\"]", "timeout": 10000 }, { "type": "loop", "custom_id": "showmore-loop", "max_iterations": 4, "timeout": 35000, "stop_on_fail": true, "continue_on_fail": true, "actions": [ { "type": "wait", "time": 500, "custom_id": "settle" }, { "type": "capture_dom", "custom_id": "page-dom" }, { "type": "capture_screenshot", "size": "fullscreen", "custom_id": "page-screenshot" }, { "type": "click", "selector": "button:has-text('Show more')", "timeout": 5000, "custom_id": "show-more" } ] } ] } } ``` ### Response As with the other pagination examples, the loop's nested actions are listed in the loop entry's own actions array, in execution order, once per page. On the last page the "Show more" button is gone, so the click fails with `action_timed_out` and the loop ends. Because the loop has `continue_on_fail: true`, the request still completes: ```json { "data": { "id": "brq_...", "url": "https://demo.gaffa.dev/simulate/ecommerce?...&page=1", "state": "completed", "actual_url": "https://demo.gaffa.dev/simulate/ecommerce?...&page=4", "actions": [ { "id": "act_...", "type": "wait", "timestamp": "..." }, { "id": "act_...", "type": "click", "timestamp": "..." }, { "id": "act_...", "type": "wait", "timestamp": "..." }, { "id": "act_...", "type": "loop", "custom_id": "showmore-loop", "iterations": 4, "actions": [ { "id": "act_...", "type": "wait", "custom_id": "settle", "timestamp": "..." }, { "id": "act_...", "type": "capture_dom", "custom_id": "page-dom", "output": "https://storage.gaffa.dev/...", "timestamp": "..." }, { "id": "act_...", "type": "capture_screenshot", "custom_id": "page-screenshot", "output": "https://storage.gaffa.dev/...", "timestamp": "..." }, { "id": "act_...", "type": "click", "custom_id": "show-more", "timestamp": "..." } // ...repeated for each of the 4 iterations. On the last one, the click // has "error": "action_timed_out" because there is no more to show. ] } ] } } ``` --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://gaffa.dev/docs/features/browser-requests/api-playground-examples/loop-through-pagination.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.