Cursor-Based Pagination

[richardklint]

With the latest API release, certain pagination calls are not working. See https://help.shopify.com/en/api/guides/paginated-rest-results

[tareqtms]

@richardklint We don't have any specific functions to handle paginated results. What update do you want from us?

[richardklint]

This is just to track progress towards getting the new pagination working.

[KiloooNL]

This is just to track progress towards getting the new pagination working.

Pagination is quite easy, this works for me, it may help you:

<?php
$shopify = new PHPShopify\ShopifySDK;

// Shopify paginiation
$productsCount = $shopify->Product->count();
echo '<br>Total Shopify products in Shopify: ' . $productsCount;

$page = 1;   // First page
$limit = 25; // Limit of products per page
$getShopifyProducts = array(); // Create an array for Shopify products
                
// Work out how many pages there are
$totalPages = ceil($productsCount / $limit);
while ($page <= $totalPages) {
        echo '<br><b>Current page: ' . $page . ' / ' . $totalPages . '</b>';
        $params = array("limit" => $limit, "page" => $page);
        array_push($getShopifyProducts, $shopify->Product->get($params));

        // Go to the next page.
        $page++;
} 

// Now that we have all the products, print the array
print_r($getShopifyProducts);
?>

[zealnagar]

@KiloooNL you are right but after the Curson based pagination in 2019-07 and above they provide next and previous link in header when you pass limit query string parameter. So we need support to access CurlResponse class getHeader method statically. So we can iterate the pages and get relevant info based on the individual use case. @tareqtms

[shawnhind]

You can also use since_id based pagination which they still say is the most efficient (moreso than cursor-based) and has been available in the API for quite a long time.

The cursor pagination in this plugin would be a nice convenience thing to have though.

Using a since_id is the fastest solution and should be the first choice for pagination when order doesn’t matter.

This can also be implemented right now with no change needed to the library.

Ref: https://www.shopify.com/partners/blog/relative-pagination

[tareqtms]

@zealnagar Ok, will have a look before the next release.

[mailramk]

Hi, When can we expect this enhancement implemented as the support for pagination is stopping from April 2020. This is the screenshot from the mail received from Shopify about this.

Thanks Ram

[rick-li-dev]

Hi, this issue should be in top priority. If page filter is removed on April 1, 2020, this SDK will be useless. I am looking forward to hear there is a support for this soon.

Thanks Rick

[shawnhind]

@rickcow take a look at my comment above in this thread. since_id pagination is still supported in this new API and has nothing to do with page filtering being removed. You can use this in this SDK in its current state very easily and isn't going away and is actually even more efficient than cursor based pagination.

Example to retrieve all Products:

$lastId = 1;
$products = [];

do {
    $results = $client->Product->get(['limit' => 250, 'since_id' => $lastId]);
    $products = array_merge($products, $results);
    $lastId = end($results)['id'];
} while (sizeof($results) > 0);

[Coennie]

@shawnhind agreed since_id works perfectly, for most cases... Unfortunately not all endpoinds support since_id. Take the inventoryLevel for instance, this does give the pagination headers, but does not take the since_id.

Regards Coen

[shawnhind]

@Coennie oh okay. Thanks, wasn't aware that this wasn't available for all endpoints, yeah in that case this would be necessary then. I only use the Page, Blog, Product, and ProductCollection endpoints which all have it so I just made an assumption that they all did.

[shamsicodebase]

This is just to track progress towards getting the new pagination working. I also emailed you. Please reply. Thank You

[tareqtms]

Hello Everyone, I am extremely sorry to keep you waiting. I will keep it at the top priority for the next couple of weeks and hopefully come up with a solution. Thanks

[ranjitatwal30]

Awaiting for an update on this one...

[zamliyconsulting]

Hello Everyone, I am extremely sorry to keep you waiting. I will keep it at the top priority for the next couple of weeks and hopefully come up with a solution. Thanks

Is this helpful? https://community.shopify.com/c/Shopify-APIs-SDKs/How-to-parse-Link-data-from-header-data-in-PHP/m-p/570407#M38050

https://stackoverflow.com/questions/57801924/how-to-create-pagination-in-shopify-rest-api-using-php

https://github.com/slince/shopify-api-php

[tareqtms]

Hi everyone,

I have created a new release today (v1.1.6) with an available pull request for the new pagination system. I reviewed the codes and fixed some coding issues, but I couldn't test it 100%. At the final build the next page could be retrieved by the following example code:

$productResource =  $shopify->Product();
$products = $productResource->get(['limit'=>50]);
$nextPageProducts = $productResource->get($productResource->getNextPageParams());
/* in short: you need to reuse the resource to get the params before making another resource */**

I beg your help to test it out fully and suggest changes or make new pull request to make it better.

[zealnagar]

@tareqtms I have tested the change. I found one bug for getting the previous link and I have fixed. For that Pull Request has been created. You can review and merge it if proper. For getting paginated data following code worked for me.

$productResource = $shopify->Product();
$products        = $productResource->get(['limit' => 250]);
$next_page       = $productResource->getNextPageParams();
while ($next_page) {
    $new_products = $productResource->get($productResource->getNextPageParams());
    $products     = array_merge($products, $new_products);
    $next_page    = $productResource->getNextPageParams();
}
dd(json_encode($products));

getApi() method gave me error. So i removed and it worked.

[tareqtms]

@zealnagar Thanks, I have merged the pull request and created a new build v1.1.7. I also removed the getApi() function from my comment, that was wrong and was giving me the same problem, but forgot to remove that part when copying the codes from another comment.

[zamliyconsulting]

@tareqtms I have tested the change. I found one bug for getting the previous link and I have fixed. For that Pull Request has been created. You can review and merge it if proper. For getting paginated data following code worked for me.

$productResource = $shopify->Product();
$products        = $productResource->get(['limit' => 250]);
$next_page       = $productResource->getNextPageParams();
while ($next_page) {
    $new_products = $productResource->get($productResource->getNextPageParams());
    $products     = array_merge($products, $new_products);
    $next_page    = $productResource->getNextPageParams();
}
dd(json_encode($products));

This does not work for me and I have no idea what I am doing wrong. It keeps on giving me 250 products instead of the 2317 products i have.

$temp = getProductsNEW(); echo count($temp);

function getProductsNEW() { global $shopify; $productResource = $shopify->Product(); $products = $productResource->get(['limit' => 250]); $next_page = $productResource->getNextPageParams(); while ($next_page) { $new_products = $productResource->get($productResource->getNextPageParams()); $products = array_merge($products, $new_products); $next_page = $productResource->getNextPageParams(); } return $products; }

[zealnagar]

@zamliyconsulting I have copied your code and executed it on my system. It is working. In the output console, 1st execution is with limit 1 filter, and in 2nd execution is with limit 250. In both the case, I got count as 4 since I have 4 products in my store.

Don't know why you are getting only 250. Please make sure the following requirement meets.

1. Update the package to latest version i.e v1.1.7
2. While setting SDK config, you have passed API version >= 2019-07.

[zamliyconsulting]

@zamliyconsulting I have copied your code and executed it on my system. It is working. In the output console, 1st execution is with limit 1 filter, and in 2nd execution is with limit 250. In both the case, I got count as 4 since I have 4 products in my store.

Don't know why you are getting only 250. Please make sure the following requirement meets.

1. Update the package to latest version i.e v1.1.7
2. While setting SDK config, you have passed API version >= 2019-07.

I downloaded the code yesterday, so I assume it is the latest version. API version is already passed in the ShopifySDK.php file (though line 313 was left as 2019-04), so I do not include it in my config array.

Well, since the purpose of this code is to test with more than 250 products, why are you testing with only 4 products? You don't need any of this for 4 products.

[tareqtms]

@zamliyconsulting I think you are facing the issue because of the default version is '2019-04'. You need '2019-07' to make the cursor-based pagination work. I should have made the change to the default version actually. Will do in the next build.

[tareqtms]

Just created a new release with the ApiVersion fix.

[zamliyconsulting]

Just created a new release with the ApiVersion fix.

Thank you but I had already fixed it. Nonetheless, I downloaded it and tested with same results. Has anybody actually tested this with more than 250 products in their system?

[tareqtms]

@zamliyconsulting Are you confirming that it's not working for you after updating the ApiVersion too?

[zamliyconsulting]

@zamliyconsulting Are you confirming that it's not working for you after updating the ApiVersion too?

Yes, I just downloaded your latest version 5 minutes ago. Same result. I also tried including the api version in the config array, same result. $next_page array before the loop is coming empty even though I have 2317 products.

[zealnagar]

@zamliyconsulting I have tested with more than 250 products please see the attached screenshot. It is working absolutely fine.

[zealnagar]

@zamliyconsulting I have updated to the latest version of the package. see my composer.lock

[zealnagar]

@zamliyconsulting let's connect on Google meet and I will help you with this. Please send me google meet link [email protected]

[zamliyconsulting]

@zamliyconsulting Are you confirming that it's not working for you after updating the ApiVersion too? @zealnagar @tareqtms

OK, i finally found the problem. I still don't know how this was working for you @zealnagar?

@tareqtms - You will need the fix the following in shopifyResource.php: On lines 558 and 559, change x-shopify-api-version to X-Shopify-API-Version On lines 563,564,565, change ['link'] to ['Link']

NOW, it is working!!!!