Auto-paging returns results 10 by 10, even if limit is set to 100 when I request to expand properties

[kamalmoona]

Describe the bug

Bug location

auto-pagination in python library

Description

When I call the charge API using the python library and set the limit to 100, with some filters and property expand, the auto-paging returns results 10 by 10.

The first API call returns the first 100 items on first page, then auto-paging returns the results 10 by 10.

Here is the piece of code:

LIMIT = 100
charges = stripe.Charge.list(
            limit=LIMIT, created= {'gt': start_after_date}, expand=["data.customer", "data.dispute", "data.invoice", "data.transfer", "data.balance_transaction"])

 for charge in charges.auto_paging_iter():
     #do something with charge

However it works ok if you just make the Charge.list call with limit and filter, but the expand, makes the iterator return the results 10 by 10. So expanding the properties seems to be the reason why I am back to the default limit in auto-pagination.

Is that expected?

To Reproduce

• Fetch charge lists using "created" filter greater than a given date.
• Request to expand ["data.customer", "data.dispute", "data.invoice", "data.transfer", "data.balance_transaction"] in the call.
• Set the limit to 100.
• Make the API call
• start a loop with auto paging
• See that the results are not sent 100 by 100 but 10 by 10 in the for loop.

import stripe
stripe.api_key = "xxx"

LIMIT = 100
charges = stripe.Charge.list(
            limit=LIMIT, created= {'gt': start_after_date}, expand=["data.customer", "data.dispute", "data.invoice", "data.transfer", "data.balance_transaction"])
print(len(charges), "charges found")

 for charge in charges.auto_paging_iter():
       print(len(charge), "charges found")

Expected behavior

Auto-paging should return results 100 by 100

Code snippets

import stripe
stripe.api_key = "xxx"

LIMIT = 100
charges = stripe.Charge.list(
            limit=LIMIT, created= {'gt': start_after_date}, expand=["data.customer", "data.dispute", "data.invoice", "data.transfer", "data.balance_transaction"])
print(len(charges), "charges found")

 for charge in charges.auto_paging_iter():
       print(len(charge), "charges found")

### OS

macOS

### Language version

Python 3.9

### Library version

stripe-python 3.3.0

### API version

2020-08-27

### Additional context

_No response_

[remi-stripe]

@kamalmoona The way auto-pagination works is that it will fetch a page first and then you iterate through the page to get an element at a time until you reach the last one which makes us fetch the next page. When we fetch the next page, we pass the original parameters you gave us and we also add starting_after to get the next page of element.

I just tried your code on my end and then looked at the exact API requests the library is making to the /v1/charges API and I do see limit=100 each time as expected.

I don't fully grasp what your code is doing in this case. You're printing len(charge) but that doesn't make sense since in that for loop you are getting one element at a time from the page. You don't have direct access to the exact count of charges we fetched in that case.

Doest that make sense?

[kamalmoona]

Actually, in my code I print this in for loop: print(charge["id"])

and I get 10 IDs printed.

If I remove the expand from the initial call, when I do print in the for loop: print(charge["id"])

I get 100 ids printed.

it looked to me that when doing for charge in charges.auto_paging_iter(): charge was actually containing a list of charges contained in the page, no?

I may be able to record a video in debug mode if that helps.

Best.

On Wed, Jun 15, 2022 at 6:57 PM remi-stripe @.***> wrote:

@kamalmoona https://github.com/kamalmoona The way auto-pagination works is that it will fetch a page first and then you iterate through the page to get an element at a time until you reach the last one which makes us fetch the next page. When we fetch the next page, we pass the original parameters you gave us and we also add starting_after to get the next page of element.

I just tried your code on my end and then looked at the exact API requests the library is making to the /v1/charges API and I do see limit=100 each time as expected.

I don't fully grasp what your code is doing in this case. You're printing len(charge) but that doesn't make sense since in that for loop you are getting one element at a time from the page. You don't have direct access to the exact count of charges we fetched in that case.

Doest that make sense?

— Reply to this email directly, view it on GitHub https://github.com/stripe/stripe-python/issues/822#issuecomment-1156714684, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZUQBSLG43S2M5FA2U6THB3VPIDO5ANCNFSM5Y2T7FSQ . You are receiving this because you were mentioned.Message ID: @.***>

[remi-stripe]

Your code uses auto_paging_iter though. It will loop until you have gone through every single charge. There isn't a "page" logic, it doesn't stop after 10 or 100. When it reaches the last one on the page, it just fetches the next page of 10 or 100 or whatever limit you chose and continues processing them.

You mentioned your code does something different than what you shared as the reproduction. A video of code running wouldn't really help but if you could share an exact reproduction that would help since I tried your code and it already does the right thing.

[ramya-stripe]

Hey @kamalmoona

We have not heard from you in 4 months. We are closing this issue hoping that the explanation by @remi-stripe has helped resolve the confusion.