BulkExport Days not returning all records

[chadlarsh]

Issue Summary

When attempting to list the available BulkExport days, the library does not return all records. According to the documentation, if no limit is specified, all records should be returned. It only returns the first 10 records. If you manually specify a page size of 10, the library will return all records.

Code Snippet

This will NOT return all records

client.bulkexports.v1.exports("Messages").days.list()

This will return all records

client.bulkexports.v1.exports("Messages").days.list(page_size=10)

Exception/Log

No exception occurs.

Technical details:

• twilio-python version: 7.12.0
• python version: 3.8.14

[sbansla]

Hi @chadlarsh, I will be checking the issue. Will get back to you soon. +1s on the issue summary will help it move up the backlog.

[sbansla]

I tested: client.bulkexports.v1.exports("Messages").days.list(). --> returns only 10 records. This issue seems to be specific to this api only.

[AsabuHere]

This issue has been added to our internal backlog to be prioritized. Pull requests and +1s on the issue summary will help it move up the backlog.

[bernardgardner]

Additional note here: setting limit=1000 results in 30 days being returned - thanks @chadlarsh for the note that setting page_size=10 works around this issue - the bug also impacts client.bulkexports.v1.exports("Messages").days.stream() with the same symptoms (same issue with exports("Calls") too) -

[bernardgardner]

This actually doesn't seem to be a python issue - I'm seeing the same behavior talking to the API via curl. Sending the request with no params to get defaults yields a response with next_page_url: null - supplying the query param ?PageSize=10 (or any other value between 2 and 30 inclusive) works as expected.

(twilio-audit) bernard.gardner@bernard twilio-audit % curl -X GET "https://bulkexports.twilio.com/v1/Exports/Messages/Days" \            
-u $TWILIO_ACCOUNT_SID:$TWILIO_AUTH_TOKEN | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1594  100  1594    0     0   6831      0 --:--:-- --:--:-- --:--:--  7022
{
  "meta": {
    "page": 0,
    "page_size": 50,
    "first_page_url": "https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=50&Page=0",
    "previous_page_url": null,
    "url": "https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=50&Page=0",
    "next_page_url": null,
    "key": "days"
  },
  "days": [
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-01",
      "resource_type": "Messages",
      "size": 1857
    },
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-02",
      "resource_type": "Messages",
      "size": 2172
    },
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-03",
      "resource_type": "Messages",
      "size": 3099
    },
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-04",
      "resource_type": "Messages",
      "size": 3582
    },
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-05",
      "resource_type": "Messages",
      "size": 4584
    },
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-06",
      "resource_type": "Messages",
      "size": 4625
    },
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-07",
      "resource_type": "Messages",
      "size": 4269
    },
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-08",
      "resource_type": "Messages",
      "size": 3504
    },
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-09",
      "resource_type": "Messages",
      "size": 2609
    },
    {
      "friendly_name": "Apr\\-Jun 2023",
      "create_date": "2023-10-24",
      "day": "2023-04-10",
      "resource_type": "Messages",
      "size": 3258
    }
  ]
}

(twilio-audit) bernard.gardner@bernard twilio-audit % for i in {1..50}
do                
echo -n "$i "; curl -s -X GET "https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=$i" -u $TWILIO_ACCOUNT_SID:$TWILIO_AUTH_TOKEN | jq -Mr .meta.next_page_url
done
1 null
2 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=2&Page=1&PageToken=ENMjAyMy0wNC0wMg%3D%3D
3 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=3&Page=1&PageToken=ENMjAyMy0wNC0wMw%3D%3D
4 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=4&Page=1&PageToken=ENMjAyMy0wNC0wNA%3D%3D
5 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=5&Page=1&PageToken=ENMjAyMy0wNC0wNQ%3D%3D
6 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=6&Page=1&PageToken=ENMjAyMy0wNC0wNg%3D%3D
7 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=7&Page=1&PageToken=ENMjAyMy0wNC0wNw%3D%3D
8 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=8&Page=1&PageToken=ENMjAyMy0wNC0wOA%3D%3D
9 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=9&Page=1&PageToken=ENMjAyMy0wNC0wOQ%3D%3D
10 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=10&Page=1&PageToken=ENMjAyMy0wNC0xMA%3D%3D
11 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=11&Page=1&PageToken=ENMjAyMy0wNC0xMQ%3D%3D
12 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=12&Page=1&PageToken=ENMjAyMy0wNC0xMg%3D%3D
13 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=13&Page=1&PageToken=ENMjAyMy0wNC0xMw%3D%3D
14 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=14&Page=1&PageToken=ENMjAyMy0wNC0xNA%3D%3D
15 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=15&Page=1&PageToken=ENMjAyMy0wNC0xNQ%3D%3D
16 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=16&Page=1&PageToken=ENMjAyMy0wNC0xNg%3D%3D
17 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=17&Page=1&PageToken=ENMjAyMy0wNC0xNw%3D%3D
18 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=18&Page=1&PageToken=ENMjAyMy0wNC0xOA%3D%3D
19 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=19&Page=1&PageToken=ENMjAyMy0wNC0xOQ%3D%3D
20 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=20&Page=1&PageToken=ENMjAyMy0wNC0yMA%3D%3D
21 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=21&Page=1&PageToken=ENMjAyMy0wNC0yMQ%3D%3D
22 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=22&Page=1&PageToken=ENMjAyMy0wNC0yMg%3D%3D
23 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=23&Page=1&PageToken=ENMjAyMy0wNC0yMw%3D%3D
24 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=24&Page=1&PageToken=ENMjAyMy0wNC0yNA%3D%3D
25 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=25&Page=1&PageToken=ENMjAyMy0wNC0yNQ%3D%3D
26 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=26&Page=1&PageToken=ENMjAyMy0wNC0yNg%3D%3D
27 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=27&Page=1&PageToken=ENMjAyMy0wNC0yNw%3D%3D
28 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=28&Page=1&PageToken=ENMjAyMy0wNC0yOA%3D%3D
29 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=29&Page=1&PageToken=ENMjAyMy0wNC0yOQ%3D%3D
30 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=30&Page=1&PageToken=ENMjAyMy0wNC0zMA%3D%3D
31 null
32 null
33 null
34 null
35 null
36 null
37 null
38 null
39 null
40 null
41 null
42 null
43 null
44 null
45 null
46 null
47 null
48 null
49 null
50 null