stream icon indicating copy to clipboard operation
stream copied to clipboard

Export stops at 10,000 items

Open cbirdsong opened this issue 1 year ago • 3 comments

Bug Report

Expected Behavior

A CSV export should contain all the data Stream has.

Actual Behavior

The export is cut off at 10,000 items.

Steps to Reproduce the Problem

  1. Set a high-traffic site to retain data for enough time to log more than 10,000 items.
  2. Wait until there are more than 10,000 items.
  3. Export to CSV or JSON
  4. Open the exported file. You should find only 10,000 rows or items.

I have only seen this on one site

System Information

  • Stream plugin version: 4.0.0
  • WordPress version: 6.4.3
  • PHP version: 7.4
  • Browser: Firefox
  • Computer operating system: Mac OS

This would be largely sidestepped by fixing #1191

cbirdsong avatar Apr 02 '24 18:04 cbirdsong

It looks like this should be an easy fix. From class-export.php:

/**
* Increase pagination limit for CSV Output
*
* @param int $records_per_page Old limit for records_per_page.
* @return int
*/
public function disable_paginate( $records_per_page ) {
    return 10000;
}

I have no idea what a sensible number to cap it at would be, though. The site I discovered this on has 164,000 records and counting and Stream otherwise works fine with that volume, so probably higher than that?

cbirdsong avatar Apr 02 '24 19:04 cbirdsong

I have no idea what a sensible number to cap it at would be, though. The site I discovered this on has 164,000 records and counting and Stream otherwise works fine with that volume, so probably higher than that?

apply_filters() makes sense here, so everybody can have it according to their environment capability.

lkraav avatar Apr 03 '24 11:04 lkraav

Support forum topic: https://wordpress.org/support/topic/csv-exported-maximum-10000/

tharsheblows avatar Jul 16 '24 15:07 tharsheblows