QXlsx icon indicating copy to clipboard operation
QXlsx copied to clipboard

Handling Excel file with a large amount of rows

Open PMime opened this issue 1 year ago • 1 comments

I am just experimenting with the QXlsx-Library concerning a large amount of rows.

Code:

QXlsx::Document* oDocument = new QXlsx::Document();

for (int column = 1; column <= 52; column++)
{
	int maxNumber = 1000;
	int randomValue = 0;
	QString valueString;

	for (int row = 1; row <= 250000; row++)
	{
		randomValue = qrand() % maxNumber;
		valueString = QString::number(randomValue);

		oDocument->write(row, column, valueString);
	}
}

oDocument->saveAs("C:\\Temp\\big.xlsx");
delete oDocument;
oDocument = nullptr;

Using Win32 is very limited in the number of rows, the memory allocation fails. In addition the runtime is very large.

rows       success
------------------
100000      NO
50000       NO
25000       YES     <-- 5936 KB

x64 supports a larger amount of memory, but the performance time is critical, too.

rows       success     time
---------------------------
100000      YES        178 s
150000      YES        299 s
250000      YES        536 s     <-- 59439 KB

Is there a way to handle large amounts of rows in the QXlsx-Library in less time an without memory allocation faults? Any help would be appreciated.

PMime avatar Oct 04 '22 13:10 PMime

I can merge patches if you manage to improve this, but since this isn't a problem for me and probably others I don't have the time to review and try to optimize it.

dantti avatar Oct 04 '22 13:10 dantti