excelize icon indicating copy to clipboard operation
excelize copied to clipboard

Exponential grown worktime of StreamWriter

Open IAkumaI opened this issue 3 years ago • 1 comments

Description

I use StreamWriter for generate large xlsx files. It's just text files: no formatting/styles.

// about 100 columns with different data. But data size does not seriously matter in my tests
dataRow := []interface{}{"Some", "Cell", "Values", "and 100 more"}

// Do not check errors in issue
excFile := excelize.NewFile()
streamWriter, _ := excFile.NewStreamWriter("Sheet0")
for rowNum := 1; rowNum < 50000; rowNum++ {
  streamWriter.SetRow("A"+strconv.Itoa(rowNum), dataRow)
}

streamWriter.Flush()
file, _ := os.Create("some.xlsx")
excFile.Write(file)

In code example we create excel file with 50 000 rows with StreamWriter. All code will consume about 1.5 minute on my server. But if rows count increased to 100 000 in will consume not (2 * 1.5) min, but about 10 minutes. If there are million rows, it takes about 1.5 hours.

Time increates not linear. Is there any way to make time more linear and speed up much rows creation?

Output of go version:

1.17.2

Excelize version or commit ID:

github.com/xuri/excelize/v2 v2.4.1

Environment details (OS, Microsoft Excel™ version, physical, etc.): Ubuntu x86_64 with SSD drive.

IAkumaI avatar Dec 22 '21 15:12 IAkumaI

Thanks for your issue, here is some performance figures for your reference: Performance Figures.

xuri avatar Dec 23 '21 02:12 xuri