easyexcel icon indicating copy to clipboard operation
easyexcel copied to clipboard

图片导出到本地可以边读边写防止OOM吗?

Open hausen1012 opened this issue 1 year ago • 7 comments

一次性导入比较多的图片,图片单个大小不大,但是数量比较多。可以边读边写防止 OOM吗?

hausen1012 avatar Jan 30 '24 06:01 hausen1012

可以分成多个小批次去写入嘛,写入一批就释放一批,然后继续写下一批,避免OOM的大概思路。

LSL1618 avatar Feb 01 '24 01:02 LSL1618

目前就是分页查询再写入的,但是还是有问题呢。

try ( ExcelWriter excelWriter = EasyExcel.write(response.getOutputStream(), ReportExportVO.class)
                .includeColumnFieldNames(includeColumnFiledNames).build();){
            // 写入概览数据
            writeSummaryData(excelWriter, taskCode, taskReportSummary);
            // 按路线划分 sheet
            routes.forEach((routeExecRecordDTO -> {
                Long routeId = routeExecRecordDTO.getRouteId();
                String routeName = routeExecRecordDTO.getRouteName();
                WriteSheet writeSheet = EasyExcel.writerSheet(routeName + "_" + routeId)
                        .registerWriteHandler(new CustomRowHeightCellWriteHandler((short) 2600)).build();
                int current = 1;
                int size = 100;
                while (true) {
                    PatrolChildPositionReportPageVO pageVO = new PatrolChildPositionReportPageVO();
                    pageVO.setRecordCode(taskCode).setRouteId(routeId).setSize(size);
                    pageVO.setCurrent(current++);
                    List<ChildPositionExecRecordDTO> records = getTaskReportDetail(pageVO).getRecords();
                    List<ReportExportVO> reportExportVOList = convertToReportExportVO(records, isMonitor);
                    excelWriter.write(reportExportVOList, writeSheet);
                    if(records.size() < size){
                        break;
                    }
                }
            }));

使用 getTaskReportDetail 一次查询100条数据。请问这个代码哪里有问题呢

hausen1012 avatar Feb 01 '24 01:02 hausen1012

目前就是分页查询再写入的,但是还是有问题呢。

try ( ExcelWriter excelWriter = EasyExcel.write(response.getOutputStream(), ReportExportVO.class)
                .includeColumnFieldNames(includeColumnFiledNames).build();){
            // 写入概览数据
            writeSummaryData(excelWriter, taskCode, taskReportSummary);
            // 按路线划分 sheet
            routes.forEach((routeExecRecordDTO -> {
                Long routeId = routeExecRecordDTO.getRouteId();
                String routeName = routeExecRecordDTO.getRouteName();
                WriteSheet writeSheet = EasyExcel.writerSheet(routeName + "_" + routeId)
                        .registerWriteHandler(new CustomRowHeightCellWriteHandler((short) 2600)).build();
                int current = 1;
                int size = 100;
                while (true) {
                    PatrolChildPositionReportPageVO pageVO = new PatrolChildPositionReportPageVO();
                    pageVO.setRecordCode(taskCode).setRouteId(routeId).setSize(size);
                    pageVO.setCurrent(current++);
                    List<ChildPositionExecRecordDTO> records = getTaskReportDetail(pageVO).getRecords();
                    List<ReportExportVO> reportExportVOList = convertToReportExportVO(records, isMonitor);
                    excelWriter.write(reportExportVOList, writeSheet);
                    if(records.size() < size){
                        break;
                    }
                }
            }));

使用 getTaskReportDetail 一次查询100条数据。请问这个代码哪里有问题呢

是有报错还是导出的文件数据与预期不符???

LSL1618 avatar Feb 01 '24 08:02 LSL1618

目前就是分页查询再写入的,但是还是有问题呢。

try ( ExcelWriter excelWriter = EasyExcel.write(response.getOutputStream(), ReportExportVO.class)
                .includeColumnFieldNames(includeColumnFiledNames).build();){
            // 写入概览数据
            writeSummaryData(excelWriter, taskCode, taskReportSummary);
            // 按路线划分 sheet
            routes.forEach((routeExecRecordDTO -> {
                Long routeId = routeExecRecordDTO.getRouteId();
                String routeName = routeExecRecordDTO.getRouteName();
                WriteSheet writeSheet = EasyExcel.writerSheet(routeName + "_" + routeId)
                        .registerWriteHandler(new CustomRowHeightCellWriteHandler((short) 2600)).build();
                int current = 1;
                int size = 100;
                while (true) {
                    PatrolChildPositionReportPageVO pageVO = new PatrolChildPositionReportPageVO();
                    pageVO.setRecordCode(taskCode).setRouteId(routeId).setSize(size);
                    pageVO.setCurrent(current++);
                    List<ChildPositionExecRecordDTO> records = getTaskReportDetail(pageVO).getRecords();
                    List<ReportExportVO> reportExportVOList = convertToReportExportVO(records, isMonitor);
                    excelWriter.write(reportExportVOList, writeSheet);
                    if(records.size() < size){
                        break;
                    }
                }
            }));

使用 getTaskReportDetail 一次查询100条数据。请问这个代码哪里有问题呢

是有报错还是导出的文件数据与预期不符???

OOM了

hausen1012 avatar Feb 01 '24 08:02 hausen1012

一个ReportExportVO对象包含几张图?你试着再缩小一次写入的数据条数,然后每次写入后释放一下缓存excelWriter.finish()但不要关闭流。

LSL1618 avatar Feb 02 '24 07:02 LSL1618

@LSL1618

一个ReportExportVO对象包含几张图?你试着再缩小一次写入的数据条数,然后每次写入后释放一下缓存excelWriter.finish()但不要关闭流。

finish了之后就不能写了 @LSL1618

zivfuture avatar Jun 26 '24 11:06 zivfuture

@zivfuture 我找了一下竟然没有flush()方法,finish()方法确实不能用,要不然直接就关闭workbook,关闭流了。

LSL1618 avatar Jun 27 '24 03:06 LSL1618