query-analyzer-app icon indicating copy to clipboard operation
query-analyzer-app copied to clipboard

Analyze File fails for large log files

Open zolstein opened this issue 7 months ago • 0 comments

The Analyze File button runs the following function:

    function loadFile() {

        let logFile = document.getElementById("queryLogFile").files[0];
        if (logFile && logFile != null) {
            msg.innerHTML = "Start Reading file";
            let reader = new FileReader();
            reader.onload = function(e) {
                let text = reader.result;
                msg.innerHTML = "Reading File Finished, start analyzing";
                qan.analyseFile(text);

            }
            reader.readAsText(logFile);
        }
    }

FileReader.readAsText is not designed to read large files. In Chrome, if the file is larger than ~260MB, rather than the file contents it returns an empty-string. Larger files (~2GB, in my usage) seem to just not trigger the onload function. This seems like it should use a reading method that can incrementally stream data from the file, and the analyzeFile method should be changed to allow it to process the file in parts.

zolstein avatar Jul 26 '24 00:07 zolstein