WhatsAppWrapped icon indicating copy to clipboard operation
WhatsAppWrapped copied to clipboard

Infinite loading when file is too large

Open andrsdt opened this issue 3 years ago • 0 comments

I'm trying to process this one-year conversation txt file:

9.5 MiB 156279 lines 9463900 characters

The website is stuck with a loading message and the following message appears in the console:

Uncaught (in promise) DOMException: Failed to execute 'setItem' on 'Storage': Setting the value of 'WAWData' exceeded the quota.
    at https://whatsappwrapped.tech/js/script.js:86:20

I have been reading about sessionStorage and localStorage and their capacity seems to be limited to 5MB each on most browsers (I have tried Chrome, Firefox, Opera and Brave without success).

An ugly workaround that would work would be to split the JSON string in two parts and store a half in localStorage and another half in sessionStorage, like this:

script.js

// (...)
const half = toStore.length/2;
const toStore1 = toStore.substring(0, half);
const toStore2 = toStore.substring(half, toStore.length);
sessionStorage.setItem('WAWData1', toStore1)
localStorage.setItem('WAWData2', toStore2)
// (...)

wrap.js

google.charts.load('current', {'packages':['corechart', 'bar']});
google.charts.setOnLoadCallback(drawChart);
data1 = sessionStorage.getItem('WAWData1')
data2 = localStorage.getItem('WAWData2')
fullData = data1.concat(data2)
JSONfullData = JSON.parse(fullData)
const {
  user1,
  user2,
  firstName1,
  firstName2,
  numMessages,
  messagesPerPerson,
  wordCountTotal,
  topThreeEmojisPerPerson,
  currentStreak,
  mostActiveHour,
} = JSONfullData || '{}'
let lastWidth;
let lastCloudWidth;
// (...)

This way, the max size would be ~10MB. There must be a cleaner way to achieve this anyway.

andrsdt avatar Dec 31 '21 20:12 andrsdt