forget
forget copied to clipboard
New component for uploading archives
Part of #64 Blocked by #458
- Present itself as a large "Import archive" button, which opens a file picker and accepts either a
tweet.js
file or a zip file - Also a drop zone
- If given a zip file, looks for a
tweet.js
file inside it. zip.js seems deece.- might be good to load zip.js on demand when a zip file is given
- Be able to parse a
tweet.js
, extract the useful metadata for each tweet, and upload it in batches to the endpoint in #458 - Some live progress reporting would be good
Also needs some copy (not inside the component) explaining what the archive is and does, and that the zip file won't be uploaded but processed locally. Some of the old copy can probably be reused.
I will not be making a no-JS plain-html alternative, because I absolutely do not want people to accidentally upload their whole archive to me. I don't want your DMs.
Of note is that the same component could potentially support ActivityPub archives in the future. Though AP archives aren't quite as straightforward to work with.
observation from the archive i ordered yesterday and got today:
I haven't seen an archive with more than one tweet.js
file but there also exists a manifest.js
file that seems to imply that more could exist, and that it would catalog them:
window.__THAR_CONFIG = {
"dataTypes" : {
"tweet" : {
"mediaDirectory" : "data/tweet_media",
"files" : [ {
"fileName" : "data/tweet.js",
"globalName" : "YTD.tweet.part0",
"count" : "495"
} ]
}
}
}
(truncated, there is much more in manifest.js
)
e: i should also include the structure of tweet.js
:
window.YTD.tweet.part0 = [ {
"tweet" : {
"retweeted" : false,
"source" : "<a href=\"https://mobile.twitter.com\" rel=\"nofollow\">Twitter Web App</a>",
"entities" : {
"hashtags" : [ ],
"symbols" : [ ],
"user_mentions" : [ ],
"urls" : [ ]
},
"display_text_range" : [ "0", "20" ],
"favorite_count" : "0",
"id_str" : "1364581575388708865",
"truncated" : false,
"retweet_count" : "0",
"id" : "1364581575388708865",
"created_at" : "Wed Feb 24 14:22:54 +0000 2021",
"favorited" : false,
"full_text" : "ah, my notifications",
"lang" : "en"
}
}, ... ]
far as i can tell, the contents of window.YTD.tweet.part0[x].tweet
are identical to what the API would return. It may be better to send the raw tweet object to the server and let the existing code there deal with it.. but that would also mean sending a lot of extraneous data that the server doesn't need and would potentially make the import process bandwidth-bound