fastyle
fastyle copied to clipboard
CopperBadger
Possibility to get all Download URLs for favorites?
Just rather curios, since i think it would be quite useful sometime sto just dump all your faves to the computer. At least I had the need to do so.
Currently I think there'd at least be one AJAX call to the submission page, referenced trough a submission anchor, to obtain the Download URL from there.
Thats possible, right? o.o
Ooooh, that'd be a wonderful feature to have! :D
Your intuition sounds right, since I don't think you can get to the content itself just from an anchor on the Favorites page (judging from what I gathered from this reddit post). You'd have to make an AJAX call on that anchor and extract the content URI from what the call returns. The problem you then run into, though, is actually compiling all of the content gathered from those AJAX calls and saving it into a compressed folder for the user to download. I came across this, though, which might be useful! ^^
Downloading all favorites is theoretically possible, but you'd run into the issue of putting a lot of load on FA's servers. Recall the announcement posted a while back that banned the use of the Unofficial Android app and the extension that made your notifications update with AJAX. These both made numerous requests to the server at once and were, therefore, banned.
I would worry if FAStyle would suffer the same fate if this were implemented. Perhaps if we made it download them one at a time with a hard-coded time delay, it wouldn't hurt the servers. So, instead of it being a one click thing, make it open up a page (or a Bootstrap modal?) with a progress bar that says it's generating your file.
Kobi's right. I actually chatted with Dragoneer last night, he says that the infinite scroll feature alone might be too much if we don't build a server-responsive time offset into it. Unfortunately, while it certainly could be done, it would be too much for the servers to take in widespread use.
I was talking to my boyfriend (who is also a programmer) and we were thinking that we could theoretically write something on a separate server using PHP to download favorites directly from the CDN. If we can get the full download URL without querying the /view/###### page, we wouldn't be causing extra Database queries. That, in combination with a time delay, could greatly reduce the load on FA.
One of the challenges we would have to overcome as well is that download images don't have predictable names and extensions, which might not make it possible to get the name without querying the webpage. I'll check with him to see if it would be permissible-- like you said, I want to make sure we don't do anything that would have our hard work blocked.
Dear lord. I thought FA had a server farm? And then they cant handle a few more requests? Do they not use frontend caching? xD That is why I hope they will release some API in the future…that would deliver CDN links instead, since a CDN is usually more workload proof.
Downloading one at a time would be one thing, but I am thinking a bit different here, kinda putting ideas together: If we hosted an external service, to which the „make my faves“ request goes, with all the submission links, we could use a php or nodejs app that puts all the stuff into a database and works down on it. And if two images match, we can use an in-service cache to deliver the images instead of requesting it from FA.
That might lead to you needing a big HDD. But if you think about it, over time, the favorited images my duplicate, hence reducing the load of downloads needed.
That at least is just an idea… o.o
Am 25.01.2015 um 16:17 schrieb Copper Badger [email protected]:
One of the challenges we would have to overcome as well is that download images don't have predictable names and extensions, which might not make it possible to get the name without querying the webpage. I'll check with him to see if it would be permissible-- like you said, I want to make sure we don't do anything that would have our hard work blocked.
— Reply to this email directly or view it on GitHub https://github.com/CopperBadger/fastyle/issues/46#issuecomment-71376901.