snarfed
batch Twitter searches
facebook and google+ both support batching API requests. facebook can even parameterize requests in the batch on the results of earlier requests in the same batch.
full polls for FB and G+ both take a fair number of requests, so this would definitely help.
https://developers.facebook.com/docs/reference/api/batch/ https://developers.google.com/api-client-library/python/guide/batch
deferring for now in favor of #13, which overlaps and will be better for cutting down effective API calls from the silos' perspective.
did G+ in snarfed/activitystreams-unofficial@dcb651b154c9c109146b67de33e5e496933b9332
interesting. i'm not sure this changed average G+ poll latency much, but it seems like it brought the tail (90%ile, etc) down. average went from ~2s to 1-2s, but 90%ile seemed to go from >10s to ~2.5s. so that's something at least.
https://console.developers.google.com/project/apps~brid-gy/appengine/logs?fromac=true&filters=path:%2F_ah%2Fqueue%2Fpoll&filters=regex:%2Fgoogleplus%2F
i also wonder if i could batch some twitter API calls, indirectly, by combining the search queries i use to find @-replies. i currently make one search call per user, but i could probably batch them with the OR operator.
motivated by #280, twitter's search API hanging more often.
we actually get facebook comments and likes for free, without extra API calls, so batching isn't as important for FB. we do make one extra call each to get photos and rsvps, and one call per (upcoming) event created by the user, but common case that's not much.
as for twitter, it looks like the OR search operator would work. we'd still have to make multiple search calls, since we have to recurse to follow chains all the way, but it'd help. it'd take some work though, and i don't know how important it is. deprioritizing for now.
i got halfway through porting Facebook.get_activities_response() in activitystreams-unofficial to the batch API, and it just wasn't working. it added too much complexity. we can revisit it later - i have it in a stash - but right now it just wasn't feeling worth the performance improvement.