pynder icon indicating copy to clipboard operation
pynder copied to clipboard

Q: Selective Downloading of Data Subsets

Open SHABINTHEGREAT opened this issue 8 years ago • 8 comments

Is it possible to download only a subset or portion of the data? For example, the data only for:

  • one matched user : session.matches(i)
  • only the ping_time data – for one user or all users : session.matches(i).user.ping_time or session.matches().ping_time
  • only the messages – for one user or all users : session.matches(i).messages or session.matches().messages

instead of downloading the whole dataset of all users? : session.matches()

Given that the tinder app does not download all the data every time, and the tinder API does not seem to require downloading all the data, is it possible to download only a small subset?

Thank you!

SHABINTHEGREAT avatar Apr 22 '16 05:04 SHABINTHEGREAT

I also agree that this would be useful. I'm currently running into an issue where session.matches() is timing out with a 504 error because I have too many matches (would be my guess).

rdrapeau avatar Apr 25 '16 18:04 rdrapeau

:+1: This is a good idea.

charliewolf avatar Apr 25 '16 18:04 charliewolf

I have 1300 total matches. Here is the output:

  matches = session.matches()

  File "/Library/Python/2.7/site-packages/pynder-0.0.10-py2.7.egg/pynder/session.py", line 32, in matches
    return [models.Match(m, self) for m in self._api.matches()]
  File "/Library/Python/2.7/site-packages/pynder-0.0.10-py2.7.egg/pynder/models/user.py", line 126, in __init__
    match['person']['_id'])['results']
  File "/Library/Python/2.7/site-packages/pynder-0.0.10-py2.7.egg/pynder/api.py", line 84, in user_info
    return self._get("/user/" + user_id)
  File "/Library/Python/2.7/site-packages/pynder-0.0.10-py2.7.egg/pynder/api.py", line 47, in _get
    return self._request("get", url)
  File "/Library/Python/2.7/site-packages/pynder-0.0.10-py2.7.egg/pynder/api.py", line 43, in _request
    raise errors.RequestError(result.status_code)
pynder.errors.RequestError: 504

Or could it be that the request times out... due to any proxy servers in between which time out?

I'd say that is the more likely case.

rdrapeau avatar Apr 26 '16 04:04 rdrapeau

I think we can fix this by caching + lazyloading user data

charliewolf avatar Apr 26 '16 04:04 charliewolf

The ability to download only a subset or portion of the data would make @rdrapeau's issue a non-issue. Plus, that will enable even more features and abilities for Pynder. Thoughts?

SHABINTHEGREAT avatar Apr 26 '16 05:04 SHABINTHEGREAT

I don't have time right now, but perhaps in the future, or if somebody else wants to open a PR they're more than welcome.

charliewolf avatar May 11 '16 18:05 charliewolf

See also #72 and #70

charliewolf avatar Jul 24 '16 02:07 charliewolf

@SHABINTHEGREAT, @charliewolf

This is a very good idea, Shabin. As a some sort of workaround to manipulate matches data I load session._api.matches() instead of loading session.matches() because the latter takes ages to load +200 matches whereas the former loads pretty fast. But the one problem I have found is that I haven't figured it out how to send a message to a match using the data downloaded from session._api.matches().

roperi avatar Oct 09 '16 11:10 roperi