InstaLooter
InstaLooter copied to clipboard
'rhx_gis' is written to the console, then nothing happens.
Environment: Instalooter 2.3.4, Windows , Python 3.7
Test: instaLooter user <username> <directory>
That outputs 'rhx_gis' (including quotes) and does not seem to do much else. Nothing is downloaded. IG user name does not matter, the error is reproducible with any user.
The issue came up a couple days ago without any changes in the environment apart from Windows updates.
Difficult to debug (for me :) because the program does not really complain, no stack trace, no descriptive error message. I did find 'rhx_gis' in two places in the code, but to understand more I'd have to burrow into the code for some hours. Any help or hints for debugging is much appreciated.
All was working fine few months ago, but now Instalooter doesn't run. The error which i get in console is:
self.rhx = get_shared_data(res.text)['rhx_gis'] KeyError: u'rhx_gis'.
I am using Instalooter 2.3.4, python 2.7 and python 3.7
Any idea what is the root cause of this error?
Same here both on macOS 10.13.6 and on Debian GNU/Linux 9.8 Instalooter version is 2.3.4
Debian: Python 2.7.13 and 3.5.3 macOS: Python 2.7.16 and 3.6.8
the only error Message given out by both systems is:
2019-05-17 19:28:04 macOS-machine instalooter.cli[14350] CRITICAL 'rhx_gis'
or
2019-05-17 19:29:09 Debian-machine instalooter.cli[3119] CRITICAL u'rhx_gis'
The only difference is that the Debian machine adds another u before 'rhx_gis'
Output from a instalooter post <url> --traceback
is:
CRITICAL 'rhx_gis'
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/instalooter/cli/__init__.py", line 146, in main
extended_dump=args['--extended-dump']
File "/usr/local/lib/python3.7/site-packages/instalooter/looters.py", line 764, in __init__
super(PostLooter, self).__init__(**kwargs)
File "/usr/local/lib/python3.7/site-packages/instalooter/looters.py", line 280, in __init__
self.rhx = get_shared_data(res.text)['rhx_gis']
KeyError: 'rhx_gis'
MacOS, instalooter 2.3.4. Was working fine, oh, 5 days ago?
KeyError: 'rhx_gis'
Well I am more of a Java guy, but AFAIK Python dictionaries raise a KeyError when the specified key is absent, so we only need to determine why :)
A few days ago, Instagram stopped providing "rhx_gis" token in "window._sharedData". Other Instagram Scrapers ( https://github.com/rarcega/instagram-scraper/issues/377 ) have fixed the issue by setting rhx_gis to an empty string, aka: self.rhx = "" (Line 288 of looters.py). However when I tried doing that in looters.py, it wouldn't work. I'm not very familiar with the looters.py script, so pretty sure I'm doing something wrong.
A few days ago, Instagram stopped providing "rhx_gis" token in "window._sharedData".
It is probably too much to wish for, but I would appreciate it if the script could check all keys for existence and in case one is missing output something like »Key "foo" missing in ... This key is necessary for ... Aborting execution.«
If someone need this, i found solution of this problem. 2 files must be edited:
~/.local/lib/python2.7/site-packages/instalooter/looters.py
- append symbol '#' at start of 281 line
~/.local/lib/python2.7/site-packages/instalooter/pages.py
- 172 line:
return cls(data['id', session, user_data['rhx_gis'])
must be replaced by:
return cls(data['id', session, "")
And don't forget about spaces (not delete it, just replace text)
If someone need this, i found solution of this problem. 2 files must be edited:
~/.local/lib/python2.7/site-packages/instalooter/looters.py
- append symbol '#' at start of 281 line~/.local/lib/python2.7/site-packages/instalooter/pages.py
- 172 line:return cls(data['id', session, user_data['rhx_gis'])
must be replaced by:return cls(data['id', session, "")
And don't forget about spaces (not delete it, just replace text)
Meaning of this is removing all "rhx_gis" variables from code.
You need to comment line 288 of looters.py aswell, to get rid of this
self.rhx = get_shared_data(res.text)['rhx_gis']
You need to comment line 288 of looters.py aswell, to get rid of this
self.rhx = get_shared_data(res.text)['rhx_gis']
Hm, in my case it was 281 line. But you're right: text, that you wrote, must be commented.
Are you using the latest git? On the latest git in looters.py that line is 288.
If someone need this, i found solution of this problem. 2 files must be edited:
~/.local/lib/python2.7/site-packages/instalooter/looters.py
- append symbol '#' at start of 281 line~/.local/lib/python2.7/site-packages/instalooter/pages.py
- 172 line:return cls(data['id', session, user_data['rhx_gis'])
must be replaced by:return cls(data['id', session, "")
And don't forget about spaces (not delete it, just replace text)Meaning of this is removing all "rhx_gis" variables from code.
Awesome! Great fix!
I'll add that in my case, I also needed to modify line 738 of looters.py to: return HashtagIterator(self._hashtag, self.session, "")
But to your point, removing all "rhx_gis"variables does the trick!
Edit: Nevermind, this worked, and then when running it a few hours later, it began erroring out with retry errors.
Encountering this same issue, and don't really want to dive in and start editing config files if it's not actually going to result in a fix. It sounds like the issue is on IG's end.
The fixes to the lines in looters.py
and pages.py
worked initially, but now I'm getting:
CRITICAL RetryError[<Future at 0x109d41710 state=finished raised KeyError>]
Traceback (most recent call last):
File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/instalooter/cli/__init__.py", line 173, in main
dlpbar_cls=None if args['--quiet'] else TqdmProgressBar)
File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/instalooter/looters.py", line 861, in download
worker.run()
File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/instalooter/worker.py", line 119, in run
self._DOWNLOAD_METHODS[media["__typename"]](media)
File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/tenacity/__init__.py", line 292, in wrapped_f
return self.call(f, *args, **kw)
File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/tenacity/__init__.py", line 358, in call
do = self.iter(retry_state=retry_state)
File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/tenacity/__init__.py", line 332, in iter
six.raise_from(retry_exc, fut.exception())
File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/six.py", line 737, in raise_from
raise value
RetryError: RetryError[<Future at 0x109d41710 state=finished raised KeyError>]
I am doing one-off calls to instalooter post <post-url>
Instagram-scraper has a more active dev then instalooter, and works currently,might as well swap over at this point. https://github.com/rarcega/instagram-scraper
If someone need this, i found solution of this problem. 2 files must be edited:
@SirobbsCrafter's patch worked for me. Might fail when using some additional options, as I have no idea what is supposed to be stored under this key, but my calls work fine.
@althonos, I think this should be elevated to a bug.
The fixes to the lines in
looters.py
andpages.py
worked initially, but now I'm getting:CRITICAL RetryError[<Future at 0x109d41710 state=finished raised KeyError>] Traceback (most recent call last): File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/instalooter/cli/__init__.py", line 173, in main dlpbar_cls=None if args['--quiet'] else TqdmProgressBar) File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/instalooter/looters.py", line 861, in download worker.run() File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/instalooter/worker.py", line 119, in run self._DOWNLOAD_METHODS[media["__typename"]](media) File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/tenacity/__init__.py", line 292, in wrapped_f return self.call(f, *args, **kw) File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/tenacity/__init__.py", line 358, in call do = self.iter(retry_state=retry_state) File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/tenacity/__init__.py", line 332, in iter six.raise_from(retry_exc, fut.exception()) File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/six.py", line 737, in raise_from raise value RetryError: RetryError[<Future at 0x109d41710 state=finished raised KeyError>]
I am doing one-off calls to
instalooter post <post-url>
I have a same error when add '-v' parameter (download videos too). I not found the solution yet. Maybe, Instagram changed something again and app shutdownes with error (same as with 'rhx_gis').
I don't know if this is the right way, but I prepared a PR which works for me: https://github.com/althonos/InstaLooter/pull/248
Maybe rhx_gis is not needed anymore, and the code could be simplified.
Instagram-scraper has a more active dev then instalooter, and works currently,might as well swap over at this point. https://github.com/rarcega/instagram-scraper
Instagram-Scraper sometimes download image files of less than the best quality.
You need to comment line 288 of looters.py aswell, to get rid of this
self.rhx = get_shared_data(res.text)['rhx_gis']
Hm, in my case it was 281 line. But you're right: text, that you wrote, must be commented.
Line 280 for me, working just fine after that!
returning empty rhx_gis
worked for video option in my case. but sometimes I get CRITICAL u'edge_media_to_comment'
error
The fixes to the lines in
looters.py
andpages.py
worked initially, but now I'm getting:CRITICAL RetryError[<Future at 0x109d41710 state=finished raised KeyError>] Traceback (most recent call last): File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/instalooter/cli/__init__.py", line 173, in main dlpbar_cls=None if args['--quiet'] else TqdmProgressBar) File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/instalooter/looters.py", line 861, in download worker.run() File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/instalooter/worker.py", line 119, in run self._DOWNLOAD_METHODS[media["__typename"]](media) File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/tenacity/__init__.py", line 292, in wrapped_f return self.call(f, *args, **kw) File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/tenacity/__init__.py", line 358, in call do = self.iter(retry_state=retry_state) File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/tenacity/__init__.py", line 332, in iter six.raise_from(retry_exc, fut.exception()) File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/six.py", line 737, in raise_from raise value RetryError: RetryError[<Future at 0x109d41710 state=finished raised KeyError>]
I am doing one-off calls to
instalooter post <post-url>
Same here. Any updates?
This is most likely caused by yet another change in the private data structure used to store media metadata. I'll give it a look, but this may be super easy or super hard to fix, depending on how much the schema changed.
Any updates on this issue?
met the same issue "RetryError" after setting "rhx_gis" to an empty string, the traceback says "KeyError: 'edge_media_to_comment'"
is this project abandoned?
Release 2.4.0
has a shallow fix for this issue thanks to @tgandor.
I suspect the edge_media_to_comment
error to be related to query limits as well: I'll check that and wrap it in a better error if that is the case.
Version 2.4.0 with Python 3.7.4 has solved my issues with this problem.