InstaLooter icon indicating copy to clipboard operation
InstaLooter copied to clipboard

'rhx_gis' is written to the console, then nothing happens.

Open haltefest opened this issue 5 years ago • 28 comments

Environment: Instalooter 2.3.4, Windows , Python 3.7

Test: instaLooter user <username> <directory>

That outputs 'rhx_gis' (including quotes) and does not seem to do much else. Nothing is downloaded. IG user name does not matter, the error is reproducible with any user.

The issue came up a couple days ago without any changes in the environment apart from Windows updates.

Difficult to debug (for me :) because the program does not really complain, no stack trace, no descriptive error message. I did find 'rhx_gis' in two places in the code, but to understand more I'd have to burrow into the code for some hours. Any help or hints for debugging is much appreciated.

haltefest avatar May 17 '19 09:05 haltefest

All was working fine few months ago, but now Instalooter doesn't run. The error which i get in console is:

self.rhx = get_shared_data(res.text)['rhx_gis'] KeyError: u'rhx_gis'.

I am using Instalooter 2.3.4, python 2.7 and python 3.7

Any idea what is the root cause of this error?

sohi05 avatar May 17 '19 10:05 sohi05

Same here both on macOS 10.13.6 and on Debian GNU/Linux 9.8 Instalooter version is 2.3.4

Debian: Python 2.7.13 and 3.5.3 macOS: Python 2.7.16 and 3.6.8

the only error Message given out by both systems is:

2019-05-17 19:28:04 macOS-machine instalooter.cli[14350] CRITICAL 'rhx_gis'

or

2019-05-17 19:29:09 Debian-machine instalooter.cli[3119] CRITICAL u'rhx_gis'

The only difference is that the Debian machine adds another u before 'rhx_gis'

Okona avatar May 17 '19 17:05 Okona

Output from a instalooter post <url> --traceback is:

 CRITICAL 'rhx_gis'
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/instalooter/cli/__init__.py", line 146, in main
    extended_dump=args['--extended-dump']
  File "/usr/local/lib/python3.7/site-packages/instalooter/looters.py", line 764, in __init__
    super(PostLooter, self).__init__(**kwargs)
  File "/usr/local/lib/python3.7/site-packages/instalooter/looters.py", line 280, in __init__
    self.rhx = get_shared_data(res.text)['rhx_gis']
KeyError: 'rhx_gis'

MacOS, instalooter 2.3.4. Was working fine, oh, 5 days ago?

artlung avatar May 17 '19 18:05 artlung

KeyError: 'rhx_gis'

Well I am more of a Java guy, but AFAIK Python dictionaries raise a KeyError when the specified key is absent, so we only need to determine why :)

haltefest avatar May 17 '19 19:05 haltefest

A few days ago, Instagram stopped providing "rhx_gis" token in "window._sharedData". Other Instagram Scrapers ( https://github.com/rarcega/instagram-scraper/issues/377 ) have fixed the issue by setting rhx_gis to an empty string, aka: self.rhx = "" (Line 288 of looters.py). However when I tried doing that in looters.py, it wouldn't work. I'm not very familiar with the looters.py script, so pretty sure I'm doing something wrong.

andrewhogan93 avatar May 17 '19 20:05 andrewhogan93

A few days ago, Instagram stopped providing "rhx_gis" token in "window._sharedData".

It is probably too much to wish for, but I would appreciate it if the script could check all keys for existence and in case one is missing output something like »Key "foo" missing in ... This key is necessary for ... Aborting execution.«

haltefest avatar May 17 '19 20:05 haltefest

If someone need this, i found solution of this problem. 2 files must be edited:

~/.local/lib/python2.7/site-packages/instalooter/looters.py - append symbol '#' at start of 281 line
~/.local/lib/python2.7/site-packages/instalooter/pages.py - 172 line: return cls(data['id', session, user_data['rhx_gis']) must be replaced by: return cls(data['id', session, "") And don't forget about spaces (not delete it, just replace text)

SirobbsCrafter avatar May 18 '19 17:05 SirobbsCrafter

If someone need this, i found solution of this problem. 2 files must be edited:

~/.local/lib/python2.7/site-packages/instalooter/looters.py - append symbol '#' at start of 281 line ~/.local/lib/python2.7/site-packages/instalooter/pages.py - 172 line: return cls(data['id', session, user_data['rhx_gis']) must be replaced by: return cls(data['id', session, "") And don't forget about spaces (not delete it, just replace text)

Meaning of this is removing all "rhx_gis" variables from code.

SirobbsCrafter avatar May 18 '19 17:05 SirobbsCrafter

You need to comment line 288 of looters.py aswell, to get rid of this self.rhx = get_shared_data(res.text)['rhx_gis']

BobMacpherson avatar May 18 '19 17:05 BobMacpherson

You need to comment line 288 of looters.py aswell, to get rid of this self.rhx = get_shared_data(res.text)['rhx_gis']

Hm, in my case it was 281 line. But you're right: text, that you wrote, must be commented.

SirobbsCrafter avatar May 18 '19 18:05 SirobbsCrafter

Are you using the latest git? On the latest git in looters.py that line is 288.

BobMacpherson avatar May 18 '19 19:05 BobMacpherson

If someone need this, i found solution of this problem. 2 files must be edited: ~/.local/lib/python2.7/site-packages/instalooter/looters.py - append symbol '#' at start of 281 line ~/.local/lib/python2.7/site-packages/instalooter/pages.py - 172 line: return cls(data['id', session, user_data['rhx_gis']) must be replaced by: return cls(data['id', session, "") And don't forget about spaces (not delete it, just replace text)

Meaning of this is removing all "rhx_gis" variables from code.

Awesome! Great fix!

I'll add that in my case, I also needed to modify line 738 of looters.py to: return HashtagIterator(self._hashtag, self.session, "")

But to your point, removing all "rhx_gis"variables does the trick!

Edit: Nevermind, this worked, and then when running it a few hours later, it began erroring out with retry errors.

andrewhogan93 avatar May 18 '19 21:05 andrewhogan93

Encountering this same issue, and don't really want to dive in and start editing config files if it's not actually going to result in a fix. It sounds like the issue is on IG's end.

purgat0ri avatar May 19 '19 11:05 purgat0ri

The fixes to the lines in looters.py and pages.py worked initially, but now I'm getting:

CRITICAL RetryError[<Future at 0x109d41710 state=finished raised KeyError>]
Traceback (most recent call last):
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/instalooter/cli/__init__.py", line 173, in main
    dlpbar_cls=None if args['--quiet'] else TqdmProgressBar)
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/instalooter/looters.py", line 861, in download
    worker.run()
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/instalooter/worker.py", line 119, in run
    self._DOWNLOAD_METHODS[media["__typename"]](media)
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/tenacity/__init__.py", line 292, in wrapped_f
    return self.call(f, *args, **kw)
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/tenacity/__init__.py", line 358, in call
    do = self.iter(retry_state=retry_state)
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/tenacity/__init__.py", line 332, in iter
    six.raise_from(retry_exc, fut.exception())
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/six.py", line 737, in raise_from
    raise value
RetryError: RetryError[<Future at 0x109d41710 state=finished raised KeyError>]

I am doing one-off calls to instalooter post <post-url>

artlung avatar May 19 '19 17:05 artlung

Instagram-scraper has a more active dev then instalooter, and works currently,might as well swap over at this point. https://github.com/rarcega/instagram-scraper

BobMacpherson avatar May 19 '19 20:05 BobMacpherson

If someone need this, i found solution of this problem. 2 files must be edited:

@SirobbsCrafter's patch worked for me. Might fail when using some additional options, as I have no idea what is supposed to be stored under this key, but my calls work fine.

@althonos, I think this should be elevated to a bug.

haltefest avatar May 19 '19 20:05 haltefest

The fixes to the lines in looters.py and pages.py worked initially, but now I'm getting:

CRITICAL RetryError[<Future at 0x109d41710 state=finished raised KeyError>]
Traceback (most recent call last):
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/instalooter/cli/__init__.py", line 173, in main
    dlpbar_cls=None if args['--quiet'] else TqdmProgressBar)
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/instalooter/looters.py", line 861, in download
    worker.run()
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/instalooter/worker.py", line 119, in run
    self._DOWNLOAD_METHODS[media["__typename"]](media)
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/tenacity/__init__.py", line 292, in wrapped_f
    return self.call(f, *args, **kw)
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/tenacity/__init__.py", line 358, in call
    do = self.iter(retry_state=retry_state)
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/tenacity/__init__.py", line 332, in iter
    six.raise_from(retry_exc, fut.exception())
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/six.py", line 737, in raise_from
    raise value
RetryError: RetryError[<Future at 0x109d41710 state=finished raised KeyError>]

I am doing one-off calls to instalooter post <post-url>

I have a same error when add '-v' parameter (download videos too). I not found the solution yet. Maybe, Instagram changed something again and app shutdownes with error (same as with 'rhx_gis').

SirobbsCrafter avatar May 20 '19 08:05 SirobbsCrafter

I don't know if this is the right way, but I prepared a PR which works for me: https://github.com/althonos/InstaLooter/pull/248

Maybe rhx_gis is not needed anymore, and the code could be simplified.

tgandor avatar May 20 '19 11:05 tgandor

Instagram-scraper has a more active dev then instalooter, and works currently,might as well swap over at this point. https://github.com/rarcega/instagram-scraper

Instagram-Scraper sometimes download image files of less than the best quality.

PfXCPI avatar May 21 '19 15:05 PfXCPI

You need to comment line 288 of looters.py aswell, to get rid of this self.rhx = get_shared_data(res.text)['rhx_gis']

Hm, in my case it was 281 line. But you're right: text, that you wrote, must be commented.

Line 280 for me, working just fine after that!

enbermudas avatar May 23 '19 04:05 enbermudas

returning empty rhx_gis worked for video option in my case. but sometimes I get CRITICAL u'edge_media_to_comment' error

yasinalm avatar May 24 '19 03:05 yasinalm

The fixes to the lines in looters.py and pages.py worked initially, but now I'm getting:

CRITICAL RetryError[<Future at 0x109d41710 state=finished raised KeyError>]
Traceback (most recent call last):
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/instalooter/cli/__init__.py", line 173, in main
    dlpbar_cls=None if args['--quiet'] else TqdmProgressBar)
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/instalooter/looters.py", line 861, in download
    worker.run()
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/instalooter/worker.py", line 119, in run
    self._DOWNLOAD_METHODS[media["__typename"]](media)
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/tenacity/__init__.py", line 292, in wrapped_f
    return self.call(f, *args, **kw)
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/tenacity/__init__.py", line 358, in call
    do = self.iter(retry_state=retry_state)
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/tenacity/__init__.py", line 332, in iter
    six.raise_from(retry_exc, fut.exception())
  File "/Users/artlung/Library/Python/2.7/lib/python/site-packages/six.py", line 737, in raise_from
    raise value
RetryError: RetryError[<Future at 0x109d41710 state=finished raised KeyError>]

I am doing one-off calls to instalooter post <post-url>

Same here. Any updates?

ogencoglu avatar May 31 '19 13:05 ogencoglu

This is most likely caused by yet another change in the private data structure used to store media metadata. I'll give it a look, but this may be super easy or super hard to fix, depending on how much the schema changed.

althonos avatar May 31 '19 18:05 althonos

Any updates on this issue?

JakeCow avatar Jun 07 '19 06:06 JakeCow

met the same issue "RetryError" after setting "rhx_gis" to an empty string, the traceback says "KeyError: 'edge_media_to_comment'"

图片

roy668899 avatar Jun 20 '19 10:06 roy668899

is this project abandoned?

sfc-gh-lrobles avatar Jun 24 '19 20:06 sfc-gh-lrobles

Release 2.4.0 has a shallow fix for this issue thanks to @tgandor.

I suspect the edge_media_to_comment error to be related to query limits as well: I'll check that and wrap it in a better error if that is the case.

althonos avatar Jun 28 '19 23:06 althonos

Version 2.4.0 with Python 3.7.4 has solved my issues with this problem.

artlung avatar Aug 11 '19 22:08 artlung