snare fail to clone web page

when i try to clone, it alerts me "aiohttp" is not installed. after then, use "python3.6 -m pip install aiohttp".. and i got this couldn't find any solutions

Nov 24 '20 15:11 Haotian-Shi-cyber

Can you try running sudo apt install python3.6-dev before the setup?

Dec 13 '20 10:12 lordlabuckdas

just found it is ./bin/clone right? not built-in command... its ok, is there requirements for pages im going to clone? eg, must be http and not for https?

Jan 29 '21 21:01 Haotian-Shi-cyber

./bin/clone contains the raw file for the cloner which will be built into a pip module by setuptools in setup.py. The recommended way to set up and use Snare (and Cloner) is to run [sudo] python setup.py install and then use the clone command directly. Also, I don't think there are no requirements regarding the protocol (http/https), you can clone any site.

Jan 30 '21 05:01 lordlabuckdas

i cloned youtube.com but still can't access page.

Jan 30 '21 11:01 Haotian-Shi-cyber

every time i cloned a page, it got so many warnings and errors, is there anything wrong

Jan 30 '21 13:01 Haotian-Shi-cyber

@Haotian-Shi-cyber Please post the output of the log files.

Feb 01 '21 05:02 mzfr

some examples

Feb 01 '21 09:02 Haotian-Shi-cyber

@Haotian-Shi-cyber it would be nice if you can post the complete output on Pastebin and then give the link. The screenshot you provided doesn't give much information.

Feb 02 '21 04:02 mzfr

http://pastebin.com/28XTaM98

Feb 02 '21 23:02 Haotian-Shi-cyber

@Haotian-Shi-cyber most of the warnings come from the css parsing library, it is not the from cloner itself. Also please remember that cloner has really limited functionality, so not every page can be successfully cloned.

Regarding the page serving, could you please provide the snare logs instead of the cloning logs? From your cloning logs I can not say anything regarding your problem with page access.

Thanks

Feb 03 '21 18:02 afeena

i cloned youtube.com but still can't access page.

Hello, i have the same issue do you find a solution ? thanks

Mar 25 '21 15:03 aabulaeva

hi,

no, i gave it up.

thanks

Mar 25 '21 20:03 Haotian-Shi-cyber

I just tested this and had no issues except the following error:

Traceback (most recent call last):                                                                                                                           
  File "/home/mzfr/dev/snare/venv/bin/clone", line 4, in <module>                                                                                            
    __import__('pkg_resources').run_script('Snare==0.3.0', 'clone')                                                                                          
  File "/home/mzfr/dev/snare/venv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 672, in run_script                                           
    self.require(requires)[0].run_script(script_name, ns)                                                                                                    
  File "/home/mzfr/dev/snare/venv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1479, in run_script                                          
    exec(script_code, namespace, namespace)                                                                                                                  
  File "/home/mzfr/dev/snare/venv/lib/python3.10/site-packages/Snare-0.3.0-py3.10.egg/EGG-INFO/scripts/clone", line 81, in <module>                          
  File "/home/mzfr/dev/snare/venv/lib/python3.10/site-packages/Snare-0.3.0-py3.10.egg/EGG-INFO/scripts/clone", line 67, in main                              
UnboundLocalError: local variable 'end' referenced before assignment

That should be fixed https://github.com/mushorg/snare/pull/320

May 25 '23 19:05 mzfr