Fix issues with the data scrapper script

[proffapt]

Was being worked on https://github.com/metakgp/wimp/pull/68. Verify the work and do the necessary changes then merge it.

[premagarwals]

Hi

[proffapt]

Assigned, should be done by today :)

Work has been done already, verify it and if there are issues fix them.

[proffapt]

@premagarwals :) ye toh kabka hone wala tha :)

[premagarwals]

Ha Maine ise check kr liya. And jo misunderstanding Hui thi ki mai wrong wale pr test kr rha tha... actually mai pr se code utha kr test kr rha tha jo ki repo code se same hi tha. Kuchh issues dikhe and unhe maine fix Kiya h-

• Kuch packages issue kr rhe thhe and unke version change krne pr now they are working fine. • Data scraping link error de rha tha and maine usme headers diye hai and ek request.post me data={'lang':'en'} use ho rha tha jo ki empty json return kr rha tha...kuch aur parameters ki jarurat thi full json k liye aur maine add kr diye h. • Now, main.py is occasionally throwing connection reset error, and ye fully iitkgp ke website k karan ho rha (jaha tak abhi mujhe pata chala h), kyuki Maine random tareeke se timeout set kr krke try Kiya, kabhi connection timeout aa jata h, kabhi read timeout and kabhi fine work krta h. So, ek maine repeating code dala tha error aane pr, lekin ek baar error aane se continuously aa rha tha. So, user ko shayad 2-3 baar run krna pad jaaye main.py successfully work krane k liye.

[proffapt]

@rohan-b-84 ^^

[rohan-b-84]

Didn't get you clearly. Do you mean kgp site pe request karne pe kabhi response aara, kabhi nhi ara ?

[harshkhandeparkar]

Done #82