scrape-linkedin-selenium icon indicating copy to clipboard operation
scrape-linkedin-selenium copied to clipboard

Added jobs and people in company_scraper

Open anilabhadatta opened this issue 3 years ago • 3 comments

Requires some company_scraper testing with temporary Linkedin accounts. command : scrapeli --company=delltechnologies -o output.json

anilabhadatta avatar Aug 24 '21 10:08 anilabhadatta

Some parts of the code are language-dependent, meaning it won't work for languages other than English (such as regular expressions etc.). But it works as expected for English. click_on_tab() function doesn't seem to be working for hidden tabs in the expander. Function can be simplified easily to tackle with that issue:

    def click_on_tab(self, tab_name):
        main_url = "https://www.linkedin.com/company/{}/".format(self.company)
        try:
            self.driver.get(main_url + tab_name)
        except:
            print("Tab cannot be found.")
            return

You have to add self.company = company to the constructor method of CompanyScraper object.

KarahanS avatar Aug 31 '21 11:08 KarahanS

def click_on_tab(self, tab_name):
        main_url = "https://www.linkedin.com/company/{}/".format(self.company)
        try:
            self.driver.get(main_url + tab_name)
        except:
            print("Tab cannot be found.")
            return

Thanks a lot, i overlooked this issue, Added your code .

anilabhadatta avatar Aug 31 '21 11:08 anilabhadatta

Some parts of the code are language-dependent, meaning it won't work for languages other than English (such as regular expressions etc.). But it works as expected for English. click_on_tab() function doesn't seem to be working for hidden tabs in the expander. Function can be simplified easily to tackle with that issue:

    def click_on_tab(self, tab_name):
        main_url = "https://www.linkedin.com/company/{}/".format(self.company)
        try:
            self.driver.get(main_url + tab_name)
        except:
            print("Tab cannot be found.")
            return

You have to add self.company = company to the constructor method of CompanyScraper object.

Added the code and verified with different language as well. Also fixed the json output where cyrillic characters were showing in \u escape characters.

anilabhadatta avatar Aug 31 '21 12:08 anilabhadatta