scrape-linkedin-selenium icon indicating copy to clipboard operation
scrape-linkedin-selenium copied to clipboard

Companyscraper doesn't work and returns error 'NoneType'

Open RaniMounikaKotakadi opened this issue 4 years ago • 4 comments

unable to get the company website Company Attrs: ['overview', 'jobs', 'life', 'insights'] Error!!! Traceback (most recent call last): File "linkedin_scrapper_chrome.py", line 72, in company_overview_dict = company.to_dict() File "/xxx/xxxx/linkedin_scrapper/scrape_linkedin/ResultsObject.py", line 13, in to_dict return dict(zip(self.attributes, vals)) File "/xxxx/xxxx/linkedin_scrapper/scrape_linkedin/ResultsObject.py", line 12, in vals = map(lambda attr: getattr(self, attr), self.attributes) File "/xxxx/xxxxx/linkedin_scrapper/scrape_linkedin/Company.py", line 34, in overview overview['description'] = container.select_one( AttributeError: 'NoneType' object has no attribute 'select_one'

RaniMounikaKotakadi avatar Aug 06 '20 15:08 RaniMounikaKotakadi

Hi @austinoboyle,

Found that there is no .error-container selector is no more on LinkedIn but using CSS selector in companyscrapper.py

load_initial() function

EC.presence_of_element_located
                    (By.CSS_SELECTOR, '.error-container'))

Also, from get_overview function, the following attribute is unavailable from the UI

tab_link = self.driver.find_element_by_css_selector(
                'a[data-control-name="page_member_main_nav_about_tab"]')

RaniMounikaKotakadi avatar Aug 08 '20 05:08 RaniMounikaKotakadi

Hi, I have the same issue -- I get the message:

Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
  File "/Users/eunhousong/anaconda3/lib/python3.7/site-packages/scrape_linkedin/ResultsObject.py", line 13, in to_dict
    return dict(zip(self.attributes, vals))
  File "/Users/eunhousong/anaconda3/lib/python3.7/site-packages/scrape_linkedin/ResultsObject.py", line 12, in <lambda>
    vals = map(lambda attr: getattr(self, attr), self.attributes)
  File "/Users/eunhousong/anaconda3/lib/python3.7/site-packages/scrape_linkedin/Company.py", line 34, in overview
    overview['description'] = container.select_one(
AttributeError: 'NoneType' object has no attribute 'select_one'

ehsong avatar Sep 14 '20 04:09 ehsong

Yes, this is due to a change in the LinkedIn UI. This happens once fairly frequently, and it's difficult to keep up. If you have a fix for the issue, I would be happy to approve a pull request.

austinoboyle avatar Oct 11 '20 02:10 austinoboyle

Hello Sir I would like to contribute to this issue

RohitRana19 avatar Sep 19 '22 05:09 RohitRana19

Hello Sir, I've left a pull request for this issue, please look into it.

Roy6801 avatar Sep 26 '22 05:09 Roy6801