scrapedin-linkedin-crawler icon indicating copy to clipboard operation
scrapedin-linkedin-crawler copied to clipboard

Published 20 hours ago • verified publisher icon linkedtales

Scraping only given profiles

Open anubhavmalik opened this issue 3 years ago • 2 comments

I am curious if I can scrape only a given set of profiles through this?

rootProfiles in config.json contains the URLs of from where to scrape and not the actual profile urls. Is there a provision to do that?

anubhavmalik avatar Sep 30 '20 13:09 anubhavmalik

Read the README file of the project, it says:

The profiles will be stored on the directory configured at config.json as individual file per profile. If you want to do something else (as saving on a database), just rewrite the src/saveProfile.js function.

abhij89 avatar Oct 23 '20 22:10 abhij89

@abhij89 That is not what I my question says. Allow me to phrase it in a more descriptive manner.

I want the crawler to just go to a pre-defined set of URLs, that's it. Right now the crawler gets profiles from related, etc.

I don't want it to do that. I just want profile data for a given list of LinkedIn profile URLs which I can provide it.

anubhavmalik avatar Oct 26 '20 13:10 anubhavmalik