scrapedin-linkedin-crawler
scrapedin-linkedin-crawler copied to clipboard
Scraping only given profiles
I am curious if I can scrape only a given set of profiles through this?
rootProfiles
in config.json contains the URLs of from where to scrape and not the actual profile urls. Is there a provision to do that?
Read the README file of the project, it says:
The profiles will be stored on the directory configured at config.json as individual file per profile. If you want to do something else (as saving on a database), just rewrite the src/saveProfile.js function.
@abhij89 That is not what I my question says. Allow me to phrase it in a more descriptive manner.
I want the crawler to just go to a pre-defined set of URLs, that's it. Right now the crawler gets profiles from related, etc.
I don't want it to do that. I just want profile data for a given list of LinkedIn profile URLs which I can provide it.