Bunsly
Glassdoor and ZipRecruiter not working for me
Hi for me zip recruiter and glassdoor has stopped working could you please tell me where is the problem
@app.get("/scrape-jobs/glassdoor")
def scrape_jobs_glassdoor(search_term: str = Query(..., description = "Search term for job scrapping"), location: str = Query("USA", description="Location for job scrapping")):
try:
print(search_term)
jobs = scrape_jobs(
site_name=["glassdoor"],
search_term=search_term,
results_wanted=100,
# hours_old parameter is not applicable to Glassdoor, omitted
location=location,
country_indeed=location # Only needed for Glassdoor
# proxies=["208.195.175.46:65095", "208.195.175.45:65095", "localhost"],
)
jobs = jobs.replace([np.inf, -np.inf], np.nan).fillna(method='ffill')
jobs = jobs.applymap(lambda x: f"{x:.2f}" if isinstance(x, (int, float)) else x)
return jobs.to_dict(orient="records")
except Exception as e:
return {"error": str(e)}
@app.get("/scrape-jobs/ziprecruiter")
def scrape_jobs_ziprecruiter(search_term: str):
try:
jobs = scrape_jobs(
site_name=["zip_recruiter"],
search_term=search_term,
location="USA",
# hours_old parameter is not applicable to ZipRecruiter, omitted
# country_indeed parameter not needed
proxies=proxies_list
)
jobs = jobs.replace([np.inf, -np.inf], np.nan).fillna(method='ffill')
jobs = jobs.applymap(lambda x: f"{x:.2f}" if isinstance(x, (int, float)) else x)
return jobs.to_dict(orient="records")
except Exception as e:
return {"error": str(e)}
I went to glassdoor/constant.py and commented out the headers except for "content-type"
headers = {
# "authority": "www.glassdoor.com",
# "accept": "*/*",
# "accept-language": "en-US,en;q=0.9",
# "apollographql-client-name": "job-search-next",
# "apollographql-client-version": "4.65.5",
"content-type": "application/json",
# "origin": "https://www.glassdoor.com",
# "referer": "https://www.glassdoor.com/",
# "sec-ch-ua": '"Chromium";v="118", "Google Chrome";v="118", "Not=A?Brand";v="99"',
# "sec-ch-ua-mobile": "?0",
# "sec-ch-ua-platform": '"macOS"',
# "sec-fetch-dest": "empty",
# "sec-fetch-mode": "cors",
# "sec-fetch-site": "same-origin",
# "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36",
}
It seemed to work, however you don't get the job listings' description. Which I believe is in the posted /graph response when the job component gets clicked
I went to glassdoor/constant.py and commented out the headers except for "content-type"
headers = { # "authority": "www.glassdoor.com", # "accept": "*/*", # "accept-language": "en-US,en;q=0.9", # "apollographql-client-name": "job-search-next", # "apollographql-client-version": "4.65.5", "content-type": "application/json", # "origin": "https://www.glassdoor.com", # "referer": "https://www.glassdoor.com/", # "sec-ch-ua": '"Chromium";v="118", "Google Chrome";v="118", "Not=A?Brand";v="99"', # "sec-ch-ua-mobile": "?0", # "sec-ch-ua-platform": '"macOS"', # "sec-fetch-dest": "empty", # "sec-fetch-mode": "cors", # "sec-fetch-site": "same-origin", # "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36", }It seemed to work, however you don't get the job listings' description. Which I believe is in the posted /graph response when the job component gets clicked
In the latest version of JobSpy the content is already commented out. but it is still not working
I went to glassdoor/constant.py and commented out the headers except for "content-type"
headers = { # "authority": "www.glassdoor.com", # "accept": "*/*", # "accept-language": "en-US,en;q=0.9", # "apollographql-client-name": "job-search-next", # "apollographql-client-version": "4.65.5", "content-type": "application/json", # "origin": "https://www.glassdoor.com", # "referer": "https://www.glassdoor.com/", # "sec-ch-ua": '"Chromium";v="118", "Google Chrome";v="118", "Not=A?Brand";v="99"', # "sec-ch-ua-mobile": "?0", # "sec-ch-ua-platform": '"macOS"', # "sec-fetch-dest": "empty", # "sec-fetch-mode": "cors", # "sec-fetch-site": "same-origin", # "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36", }It seemed to work, however you don't get the job listings' description. Which I believe is in the posted /graph response when the job component gets clicked
In the latest version of JobSpy the content is already commented out. but it is still not working
Interesting, going to this file in this repo, it doesn't look like it's already commented out. Anyway, it looks like they figured it out in another issue forum: https://github.com/speedyapply/JobSpy/issues/270