export results to csv

[zaiinabyy]

hello, is there a way to export the query results to csv file?

[jessebehrens]

Yes! Save it to a dataframe.

First I create a list:

def on_data(data: EventData): job_postings.append([data.job_id, data.location, data.title, data.company, data.date, data.link, data.description])

Then I put that in a DF

#Put the job postings into a dataframe df = pd.DataFrame(job_postings, columns=['Job_ID','Location','Title', 'Company', 'Date', 'Link', 'Description'])

[leonpawelzik]

Where do you place that code? Also, it prompts, that job_postings is not defined.

[Prashanthvsdvn]

Find the function below and add the last line there. Be careful of the tab space. Need to tab once

def on_data(data: EventData):
    print('[ON_DATA]', data.title, data.company, data.company_link, data.date, data.link, data.insights,
          len(data.description))
    job_postings.append([data.job_id, data.location, data.title, data.company, data.date,
data.link, data.description])

and add these at the bottom

job_postings = []
scraper.run(queries)
df = pd.DataFrame(job_postings, columns=['Job_ID','Location','Title', 'Company','Date', 'Link', 'Description'])
df.to_csv("jobs.csv")