Papers Are All You Need

Metadata coverage

Setup

1. chmod +x setup.sh
2. ./setup.sh

Usage

Run redis-server in a separate terminal.

Edit config.ini with your parameters. It's ok to leave title_kw, author_kw, or affiliation_kw blank.

Confirguation options

conference     Name of conferences
year           Year(s) of conference(s)
title_kw       Keywords to search for in title of papers
author_kw      Names of authors to search for
affiliation_kw Names of affiliations/labs to search for
template       File name format for saving pdfs e.g. `year-author-title` will save papers 
               as the year the paper was last published, followed by the first author's name, 
               followed by the title, all in lower case. You can rearrange these three pieces.
save_dir       Directory to save pdfs to
mode           Default mode "download".
               download - Download all papers.
               search   - Search for papers and save to log. Will not download papers.
               stats    - Count number of papers in each conference by year. Saves to json stats.json.

[DEFAULT]
conference = neurips
year = 2020
title_kw = graph, message-passing, node
author_kw = 
affiliation_kw = 
template = year-author-title
save_dir = /Downloads/neurips
mode = download

and then run

python main.py

Search all conferences

[DEFAULT]
conference = *
year = 2021, 2022
title_kw = graph, message-passing, node
author_kw = 
affiliation_kw = 
template = year-author-title
save_dir = /Downloads/all_confs_2021_2022
mode = search

Search year range

[DEFAULT]
conference = neurips
year = 1987:2019
title_kw = graph, message-passing, node
author_kw = 
affiliation_kw = 
template = year-author-title
save_dir = /Downloads/neurips
mode = search

Search every proceeding across all conferences

[DEFAULT]
conference = *
year = *
title_kw = graph, message-passing, node
author_kw = 
affiliation_kw = 
template = year-author-title
save_dir = /Downloads/grid_search
mode = search

Search specific years

[DEFAULT]
conference = neurips
year = 1999, 2012, 2018
title_kw = graph, message-passing, node
author_kw = 
affiliation_kw = 
template = year-author-title
save_dir = /Downloads/neurips
mode = download

Download all titles

[DEFAULT]
conference = neurips
year = 1999, 2012, 2018
title_kw = *
author_kw = 
affiliation_kw = 
template = year-author-title
save_dir = /Downloads/neurips
mode = download