affiliation_parser
affiliation_parser copied to clipboard
Implement affiliation matcher with pyspark
We can access row number by this following post:
- http://stackoverflow.com/questions/39544796/pyspark-spark-how-to-select-last-row-and-also-how-to-access-pyspark-dataframe-b
- http://stackoverflow.com/questions/32760888/pyspark-dataframes-way-to-enumerate-without-converting-to-pandas
- http://stackoverflow.com/questions/33681487/how-do-i-add-a-new-column-to-spark-data-frame-pyspark