Bugs when using here_api from the command-line without environment variable for pgpass

[radumas]

First run

2021-06-07 10:02:36,377 __main__ INFO Getting Access Token
2021-06-07 10:02:36,516 __main__ INFO Querying data from 2019-02-01 to 2019-02-28
2021-06-07 10:03:36,648 __main__ INFO Polling status of query request: 139025
2021-06-07 10:03:36,705 __main__ INFO Requested query completed
2021-06-07 10:03:36,706 __main__ INFO Downloading data
2021-06-07 10:03:43,915 __main__ INFO Sending data to database
2021-06-07 10:03:43,917 __main__ WARNING No here_bot environment variable detected, assuming .pgpass value exists
psql: error: invalid variable name: ""ON_ERROR_STOP"
2021-06-07 10:03:43,960 __main__ CRITICAL Error sending data to database
2021-06-07 10:03:43,961 __main__ CRITICAL Fatal error in pulling data
2021-06-07 10:03:43,961 __main__ CRITICAL None

Then after fixing the double quotes in the subprocess call https://github.com/CityofToronto/bdit_data-sources/blame/fda4b7848ec4463d43058ba7a2c5e9fe8d1839b0/here/traffic/here_api.py#L174

2021-06-07 11:15:02,308 __main__ INFO Getting Access Token
2021-06-07 11:15:02,445 __main__ INFO Querying data from 2019-03-01 to 2019-03-31
2021-06-07 11:16:02,608 __main__ INFO Polling status of query request: 139030
2021-06-07 11:16:02,642 __main__ INFO Requested query completed
2021-06-07 11:16:02,643 __main__ INFO Downloading data
2021-06-07 11:16:10,942 __main__ INFO Sending data to database
2021-06-07 11:16:10,944 __main__ WARNING No here_bot environment variable detected, assuming .pgpass value exists
NOTICE:  identifier "\COPY here.ta FROM STDIN WITH (FORMAT csv, HEADER 
                    TRUE);" will be truncated to "\COPY here.ta FROM STDIN WITH (FORMAT csv, HEADER 
            "
ERROR:  syntax error at or near ""\COPY here.ta FROM STDIN WITH (FORMAT csv, HEADER 
                    TRUE);""
LINE 1: "\COPY here.ta FROM STDIN WITH (FORMAT csv, HEADER 

[radumas]

@chmnata can this be closed and is the https://github.com/CityofToronto/bdit_data-sources/compare/here_pull_old_data branch merged?

[chmnata]

@radumas It is not merged, and I think the files other than here_api.py are outdated since we are not re-pulling old here data anymore. I guess we can still merge it but move them into like an archived folder?

[radumas]

or maybe we split off the here_api.py fix and you can continue working on the backfill stuff when we backfill in August?