pvoutput
pvoutput copied to clipboard
openclimatefix
pvoutput.get_status throws error: NotImplementedError
Describe the bug pvoutput.get_status throws error: NotImplementedError: file structure not yet supported
To Reproduce Steps to reproduce the behavior: pv_system_id = 39266 date = '2022-01-13' status = self.pv.get_status(pv_system_id, date=date) See error NotImplementedError: file structure not yet supported
Expected behavior Download pv_system status without error
Additional context I'm seeing pvoutput error: NotImplementedError: file structure not yet supported when requesting status of system_id 29291: Requesting system status for 2022-01-13 See appended. DEBUG oputput and stack traces.
I have seen a similar error with another system pvoutput get_status 209 INFO system_id 39266: Requesting system status for 2022-01-13
2022-01-13 15:36:13,539 pvoutput get_status 209 INFO system_id 39266: Requesting system status for 2022-01-13
2022-01-13 15:36:13,543 connectionpool _new_conn 1001 DEBUG Starting new HTTPS connection (1): pvoutput.org:443
2022-01-13 15:36:14,063 connectionpool _make_request 456 DEBUG https://pvoutput.org:443 "GET /service/r2/getstatus.jsp?d=20220113&h=1&limit=288&ext=0&sid1=39266 HTTP/1.1" 200 None
2022-01-13 15:36:14,064 utils _get_response 81 DEBUG response: status_code=200; headers={'Date': 'Thu, 13 Jan 2022 15:36:13 GMT', 'Set-Cookie': 'JSESSIONID=us9myf32fu8ndpm2h7qwsc5k;Path=/; Secure', 'Expires': 'Thu, 01 Jan 1970 00:00:00 GMT', 'X-Rate-Limit-Remaining': '230', 'X-Rate-Limit-Limit': '300', 'X-Rate-Limit-Reset': '1642089600', 'Content-Type': 'text/plain', 'Content-Encoding': 'gzip', 'Vary': 'Accept-Encoding, User-Agent', 'Connection': 'close', 'Server': 'Jetty(7.6.17.v20150415)', 'Strict-Transport-Security': 'max-age=15768000'}
2022-01-13 15:36:14,064 pvoutput _set_rate_limit_params 928 DEBUG {'rate_limit_remaining': 230, 'rate_limit_total': 300, 'rate_limit_reset_time': Timestamp('2022-01-13 16:00:00+0000', tz='UTC')}
Traceback (most recent call last):
File "/home/ph1jb/git/pvoutput/src/downloader.py", line 240, in
2022-01-13 14:34:06,763 pvoutput get_status 209 INFO system_id 29291: Requesting system status for 2022-01-13
2022-01-13 14:34:06,772 connectionpool _new_conn 1001 DEBUG Starting new HTTPS connection (1): pvoutput.org:443
2022-01-13 14:34:07,310 connectionpool _make_request 456 DEBUG https://pvoutput.org:443 "GET /service/r2/getstatus.jsp?d=20220113&h=1&limit=288&ext=0&sid1=29291 HTTP/1.1" 200 None
2022-01-13 14:34:07,312 utils _get_response 81 DEBUG response: status_code=200; headers={'Date': 'Thu, 13 Jan 2022 14:34:07 GMT', 'Set-Cookie': 'JSESSIONID=10fqtsfp2518apqdw1dfmpdbl;Path=/; Secure', 'Expires': 'Thu, 01 Jan 1970 00:00:00 GMT', 'X-Rate-Limit-Remaining': '163', 'X-Rate-Limit-Limit': '300', 'X-Rate-Limit-Reset': '1642086000', 'Content-Type': 'text/plain', 'Content-Encoding': 'gzip', 'Vary': 'Accept-Encoding, User-Agent', 'Connection': 'close', 'Server': 'Jetty(7.6.17.v20150415)', 'Strict-Transport-Security': 'max-age=15768000'}
2022-01-13 14:34:07,312 pvoutput _set_rate_limit_params 928 DEBUG {'rate_limit_remaining': 163, 'rate_limit_total': 300, 'rate_limit_reset_time': Timestamp('2022-01-13 15:00:00+0000', tz='UTC')}
Traceback (most recent call last):
File "/home/ph1jb/git/pvoutput/src/downloader.py", line 262, in
Hi peter, No. I query for 225 systems in the Balkans and have noticed 2 systems throwing this error. See end of github issue. There may be more. Can you query the same system ids to reproduce the error? Best regards Julian Sent from phone
On Fri, 14 Jan 2022, 15:23 Peter Dudfield, @.***> wrote:
It's this specific to this one system_id?
— Reply to this email directly, view it on GitHub https://github.com/openclimatefix/pvoutput/issues/84#issuecomment-1013219928, or unsubscribe https://github.com/notifications/unsubscribe-auth/APAUAOI2LFVZXXKSD7WWWZ3UWA5WPANCNFSM5L6XG3MA . You are receiving this because you authored the thread.Message ID: @.***>
Hi Peter,
I've not downloaded UK data yet. The error arises from at least one system having the wrong number of fields in its status.
Happy to share the code of the lightweight PVOutput downloader I have developed.
best regards Julian
On Thu, 3 Feb 2022 at 16:16, Peter Dudfield @.***> wrote:
Sorry I havent looked into this yet. Does this happen in the UK too?
— Reply to this email directly, view it on GitHub https://github.com/openclimatefix/pvoutput/issues/84#issuecomment-1029155189, or unsubscribe https://github.com/notifications/unsubscribe-auth/APAUAOO2UINA464GBI5BEYTUZKS6JANCNFSM5L6XG3MA . You are receiving this because you authored the thread.Message ID: @.***>
-- Julian Briggs Research Software Developer Sheffield Solar Physics & Astronomy University of Sheffield In line with government advice, I am currently working remotely. My preferred method of contact is email. My normal working hours are 9:30-5 Mon-Thu
Hi Peter,
here's an example warning about the PVOutput system with Wrong number of values in row
2022-02-04T09:30:23.838688674Z 2022-02-04 09:30:23,838 downloaderpvoutput download_insert_readings 416 WARNING Bad status: Wrong number of values in row (12), expected (11): ('20220204', '09:10', '700', '0.156', '3459', '0', '0.000', 'NaN', 'NaN', '9.5', '2', '227.9').. pv_system_id: 39266, date: 2022-02-04.
I trust you can reproduce this and catch the PVOutput error: Bad status
more anon best Julian
On Thu, 3 Feb 2022 at 19:30, Julian Briggs @.***> wrote:
Hi Peter,
I've not downloaded UK data yet. The error arises from at least one system having the wrong number of fields in its status.
Happy to share the code of the lightweight PVOutput downloader I have developed.
best regards Julian
On Thu, 3 Feb 2022 at 16:16, Peter Dudfield @.***> wrote:
Sorry I havent looked into this yet. Does this happen in the UK too?
— Reply to this email directly, view it on GitHub https://github.com/openclimatefix/pvoutput/issues/84#issuecomment-1029155189, or unsubscribe https://github.com/notifications/unsubscribe-auth/APAUAOO2UINA464GBI5BEYTUZKS6JANCNFSM5L6XG3MA . You are receiving this because you authored the thread.Message ID: @.***>
-- Julian Briggs Research Software Developer Sheffield Solar Physics & Astronomy University of Sheffield In line with government advice, I am currently working remotely. My preferred method of contact is email. My normal working hours are 9:30-5 Mon-Thu
-- Julian Briggs Research Software Developer Sheffield Solar Physics & Astronomy University of Sheffield In line with government advice, I am currently working remotely. My preferred method of contact is email. My normal working hours are 9:30-5 Mon-Thu