Inconsistant meter reading

[hindseb]

Hi First, thank you so much for creating this tool to read the coned meter data and get this into Home Assistant!

I noticed that the tool does not constantly read all meter updates published by ConEd. Somehow, it is only successful in ~1 of 8 updates available.

Below are some screenshots. I am not sure where exactly the issue is, but maybe it helps to fix this so that the tool does catch all updates.

Thanks a lot for your help!

`DevTools listening on ws://127.0.0.1:47855/devtools/browser/743a81eb-383d-4ab4-8d0b-b12366843dbc [0820/223427.304942:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24) [0820/223428.612899:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24) [0820/223431.125800:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed. (error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Flogin&t=1661049270747 (3) [0820/223431.141310:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed. (error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Flogin&t=1661049270737 (3) [0820/223500.242722:INFO:CONSOLE(2)] "Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check https://xhr.spec.whatwg.org/.", source: https://acdn-prod-bsdce-ngcx-va.azureedge.net/va/coned-va-build.min.js (2) [0820/223501.557375:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24) [0820/223502.520911:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24) [0820/223503.017514:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed. (error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Flogin%3Furl%3D%252fen%252faccounts-billing%252fdashboard%253ftab1%253dbillingandusage-1%2526tab3%253dsectionRealTimeData-3%26returnUrl%3D%252Fen%252Faccounts-billing%252Fdashboard%253Ftab1%253Dbillingandusage-1%2526tab3%253DsectionRealTimeData-3&t=1661049302844 (3) [0820/223506.979122:INFO:CONSOLE(1)] "::-webkit-details-marker pseudo element selector is deprecated. Please use ::marker instead. See https://chromestatus.com/feature/6730096436051968 for more details.", source: https://nextweb-static.opower.com/maestro/themes/maestro-theme-cned/291e529f859e134f140a21f4b32a070d/pages/apps/maestro-widget-library-core/index.generated.embedded-api.40.min.js (1) Task exception was never retrieved future: <Task finished coro=<resp() done, defined at /usr/local/lib/python3.7/dist-packages/coned/meter.py:225> exception=NetworkError('Protocol error (Network.getResponseBody): No resource with given identifier found')> Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/coned/meter.py", line 231, in resp raw_data = await res.text() File "/usr/local/lib/python3.7/dist-packages/pyppeteer/network_manager.py", line 669, in text content = await self.buffer() File "/usr/local/lib/python3.7/dist-packages/pyppeteer/network_manager.py", line 654, in _bufread 'requestId': self._request._requestId pyppeteer.errors.NetworkError: Protocol error (Network.getResponseBody): No resource with given identifier found [0820/223512.896145:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed. (error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Faccounts-billing%2Fdashboard%3Ftab1%3Dbillingandusage-1%26tab3%3DsectionRealTimeData-3&t=1661049310566 (3) Creating Meter Calling meter.last_read().. res.url: https://cned.opower.com/ei/edge/apis/cws-real-time-ami-v1/cws/cned/accounts/8ff0f432-7638-11eb-99d8-02001700dbdb/meters/13798110/usage res.status: 200 res.url: https://cned.opower.com/ei/edge/apis/cws-real-time-ami-v1/cws/cned/accounts/8ff0f432-7638-11eb-99d8-02001700dbdb/meters/13798110/usage res.status: 200 message: {'startTime': '2022-08-20T21:45:00-04:00', 'endTime': '2022-08-20T22:00:00-04:00', 'value': 0.021, 'uom': 'KWH'} Connecting to mqtt 192.168.5.149 as mqtt_user Publishing to mqtt Publishing electric_meter/value: 0.021 Publishing electric_meter/uom: KWH Publishing electric_meter/startTime: 2022-08-20T21:45:00-04:00 Publishing electric_meter/endTime: 2022-08-20T22:00:00-04:00 Publishing electric_meter/message: {"startTime": "2022-08-20T21:45:00-04:00", "endTime": "2022-08-20T22:00:00-04:00", "value": 0.021, "uom": "KWH"} Disconnectig from mqtt DONE Sleeping 5 mins..

[0820/224118.863236:ERROR:bus.cc(393)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory DevTools listening on ws://127.0.0.1:50145/devtools/browser/9b452db1-1157-4498-bb8e-6eeb69116340 [0820/224119.702062:ERROR:bus.cc(393)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory DevTools listening on ws://127.0.0.1:42163/devtools/browser/a60da16c-a6cb-401b-b8d8-982c2ecf5749 [0820/224124.877010:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24) [0820/224127.177833:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed. (error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Flogin&t=1661049686882 (3) [0820/224155.751674:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24) [0820/224156.594159:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed. (error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Flogin&t=1661049716339 (3) [0820/224210.437321:INFO:CONSOLE(2)] "Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check https://xhr.spec.whatwg.org/.", source: https://acdn-prod-bsdce-ngcx-va.azureedge.net/va/coned-va-build.min.js (2) Task exception was never retrieved future: <Task finished coro=<Meter.browse() done, defined at /usr/local/lib/python3.7/dist-packages/coned/meter.py:120> exception=TimeoutError('Navigation Timeout Exceeded: 10000 ms exceeded.')> Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/coned/meter.py", line 199, in browse await page.goto(usage_page_url, {'waitUntil': 'domcontentloaded', 'timeout': 10000}) File "/usr/local/lib/python3.7/dist-packages/pyppeteer/page.py", line 837, in goto raise error pyppeteer.errors.TimeoutError: Navigation Timeout Exceeded: 10000 ms exceeded. [0820/224216.089391:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24) [0820/224218.489546:INFO:CONSOLE(2)] "Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check https://xhr.spec.whatwg.org/.", source: https://acdn-prod-bsdce-ngcx-va.azureedge.net/va/coned-va-build.min.js (2) [0820/224219.567349:INFO:CONSOLE(1)] "::-webkit-details-marker pseudo element selector is deprecated. Please use ::marker instead. See https://chromestatus.com/feature/6730096436051968 for more details.", source: https://nextweb-static.opower.com/maestro/themes/maestro-theme-cned/291e529f859e134f140a21f4b32a070d/pages/apps/maestro-widget-library-core/index.generated.embedded-api.40.min.js (1) Creating Meter Calling meter.last_read().. Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/coned/meter.py", line 96, in last_read await self.browse() File "/usr/local/lib/python3.7/dist-packages/coned/meter.py", line 199, in browse await page.goto(usage_page_url, {'waitUntil': 'domcontentloaded', 'timeout': 10000}) File "/usr/local/lib/python3.7/dist-packages/pyppeteer/page.py", line 837, in goto raise error pyppeteer.errors.TimeoutError: Navigation Timeout Exceeded: 10000 ms exceeded. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "run.py", line 26, in startTime, endTime, value, uom = asyncio.get_event_loop().run_until_complete(meter.last_read()) File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete return future.result() File "/usr/local/lib/python3.7/dist-packages/coned/meter.py", line 118, in last_read raise MeterError("Error requesting meter data") coned.meter.MeterError: Error requesting meter data [0820/224250.029086:ERROR:database.cc(1707)] Cookie sqlite error 1032, errno 0: attempt to write a readonly database, sql: DELETE FROM cookies WHERE name=? AND host_key=? AND path=? [0820/224250.031757:ERROR:database.cc(1707)] Cookie sqlite error 2067, errno 0: UNIQUE constraint failed: cookies.host_key, cookies.name, cookies.path, sql: INSERT INTO cookies (creation_utc, host_key, name, value, path, expires_utc, is_secure, is_httponly, last_access_utc, has_expires, is_persistent, priority, encrypted_value, samesite, source_scheme, source_port, is_same_party) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) [0820/224250.033149:ERROR:database.cc(1707)] Cookie sqlite error 1032, errno 0: attempt to write a readonly database, sql: DELETE FROM cookies WHERE name=? AND host_key=? AND path=? [0820/224250.034569:ERROR:database.cc(1707)] Cookie sqlite error 2067, errno 0: UNIQUE constraint failed: cookies.host_key, cookies.name, cookies.path, sql: INSERT INTO cookies (creation_utc, host_key, name, value, path, expires_utc, is_secure, is_httponly, last_access_utc, has_expires, is_persistent, priority, encrypted_value, samesite, source_scheme, source_port, is_same_party) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) [0820/224250.036206:ERROR:database.cc(1707)] Cookie sqlite error 1032, errno 0: attempt to write a readonly database, sql: DELETE FROM cookies WHERE name=? AND host_key=? AND path=? [0820/224250.037849:ERROR:database.cc(1707)] Cookie sqlite error 2067, errno 0: UNIQUE constraint failed: cookies.host_key, cookies.name, cookies.path, sql: INSERT INTO cookies (creation_utc, host_key, name, value, path, expires_utc, is_secure, is_httponly, last_access_utc, has_expires, is_persistent, priority, encrypted_value, samesite, source_scheme, source_port, is_same_party) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) Sleeping 5 mins..`

[0820/225413.007690:ERROR:bus.cc(393)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory DevTools listening on ws://127.0.0.1:34941/devtools/browser/9c9bc563-97a8-4425-ab97-e46dd448fc07 [0820/225413.896444:ERROR:bus.cc(393)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory DevTools listening on ws://127.0.0.1:36045/devtools/browser/221b9a78-f2af-482e-8b6d-8cc4a08f1bea [0820/225419.102500:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24) [0820/225419.448758:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24) [0820/225421.218994:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed. (error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Flogin&t=1661050460803 (3) [0820/225441.969077:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed. (error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Flogin&t=1661050481436 (3) [0820/225458.051161:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24) [0820/225459.603692:INFO:CONSOLE(24)] "ERR_DUP_SCRIPTS", source: https://www.glancecdn.net/cobrowse/CobrowseJS.ashx?group=19641&site=production (24) [0820/225459.924115:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed. (error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Flogin%3Furl%3D%252fen%252faccounts-billing%252fdashboard%253ftab1%253dbillingandusage-1%2526tab3%253dsectionRealTimeData-3%26returnUrl%3D%252Fen%252Faccounts-billing%252Fdashboard%253Ftab1%253Dbillingandusage-1%2526tab3%253DsectionRealTimeData-3&t=1661050499607 (3) [0820/225501.157648:INFO:CONSOLE(3)] "Uncaught (in promise) Error: Loading chunk 11 failed. (error: https://siteintercept.qualtrics.com/dxjsmodule/11.538b4e05effab334888e.chunk.js?Q_CLIENTVERSION=1.76.0&Q_CLIENTTYPE=web&Q_BRANDID=www.coned.com)", source: https://znetevwa2wlb6en9p-coned.siteintercept.qualtrics.com/WRSiteInterceptEngine/?Q_ZID=ZN_etEvWA2WLb6eN9P&Q_LOC=https%3A%2F%2Fwww.coned.com%2Fen%2Flogin%3Furl%3D%252fen%252faccounts-billing%252fdashboard%253ftab1%253dbillingandusage-1%2526tab3%253dsectionRealTimeData-3%26returnUrl%3D%252Fen%252Faccounts-billing%252Fdashboard%253Ftab1%253Dbillingandusage-1%2526tab3%253DsectionRealTimeData-3&t=1661050500951 (3) Creating Meter Calling meter.last_read().. Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/coned/meter.py", line 96, in last_read await self.browse() File "/usr/local/lib/python3.7/dist-packages/coned/meter.py", line 214, in browse self._LOGGER.debug(f"raw_data = {raw_data}") NameError: name 'raw_data' is not defined During handling of the above exception, another exception occurred: Traceback (most recent call last): File "run.py", line 26, in startTime, endTime, value, uom = asyncio.get_event_loop().run_until_complete(meter.last_read()) File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete return future.result() File "/usr/local/lib/python3.7/dist-packages/coned/meter.py", line 118, in last_read raise MeterError("Error requesting meter data") coned.meter.MeterError: Error requesting meter data Task was destroyed but it is pending! task: <Task pending coro=<sleep() running at /usr/lib/python3.7/asyncio/tasks.py:568> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7fa7bb6ee8>()]> cb=[<TaskWakeupMethWrapper object at 0x7fa7af97c8>()]> Task was destroyed but it is pending! task: <Task pending coro=<Meter.browse() running at /usr/local/lib/python3.7/dist-packages/coned/meter.py:205> wait_for=<Task pending coro=<sleep() done, defined at /usr/lib/python3.7/asyncio/tasks.py:555> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7fa7bb6ee8>()]> cb=[<TaskWakeupMethWrapper object at 0x7fa7af97c8>()]>> Sleeping 5 mins..