heissepreise-cz icon indicating copy to clipboard operation
heissepreise-cz copied to clipboard

Published 20 hours ago • verified publisher icon xsuchy

Improved status logs

Open eltomjan opened this issue 1 year ago • 3 comments

Detailní logy - když je vše ok, tak tyhle lokálně vypadají nějak takhle:

Fetching data for date: 2024-01-08
lidl.fetchData
billa.fetchData
penny.fetchData
dm.fetchData
albert.fetchData
Albert https://www.albert.cz/api/v1/ {"query":"query DeviceId {deviceId}"} {"headers":{"Content-Type":"application/json;charset=UTF-8","X-Apollo-Operation-Name":"DeviceId"}}
globus.fetchData
tesco.fetchData
kaufland.fetchData
Albert https://www.albert.cz/api/v1/?operationName=LeftHandNavigationBar&variables={"rootCategoryCode":"","cutOffLevel":"4","lang":"cs"}&extensions={"persistedQuery":{"version":1,"sha256Hash":"..."}} {"headers":{"Content-Type":"application/json;charset=UTF-8","X-Apollo-Operation-Name":"DeviceId"}}
Tesco 0 - 1. z 15 https://nakup.itesco.cz/groceries/cs-CZ/shop/ovoce-a-zelenina/all?page=1&count=48
Tesco https://nakup.itesco.cz/groceries/cs-CZ/shop/ovoce-a-zelenina/all?page=1&count=48 {"headers":{"accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/...
Tesco cookies:akavpau_czech_vp=...~id=...; _abck=...
Tesco firstGet content (1934) <!DOCTYPE html> <html> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"> <meta http-equiv="refresh" content="5; URL='/groceries/cs-CZ/shop/ovoce-a-zelenina/all?page=1&count=48&bm-verify=...
Tesco firstGet 2 scripts / last <script> var xhr = new XMLHttpRequest(); xhr.withCredentials = true; xhr.addEventListener("loadend", function() { try { var data = JSON.parse(xhr.responseText); if (data.hasOwnProperty('reload')) { if (data["reload"] == true) { window.location.replace(window.location.href.replace(/[&?]bm-verify=[^#]*/, ""));  if(window.location.hash){ window.location.reload(); } } } else if (data.hasOwnProperty('location')) { window.location.replace(data["location"]); } else { window.location.reload(); } } catch (e) { var data = {}; window.location.reload(); }}); xhr.open("POST", "/_sec/verify?provider=interstitial", false); xhr.setRequestHeader("Content-Type", "application/json"); xhr.send(JSON.stringify({"bm-verify": "...", "pow": j})); </script>
=======
<script>        var i = 1704693195; var j = i + Number("3035" + "1161"); </script>
Tesco firstGet POST https://nakup.itesco.cz/_sec/verify?provider=interstitial {"headers":{"accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/...
Tesco firstGet cookies akavpau_czech_vp=...; ak_bmsc=...
Tesco https://nakup.itesco.cz/groceries/cs-CZ/shop/ovoce-a-zelenina/all?page=1&count=48 {"headers":{"accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/...
Fetched PENNY (1. of 8) data, took 2.5081322000026702 seconds, 32/32 items without category.
Albert https://www.albert.cz/api/v1/?operationName=GetCategoryProductSearch&variables={"lang":"cs","searchQuery":"","category":"zeB001","pageNumber":0,"pageSize":40,"filterFlag":true,"plainChildCategories":true}&extensions={"persistedQuery":{"version":1,"sha256Hash":"..."}}
Albert 14 pages x 40 items => 0 existing + 521 new
Albert https://www.albert.cz/api/v1/?operationName=GetCategoryProductSearch&variables={"lang":"cs","searchQuery":"","category":"zeB001","pageNumber":1,"pageSize":40,"filterFlag":true,"plainChildCategories":true}&extensions={"persistedQuery":{"version":1,"sha256Hash":"..."}}
Tesco https://nakup.itesco.cz/groceries/cs-CZ/shop/ovoce-a-zelenina/all?page=1&count=48 got 2184265 bytes.
Tesco data-redux-state indexes 191885-948819 => 48 items.
Tesco 1/7 of 326.
Tesco 48 - 1. z 15 https://nakup.itesco.cz/groceries/cs-CZ/shop/ovoce-a-zelenina/all?page=2&count=48
Tesco https://nakup.itesco.cz/groceries/cs-CZ/shop/ovoce-a-zelenina/all?page=2&count=48 {"headers":{"accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/...
Fetched KAUFLAND (2. of 8) data, took 4.376207899987698 seconds, 0/346 items without category.
Albert https://www.albert.cz/api/v1/?operationName=GetCategoryProductSearch&variables={"lang":"cs","searchQuery":"","category":"zeB001","pageNumber":2,"pageSize":40,"filterFlag":true,"plainChildCategories":true}&extensions={"persistedQuery":{"version":1,"sha256Hash":"..."}}
Albert https://www.albert.cz/api/v1/?operationName=GetCategoryProductSearch&variables={"lang":"cs","searchQuery":"","category":"zeB001","pageNumber":3,"pageSize":40,"filterFlag":true,"plainChildCategories":true}&extensions={"persistedQuery":{"version":1,"sha256Hash":"..."}}
Tesco https://nakup.itesco.cz/groceries/cs-CZ/shop/ovoce-a-zelenina/all?page=2&count=48 got 2161234 bytes.
Tesco data-redux-state indexes 195618-950908 => 48 items.
Tesco 2/7 of 326.
Tesco 96 - 1. z 15 https://nakup.itesco.cz/groceries/cs-CZ/shop/ovoce-a-zelenina/all?page=3&count=48
Tesco https://nakup.itesco.cz/groceries/cs-CZ/shop/ovoce-a-zelenina/all?page=3&count=48 {"headers":{"accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/...
Albert https://www.albert.cz/api/v1/?operationName=GetCategoryProductSearch&variables={"lang":"cs","searchQuery":"","category":"zeB001","pageNumber":4,"pageSize":40,"filterFlag":true,"plainChildCategories":true}&extensions={"persistedQuery":{"version":1,"sha256Hash":"..."}}
Albert https://www.albert.cz/api/v1/?operationName=GetCategoryProductSearch&variables={"lang":"cs","searchQuery":"","category":"zeB001","pageNumber":5,"pageSize":40,"filterFlag":true,"plainChildCategories":true}&extensions={"persistedQuery":{"version":1,"sha256Hash":"..."}}
Tesco https://nakup.itesco.cz/groceries/cs-CZ/shop/ovoce-a-zelenina/all?page=3&count=48 got 2153865 bytes.
Tesco data-redux-state indexes 195618-954953 => 48 items.
Tesco 3/7 of 326.
Tesco 144 - 1. z 15 https://nakup.itesco.cz/groceries/cs-CZ/shop/ovoce-a-zelenina/all?page=4&count=48
Tesco https://nakup.itesco.cz/groceries/cs-CZ/shop/ovoce-a-zelenina/all?page=4&count=48 {"headers":{"accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/...
Albert https://www.albert.cz/api/v1/?operationName=GetCategoryProductSearch&variables={"lang":"cs","searchQuery":"","category":"zeB001","pageNumber":6,"pageSize":40,"filterFlag":true,"plainChildCategories":true}&extensions={"persistedQuery":{"version":1,"sha256Hash":"..."}}
...

eltomjan avatar Jan 08 '24 06:01 eltomjan