Recent cookies not reported (they are in the sqlite3 WAL)

[plicit]

browser_cookie3 currently creates a tmp_cookie_file copy of the browser's sqlite3 db, but does not copy the associated journal/wal/shm files which stores the most recent cookie writes, so those are not reported by browser_cookie3.

I don't think that the tmp_cookie_file mechanism is necessary anymore since Chrome and Firefox now appear to be using the sqlite3 WAL which should not throw a locking error. If I'm wrong, then some other solution should be found; perhaps polling.

I can create a PR if you want, but I basically just removed the whole tmp_cookie_file mechanism.

[SachiaLanlus]

@plicit I encounter the aame issue. How do you work around with this bug.

[plicit]

@SachiaLanlus, I just created a public fork you can try that is up-to-date with barisbabic's master. It uses the browser's cookie_file sqlite db directly instead of create_local_copy and tmp_cookie_file. It needs more testing, but I haven't run into any locking errors so far. If you do, then you might try opening it in read-only mode:

https://stackoverflow.com/questions/10205744/opening-sqlite3-database-from-python-in-read-only-mode#21794758

[SachiaLanlus]

@plicit I have tried the no-tmp-copy version of cookie_browser3. The problem is still there. It looks like this might not be the cause of this issue. In my observation, it takes ~30 secs to get the most recent cookie. My workflow is as follows:

1. Try to access the target website with cookie.
2. If cookie expired, open that website with chrome.
3. (User login.)
4. Retry every 5 secs.

The step 4 is running in the background while user login. In the most time, it gets the updated cookie after 6~7 retries, which is equivalents to 30~35 secs. This result is the same as the master branch. So it might be that the reason is chrome does not write the cookie to Cookie file immediately, rather than the tmp cookie.

[plicit]

@SachiaLanlus , that's odd. Chrome on Windows 10 updated the Cookies db immediately on a local site that I just tested. To remove your code as a variable, you can try querying the Chrome Cookies db directly with the sqlite3.exe cli or with a gui like SQLite Browser to see how quickly Chrome updates the db.

[SachiaLanlus]

I use SQLite Browser to test the latency. Chrome 103.0.5060.134 on Macos 12.4 update the cookie file 30~35 sec after I update the cookie on the website which is the same as I measured previously. However sometimes, it NEVER updated the cookie... I have no idea. Maybe this is a bug on the Macos build of Chrome. Such annoying... I will try with other platform and browser.