Cookie importing in firefox fails

[HoroTW]

Hey there, I am a user of MILK for some time now, today I wanted to import some cookies from Chromium into Firefox... It didn't work. I tried exporting cookies in Firefox, deleting these cookies and then reimporting them - that import failed too..

Any ideas why it fails or where to look at?

MILK version: 2.3.1 Firefox version: 105.0.3 (64-bit)

Test JSON with four cookies created from MILK in Firefox:

[
	{
		"name": "_gat",
		"value": "1",
		"domain": ".mozilla.org",
		"hostOnly": false,
		"path": "/",
		"secure": false,
		"httpOnly": false,
		"sameSite": "no_restriction",
		"session": false,
		"firstPartyDomain": "",
		"partitionKey": null,
		"expirationDate": 1666615441,
		"storeId": "firefox-default"
	},
	{
		"name": "id",
		"value": "PNwGVQC0SDefE2hOEIGvRff%2FiKZ6iy%2Fv8CYARvZkWiZoLjSZ2BQ+z0%2FQMolzDc%2FIwQRPJQNXHF4E1+DeXDT4pMaYoNF484Urt4w44BYGo0QSM6XylMu9YvevbEYq28yTiBHk9ANrqWng2rSWA0SyJT+acAyPvXXQJuBWJw6yptaM70W41w%3D%3D",
		"domain": "developer.mozilla.org",
		"hostOnly": true,
		"path": "/",
		"secure": true,
		"httpOnly": true,
		"sameSite": "lax",
		"session": false,
		"firstPartyDomain": "",
		"partitionKey": null,
		"expirationDate": 1666616307,
		"storeId": "firefox-default"
	},
	{
		"name": "_ga",
		"value": "GA1.2.624241648.1666613368",
		"domain": ".mozilla.org",
		"hostOnly": false,
		"path": "/",
		"secure": false,
		"httpOnly": false,
		"sameSite": "no_restriction",
		"session": false,
		"firstPartyDomain": "",
		"partitionKey": null,
		"expirationDate": 1729687416,
		"storeId": "firefox-default"
	},
	{
		"name": "_gid",
		"value": "GA1.2.1880916581.1666613368",
		"domain": ".mozilla.org",
		"hostOnly": false,
		"path": "/",
		"secure": false,
		"httpOnly": false,
		"sameSite": "no_restriction",
		"session": false,
		"firstPartyDomain": "",
		"partitionKey": null,
		"expirationDate": 1666701816,
		"storeId": "firefox-default"
	}
]

[null93]

Hey! Thanks for using the extension and I am glad you like it enough to use it. It looks like this is something with the storeId. I think Firefox and Chrome use a different one by default.

[HoroTW]

Hey @null93, thank you for your reply. 😊 That was what I thought at first too, but that is also why I tried to only use Firefox.

For all the following I only used Firefox: Export cookies -> cookies.json Delete cookies (with MILK) Import cookies <- cookies.json --> No cookies where imported....

Since this is the most basic function of the export and import I think there is a bug ^^

[null93]

Hey! I totally agree with you. I wonder if this is the case with Chrome as well. I'll have to check this later.

I am currently on paternity leave so it may be a while till I get to this issue, but I will do my best. PRs are also always welcome if you wanna tackle this issue yourself 🙂.

[HoroTW]

Yeah - MILK is open source and I really like it so naturally I will at least try to tackle this myself :)

Do you have a starting point that comes to mind directly?

First thing is testing if it works on chrome (which I think it does [will test later]) when it works on chrome the next step would be to get the extension build/installed locally and find the point where the Firefox API is used to create the cookies and poke around there..?

Does this seem right?

[null93]

I'm glad you like it and thank you for attempting to solve this issue!

That seems about right. You can debug the code in the service worker (background script).

To compile it should look something like this:

• nvm use
• yarn install
• NODE_ENV=dev yarn build

Using NODE_ENV=dev should output debug info into console.

It would also be interesting to see what the export looks like in Chrome v. Firefox, could just be a differently structured output? Although I am using a shim library that Mozilla made for cross compatiblility browser API calls.

Hope that helps! Looking forward to see what you come up with, and thanks again for tackling it! 😀

[HoroTW]

Hey @null93 thank you for your guidance! I checked in Chromium and Google Chrome and both have the same issue - I also checked on another computer (to rule out some system miss configuration) - Since the import did not work on any of these :see_no_evil: it's more general I guess.. 😅 I will give another update tomorrow evening.

[HoroTW]

Soo I had not as much time poking around as I hoped but I found out one part of the issue is due to my timezone.

It took me some time to realize why I could not build the extension :see_no_evil: - It seems that some packages were written with "wrong" capitalization, I would guess that on some systems this would not be a problem but on my Linux it is.

Sorry for the slow progress - I'm new to web development... Tomorrow evening I will try to get some kind of useful debugging working. My current build command:

NODE_ENV=development; parcel build --no-cache --no-source-maps --no-minify src/service-worker.jsx src/*.html

But even with this command I don't know how to debug the code in a useful manner - if I inspect the extension window and go into it's source to debug it looks like this:

  function $60b5fa1ff71b373d43af71b18ffd4f1a$export$default(condition, message) {
    if (!$60b5fa1ff71b373d43af71b18ffd4f1a$var$isProduction) {
      if (condition) {
        return;
      }

      var text = "Warning: " + message;

      if (typeof console !== 'undefined') {
        console.warn(text);
      }

      try {
        throw Error(text);
      } catch (x) {}
    }
  }

And that is not very readable :laughing: - I obviously miss something. Should I try to debug it in the Browsers web tools or should I try to debug it from e.g. VSCode - any opinion?

[null93]

Soo I had not as much time poking around as I hoped but I found out one part of the issue is due to my timezone

I wonder what that is because exporting and importing deals with UNIX timestamps

It took me some time to realize why I could not build the extension 🙈 - It seems that some packages were written with "wrong" capitalization, I would guess that on some systems this would not be a problem but on my Linux it is.

Anything I need to change in package.json? What package was giving you issues?

Sorry for the slow progress - I'm new to web development...

Please don't even worry about it, you are doing me a favor :)

NODE_ENV=development; parcel build --no-cache --no-source-maps --no-minify src/service-worker.jsx src/*.html

Don't forget about the prebuild that runs before the build command. It would be easiest to execute with yarn or npm:

NODE_ENV=dev yarn build

Should I try to debug it in the Browsers web tools or should I try to debug it from e.g. VSCode - any opinion?

Once you run the build command, you should see the build files in the dist folder. Do the following to load your built version:

1. Open chrome://extensions
2. Make sure Developer mode is turned on (top-right)
3. Click "Load unpacked" button (top-left) and find the dist folder with the built files

Now you loaded your built of the extension into Chrome. It is a similar process for firefox, just different places to look.

If you are using the master branch, then that still uses background scripts. You can debug the background script by visiting chrome://extensions/ and clicking on the background script link. From there just open the inspector panel.

If you are on the next branch, that uses service workers as background scripts and you can debug the service worker by just visiting the options page for the extension and opening up the inspector panel from there.

Remember, if you are making changes to the background script / service worker, you should reload the extension by clicking the "update" button (refresh icon)

Hope that helps!

[HoroTW]

Hey hey, I got to the bottom of this issue - Thanks @x3raX for helping me .... and the issue is stupidly easy... :laugh: I tried debugging it step by step, only to find that everything did what I expected..... And in the end I saw that it worked.... which stunned me.... I thought, oh no, one of those "heisenbug issues" that disappear when you try to debug it - probably related to some race condition or something.... But then I saw the obvious difference.... When I select "debug this popup".... the popup stays open even if it loses focus.... what happens when I select the file for import.... So I checked it in full screen view, and sure enough, it worked.

So the issue is that when you use the popup window and try to import cookies, after the file dialog the popup gets closed and since the importing is done in the JS of the popup it gets canceled by the browser and the import does not work as expected....

So the current workarounds are: Use of the full screen view or to select "debug this popup" To solve the issue we would need to look at a way to let the popup stay even if it loses focus or do the processing in the background worker...

As a quick mitigation of the issue I would say that the import button should open the full screen view? What do you think?

[null93]

Hey!

Good work! Yeah, that is definitely expected. Initially, I had it import all cookies at once, but that destroyed the CPU if you have as many cookies as me lol (~3000). I came up with the solution to loop through them and do it in batches. That way I could also have a nice progress indicator. The problem as you pointed out is that the client JS controls the looping of all the cookies and if the window or popup is closed, it stops the execution of the loop.

I would say the perfect solution is to do it in the background, but with the introduction of Manifest V3 where service workers replace background scripts, it might not be possible. With background scripts (Manifest V2), this would be possible. If I remember correctly, in the docs it said that Chrome can stop the service worker at any point it wants to. This is probably so Chrome was more power efficient etc, but this really affects what we are trying to do here.

I would say for now, the best solution is to warn the user not to close the popup/window. To the best of my knowledge, there is no way to prevent the popup window from closing (other than dev console but nothing programatically).

[x3rAx]

Hi @null93,

awesome extension, thanks!

Since the popup is closed as soon as the file chooser dialog opens (at least on chrome), warning the user won't do anything.

I'd suggest to open the full-page view (for example with a URL Parameter like ?start_import=true) and then automatically showing the file chooser (if it's possible to trigger it from JS, but AFAIK it is). Doing so would have the benefits that the user can always see the import progress, even with Manifest V2.

Also, a warning would then not be necessary (although it wouldn't hurt) because closing the tab can be prevented using the beforeunload event:

class PendingJobs<T> {
  // NOTE: You might need to replace `#` with `_` or enable transpiling
  //       "class properties", depending on your transpiler config
  #jobs: Set<Promise<T>> = new Set()

  add(promise: Promise<T>) {
    this.#jobs.add(promise)
    const cleanup = () => this.#jobs.delete(promise)
    promise.then(cleanup).catch(cleanup)
  }

  hasJobs(): boolean {
    return this.#jobs.size > 0
  }
}


// --- On page load ---

// Make this globally accessible
const PENDING_JOBS = new PendingJobs()

window.addEventListener("beforeunload", (event) => {
  if (!PENDING_JOBS.hasJobs()) {
    return
  }

  // This message won't be displayed in modern browsers but it
  // must not be empty for the confirmation dialog to be displayed.
  event.returnValue =
    "There is pending work. Do you really want to leave the page?"
})


// --- When starting the import ---

let importPromise = new Promise<void>((resolve, reject) => {
  // ...
  resolve()
})

PENDING_JOBS.add(importPromise)

[null93]

Hey @x3rAx,

Thank you! Glad you like it 😄.

Since the popup is closed as soon as the file chooser dialog opens (at least on chrome), warning the user won't do anything.

For the warning, I was thinking of warning before the file-picker opens.

I'd suggest to open the full-page view (for example with a URL Parameter like ?start_import=true) and then automatically showing the file chooser (if it's possible to trigger it from JS, but AFAIK it is). Doing so would have the benefits that the user can always see the import progress, even with Manifest V2.

I am not against that solution, good work!

1. Detected import click in popup A. Open fullscreen with hash param B. Prompt file-picker and open import GUI C. Remove hash param so that on refresh it wouldn't open file-picker again D. Import as usual
2. Detected import click in fullscreen A. Import as usual

I currently just got back to work from paternity leave so things are a bit too busy for me to work on this. If you are able to implement this in the outlined way above, I would be more than happy to merge in a PR 😄

[GrassBlock1]

awesome extension, thanks!

Since the popup is closed as soon as the file chooser dialog opens (at least on chrome), warning the user won't do anything.

I'd suggest to open the full-page view (for example with a URL Parameter like ?start_import=true) and then automatically showing the file chooser (if it's possible to trigger it from JS, but AFAIK it is). Doing so would have the benefits that the user can always see the import progress, even with Manifest V2.

Hi @x3rAx , I would say that the file chooser dialog and the extension popup appear at the same time on my chrome (running on Windows), but I think what you suggest is useful. Unforunately I found this problem on firefox and importing to firefox is impossible,as mentioned. @null93

[HoroTW]

Huhu @GrassBlock1, no it is not impossible - you just have to right click the milk icon and select "Fullscreen" - then you can import normally. 😊

[GrassBlock1]

Huhu @GrassBlock1, no it is not impossible - you just have to right click the milk icon and select "Fullscreen" - then you can import normally. 😊

Hi @HoroTW , Yes I tried again with cookies exported from milk on FireFox with the way you said, and it works. But when I tried to import cookies exported from milk on Chrome in the same way, it fails. like this:

[null93]

closing :)