[🐛 Bug]: Firefox does not use the profile path when passed via options. Python

[DollarAkshay]

What happened?

It seems like firefox does not use the profile path when passed via options.

I have tried many different ways to set the profile.

Method 1 (Failing)

options.set_preference('profile', profile_path)

Method 2 (Failing)

profile = FirefoxProfile(profile_path)
options.profile = profile

Method 3 (Failing)

profile = FirefoxProfile(profile_path)
driver = webdriver.Firefox(options=options, service=service, firefox_profile=profile)

All of these methods result in a temp profile being created somewhere else. This can be verified by looking at the geckodriver.log and seeing the profile path in the command.

The only way that does seem to work is passing it as an argument

Method 4 (Working)

options.add_argument("-profile")
options.add_argument(profile_path)

Setting the profile via the arguments is the only way it seems to use the profile_path. This can be verified by looking at the geckodriver.log and seeing the profile path in the command and also looking at the profile_path folder which gets populated by a bunch of files and folders.

How can we reproduce the issue?

from selenium import webdriver
from selenium.webdriver.firefox.options import Options as FirefoxOptions
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox.service import Service as FirefoxService


profile_path = "D:\\test"
driver_path = "./src/drivers/geckodriver_win64.exe"

service = FirefoxService(driver_path)
options = FirefoxOptions()
options.add_argument("--log-level=3")
options.set_preference('profile', profile_path)

driver = webdriver.Firefox(options=options, service=service)

Relevant log output

1663163281111	geckodriver	INFO	Listening on 127.0.0.1:49637
1663163281617	mozrunner::runner	INFO	Running command: "C:\\Program Files\\Mozilla Firefox\\firefox.exe" "--marionette" "--log-level=3" "--remote-debugging-port" "49638" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\AKSHAY~1.ARA\\AppData\\Local\\Temp\\rust_mozprofileDsRaUY"
1663163282023	Marionette	INFO	Marionette enabled
console.warn: SearchSettings: "get: No settings file exists, new profile?" (new NotFoundError("Could not open the file at C:\\Users\\akshay.aradhya\\AppData\\Local\\Temp\\rust_mozprofileDsRaUY\\search.json.mozlz4", (void 0)))

Operating System

Windows 10

Selenium version

Python 4.4.3

What are the browser(s) and version(s) where you see this issue?

Firefox

What are the browser driver(s) and version(s) where you see this issue?

GeckoDriver 0.31.0

Are you using Selenium Grid?

No

[github-actions[bot]]

@DollarAkshay, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[titusfortner]

Most likely, this issue is a geckodriver problem. But...

Selenium Firefox profiles are kind of a pain in general, and we have a lot of code in various languages that worked with the old Firefox extension, not geckodriver. So it's probably a good idea to do some cleanup and verify things work.

[Vito-Santana]

I have fixed this bug, it's actually on the webdriver.py for firefox. If you check line 168 it states

    if options:
            if options.binary:
                self.binary = options.binary
            if options.profile:
                self.profile = options.profile

I checked the values and such key(options.profile) is non-existent, upon inspection the values set during the set_preference operation are not going through, thus I added a capability as following

 caps = {

        "os" : "OS X",
        "osVersion" : "Monterey",
        "buildName" : "firefoxprofile- python",
        "sessionName" : "firefoxprofile- python",
        "browserName" : "Firefox",
        "profile": profilePath,
}
opt.set_capability('bstack:options', caps)

Therefore if you overwrite line 168 to 172 with the following:

    if options:
            if options.binary:
                self.binary = options.binary
            if options.capabilities.get("bstack:options",False):
                if options.capabilities["bstack:options"].get("profile",False):
                    firefox_profile = str(options.capabilities["bstack:options"]["profile"])

the parsing of info actually takes place, that meant for me that values we're under a key related to the setter method name. Thus if you wish to fix it for proper preference usage :

if options:
            if options.binary:
                self.binary = options.binary
            if options.preferences.get("profile",False):
                firefox_profile = str(options.preferences["profile"])

Hope this is helpful in any manner.

[titusfortner]

There are 3 types of capabilities: w3c specified capabilities, browser specific capabilities and vendor specific capabilities. The browser options classes are designed to natively manage the first two, and you should not be putting those values inside vendor name-spaced capability dictionaries.

Profile should be set top-level, not as a preference, so we need to figure out why this isn't working:

profile = FirefoxProfile(profile_path)
options.profile = profile

[Vito-Santana]

Is not working simply because of the handling of the script itself, if you edit the script and print the options object using vars() you get:

'{'_arguments': [],
 '_binary': None,
 '_caps': {'acceptInsecureCerts': True,
           'browserName': 'firefox',
           'moz:debuggerAddress': True,
           'pageLoadStrategy': 'normal'},
 '_ignore_local_proxy': False,
 '_preferences': {'profile': 'C:\\MY_DECLARED_PROFILE_PATH'},
 '_profile': None,
 '_proxy': None,
 'log': <selenium.webdriver.firefox.options.Log object at 0x000001AE364B0F40>,
 'mobile_options': None}'

There's no profile value in the key since the method stated in the script firefox_profile.py is setting as a "sub-key", you can verify that in the following:

    # Public Methods
   def set_preference(self, key, value):
       """
       sets the preference that we want in the profile.
       """
       self.default_preferences[key] = value

Thus a whole new set_preference method would be need to be written for the sake of such automatic key matching, but in practice it is working as intended, preferences are all a subset which is handled and traversed under that tree node, therefore the issue lies on the usage of the options object on the script webdriver.py itself as stated on my previous comment. Is also stated in that same script that :

 The keyword arguments given to this constructor are helpers to
        more easily allow Firefox WebDriver sessions to be customised
        with different options.  They are mapped on to a capabilities
        dictionary that is passed on to the remote end.
        As some of the options, such as `firefox_profile` and
        `options.profile` are mutually exclusive, precedence is
        given from how specific the setting is.  `capabilities` is the
        least specific keyword argument, followed by `options`,
        followed by `firefox_binary` and `firefox_profile`.
        In practice this means that if `firefox_profile` and
        `options.profile` are both set, the selected profile
        instance will always come from the most specific variable.
        In this case that would be `firefox_profile`.  This will result in
        `options.profile` to be ignored because it is considered
        a less specific setting than the top-level `firefox_profile`
        keyword argument.  Similarly, if you had specified a
        `capabilities["moz:firefoxOptions"]["profile"]` Base64 string,
        this would rank below `options.profile`.

Now what happens in the script itself is that no such checkings are made but instead a direct assumption that the value within options is already allocated/assigned under options.profile. This issue has nothing to do with capabilities types or anything of similar fashion, is solely about internal handling of the output of the objects which are then passed/transmited to the geckodriver.

[titusfortner]

Ok, dug into this and found the location of the actual issue at least.

The problem is that the FirefoxProfile class was not properly updated from the switch to geckodriver. Back when we used to have our own Firefox driver and it used the extension, it was managed by FirefoxProfile class, so it needed to set default values from a manifest. This is no longer needed.

Instead of just zipping the provided directory, the existing code wants to update things, and whatever it is dong to update them is breaking it. If this line gets commented out - https://github.com/SeleniumHQ/selenium/blob/trunk/py/selenium/webdriver/firefox/firefox_profile.py#L163

Then this code works as expected:

    options = webdriver.FirefoxOptions()
    options.profile = "/Users/titusfortner/Library/Application Support/Firefox/Profiles/1amwoj91.testing"
    driver = webdriver.Firefox(options=options)

Let me try deleting a bunch of things and see if everything we want to work still works. :)

[titusfortner]

Java recently stopped doing this in 7c4a0501 Though Ruby maintained a list of defaults to match what Selenium 3 was doing - https://github.com/SeleniumHQ/selenium/pull/9138

[ghodss]

Any update on a fix for this getting merged in? The workaround listed in the original issue (Method 4) is working successfully for me, but I lost over a day to this bug and it would be nice to not have that happen to others.

[titusfortner]

I made a PR that deleted a bunch of stuff, but we agreed that we should add deprecation notices instead of deleting things. Issue is being tracked here - https://github.com/SeleniumHQ/selenium/issues/11587

My paid work has picked up, and this is a lower priority item. Not sure ETA.

[asifhg]

I tried doing it with: options.set_preference('profile', '/home/asif/snap/firefox/common/.mozilla/firefox/Selenium') but it does not work. I am only able to use a custom profile for Firefox with the following options configuration as in the example. Note that I previously created the Selenium profile. Please let me know and I can post the full source code of the simple example.

Example

...
options = Options()
options.page_load_strategy = 'normal'
profile_arg_param = '-profile' ;
profile_arg_value = os.getenv('HOME') + '/snap/firefox/common/.mozilla/firefox/Selenium'
options.add_argument(profile_arg_param)
options.add_argument(profile_arg_value)
browser = webdriver.Firefox(service=Service(GeckoDriverManager().install()), options=options)
...

Versions

pip3 list | grep -E "selenium|webdriver" ;
selenium               4.10.0
webdriver-manager      3.8.6

[jorgejesus]

I've firefox installed with snap, and the above code was the only think that worked for me. After making a blank folder on: ~/snap/firefox/common/.mozilla/firefox/Selenium

[titusfortner]

I forgot about this one. I really need to look at it.

[titusfortner]

All of these methods result in a temp profile being created somewhere else

Yes, the idea is to copy the profile to a temp directory, allow the user to modify it with the Profile class if desired, then zip it and send it to the driver. The driver will then unzip it to its own temp directory and use that.

Are you having an issue with what should be in the profile, or are you just seeing a different path?

[github-actions[bot]]

This issue was closed because we did not receive any additional information after 14 days.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity since it was closed. Please open a new issue for related bugs.