geckodriver
geckodriver copied to clipboard
mozilla
offsets are from the center of element instead of from the top-left corner
In order to help us efficiently investigate your issue, please provide the following information:
Platform and application details
- Platform: macOS Sierra 10.12.5
- Firefox: Firefox 54
- Selenium: 3.4.0
Steps to reproduce
In org.openqa.selenium.interactions.Actions the x, y offsets for moveToElement has been Offset from the top-left corner. But with geckodriver I'm getting: Jun 16, 2017 5:55:05 PM org.openqa.selenium.interactions.Actions moveToElement INFO: When using the W3C Action commands, offsets are from the center of element
Has the offset changed to center of element instead of top-left corner? While chromedriver, old firefox driver and safaridriver calculates the offset from top-left corner is this a bug in geckodriver?
-
[ ] Reproducable testcase: Try moveToElement method in org.openqa.selenium.interactions.Actions It differs between old firefox drivers.
-
[ ] A trace level log: Jun 16, 2017 5:55:05 PM org.openqa.selenium.interactions.Actions moveToElement INFO: When using the W3C Action commands, offsets are from the center of element
moveToElement is a shim provided by Selenium so it’s possible there is a bug with that. But without a reproducible test case and a trace-level geckodriver log, it is hard to which level this fails at. Can you please provide that?
The problem is the x,y offsets I used for old firefox driver and chromedriver don't work for geckodriver. So I get 'indexoutofbounds' exception when using the x,y offsets that I used for earlier versions of the driver. I will try to come up with a reproducible generic test case.
Sure, but please note that neither FirefoxDriver (from Selenium) or chromedriver are implementations of the WebDriver specification. It would be useful to see a trace-level log.
Here is a simple code to demonstrate the problem. The actual application I automate needs drawing that is why I rely on the Action.moveToElement but I just picked this one to demo the problem.
System.setProperty("webdriver.gecko.driver", "/Users/Downloads/geckodriver"); System.setProperty("webdriver.chrome.driver", "/Users/Downloads/chromedriver"); WebDriver driver = new FirefoxDriver(); // WebDriver driver = new ChromeDriver(); driver.get("http://book.theautomatedtester.co.uk/"); Actions action = new Actions(driver); WebElement mainbody = driver.findElement(By.xpath("html/body/div[2]/ul/li[1]")); action.moveToElement(mainbody,10,10); action.click().build().perform();
Please run this code using geckdriver and then comment out the FirefoxDriver and use the chromedriver line by uncommenting. Notice that the moveToElement works fine for chrome(it worked in old firefox driver as well) but in geckodriver it fails as the offset is computed from the center of the element instead of from top-left corner.
I don’t know what action.moveToElement does in the Selenium client, which is why I asked to see the geckodriver trace log so I can see what primitives are actually sent across the wire to geckodriver.
That it “worked before” is not a great bug.
Below is the log:
1497979442395 geckodriver INFO Listening on 127.0.0.1:44950
1497979446968 geckodriver::marionette INFO Starting browser /Applications/Firefox.app/Contents/MacOS/firefox-bin with args ["-marionette"]
1497979453319 Marionette INFO Listening on port 59254
1497979454432 Marionette DEBUG loaded listener.js
2017-06-20 10:24:14.896 plugin-container[90592:1542522] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x9c3b, name = 'com.apple.tsm.portname'
See /usr/include/servers/bootstrap_defs.h for the error codes.
2017-06-20 10:24:14.897 plugin-container[90592:1542522] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x9f03, name = 'com.apple.CFPasteboardClient'
See /usr/include/servers/bootstrap_defs.h for the error codes.
Jun 20, 2017 10:24:14 AM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
1497979455174 Marionette DEBUG Received DOM event "beforeunload" for "about:blank"
1497979455350 Marionette DEBUG Received DOM event "pagehide" for "about:blank"
1497979455350 Marionette DEBUG Received DOM event "unload" for "about:blank"
1497979455782 Marionette DEBUG Received DOM event "DOMContentLoaded" for "http://book.theautomatedtester.co.uk/"
1497979455785 Marionette DEBUG Received DOM event "pageshow" for "http://book.theautomatedtester.co.uk/"
Jun 20, 2017 10:24:15 AM org.openqa.selenium.interactions.Actions moveToElement
INFO: When using the W3C Action commands, offsets are from the center of element
With the old protocol (Wire) the command to move the cursor was moveto and the provided offset was relative to the top-left corner of the targeted element:
https://github.com/SeleniumHQ/selenium/wiki/JsonWireProtocol#sessionsessionidmoveto
But with the new W3C specification and implementation in marionette, moving the cursor to an element now requires a composite action where the offset is relative to the element's center:
https://w3c.github.io/webdriver/webdriver-spec.html#dfn-dispatch-a-pointermove-action
An object that represents a web element Let element be equal to the result of trying to get a known element with argument origin.
Let x element and y element be the result of calculating the in-view center point of element.
Let x equal x element + x offset, and y equal y element + y offset.
It seems that the specification was updated to accommodate the implementation of marionette which was already relative to center even before Firefox 45.
So this behavior is by design.
The current api doesn't provide a way to directly move the cursor with an offset relative to the top-left corner. It would make much more sense to provide an offset relative to the top-left corner especially when testing canvas.
That said, I would still consider this an issue, first because it breaks a previous feature and second because this change doesn't bring any value to the api.
The current api doesn't provide a way to directly move the cursor with an offset relative to the top-left corner. It would make much more sense to provide an offset relative to the top-left corner especially when testing canvas.
I think this is a fair point. @florentbr, do you want to raise an issue against the specification?
cc @AutomatedTester
I can add some info based on Selenium Java test suite, there is a test against this page: https://github.com/SeleniumHQ/selenium/blob/master/common/src/web/mousePositionTracker.html
public void testMovingMouseToRelativeElementOffset() {
driver.get(pages.mouseTrackerPage);
WebElement trackerDiv = driver.findElement(By.id("mousetracker"));
new Actions(driver).moveToElement(trackerDiv, 95, 195).perform();
WebElement reporter = driver.findElement(By.id("status"));
wait.until(fuzzyMatchingOfCoordinates(reporter, 95, 195));
}
The result is "78, 191" instead of expected "95, 195"
Trace log:
1503771497876 geckodriver INFO geckodriver 0.18.0
1503771497877 webdriver::httpapi DEBUG Creating routes
1503771497887 geckodriver INFO Listening on 127.0.0.1:36706
1503771498529 webdriver::server DEBUG -> POST /session {
"desiredCapabilities": {
"nativeEvents": false,
"firefox_binary": {
"extraEnv": {},
"hCode": 2008966511,
"class": "org.openqa.selenium.firefox.FirefoxBinary",
"timeout": 45000
},
"marionette": true,
"acceptInsecureCerts": true,
"browserName": "firefox",
"moz:firefoxOptions": {
"binary": "c:\\Program Files\\Nightly\\firefox.exe",
"prefs": {},
"args": []
},
"platformName": "ANY",
"version": "",
"platform": "ANY"
},
"requiredCapabilities": {},
"capabilities": {
"desiredCapabilities": {
"nativeEvents": false,
"firefox_binary": {
"extraEnv": {},
"hCode": 2008966511,
"class": "org.openqa.selenium.firefox.FirefoxBinary",
"timeout": 45000
},
"marionette": true,
"acceptInsecureCerts": true,
"browserName": "firefox",
"moz:firefoxOptions": {
"binary": "c:\\Program Files\\Nightly\\firefox.exe",
"prefs": {},
"args": []
},
"platformName": "ANY",
"version": "",
"platform": "ANY"
},
"requiredCapabilities": {},
"alwaysMatch": {
"acceptInsecureCerts": true
},
"firstMatch": [
{
"browserName": "firefox",
"moz:firefoxOptions": {
"binary": "c:\\Program Files\\Nightly\\firefox.exe",
"prefs": {},
"args": []
}
}
]
}
}
1503771498547 geckodriver::capabilities DEBUG Trying to read firefox version from ini files
1503771498548 geckodriver::capabilities DEBUG Found version 57.0a1
1503771498560 geckodriver::marionette INFO Starting browser c:\Program Files\Nightly\firefox.exe with args ["-marionette"]
1503771498918 addons.xpi WARN Error parsing extensions state: [Exception... "Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [amIAddonManagerStartup.readStartupData]" nsresult: "0x80520012 (NS_ERROR_FILE_NOT_FOUND)" location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm :: loadExtensionState :: line 1523" data: no] Stack trace: loadExtensionState()@resource://gre/modules/addons/XPIProvider.jsm:1523 < getInstallState()@resource://gre/modules/addons/XPIProvider.jsm:1558 < checkForChanges()@resource://gre/modules/addons/XPIProvider.jsm:3093 < startup()@resource://gre/modules/addons/XPIProvider.jsm:2160 < callProvider()@resource://gre/modules/AddonManager.jsm:263 < _startProvider()@resource://gre/modules/AddonManager.jsm:733 < startup()@resource://gre/modules/AddonManager.jsm:900 < startup()@resource://gre/modules/AddonManager.jsm:3084 < observe()@jar:file:///C:/Program%20Files/Nightly/omni.ja!/components/addonManager.js:65
1503771499180 Marionette DEBUG Received observer notification "profile-after-change"
1503771499273 Marionette DEBUG Received observer notification "command-line-startup"
1503771499273 Marionette INFO Enabled via --marionette
1503771499564 geckodriver::marionette TRACE connection attempt 0/600
1503771500664 geckodriver::marionette TRACE connection attempt 1/600
Unable to read VR Path Registry from C:\Users\alexei\AppData\Local\openvr\openvrpaths.vrpath
[Child 5292] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 346
1503771501526 Marionette DEBUG Received observer notification "sessionstore-windows-restored"
1503771501765 geckodriver::marionette TRACE connection attempt 2/600
1503771502122 Marionette DEBUG Setting recommended pref toolkit.cosmeticAnimations.enabled to false
1503771502122 Marionette DEBUG Setting recommended pref datareporting.policy.dataSubmissionPolicyAccepted to false
1503771502122 Marionette DEBUG Setting recommended pref dom.file.createInChild to true
1503771502123 Marionette INFO Listening on port 63806
1503771502372 geckodriver::marionette DEBUG Connected to Marionette on localhost:63806
1503771502375 Marionette DEBUG Accepted connection 0 from 127.0.0.1:63821
1503771502375 geckodriver::marionette TRACE <- {"applicationType":"gecko","marionetteProtocol":3}
1503771502375 geckodriver::marionette TRACE -> 163:[0,1,"newSession",{"acceptInsecureCerts":true,"browserName":"firefox","capabilities":{"desiredCapabilities":{"acceptInsecureCerts":true,"browserName":"firefox"}}}]
1503771502376 Marionette TRACE 0 -> [0,1,"newSession",{"acceptInsecureCerts":true,"browserName":"firefox","capabilities":{"desiredCapabilities":{"acceptInsecureCerts":true,"browserName":"firefox"}}}]
1503771502378 Marionette WARN TLS certificate errors will be ignored for this session
1503771502427 Marionette DEBUG Register listener.js for window 4294967297
1503771502446 Marionette TRACE 0 <- [1,1,null,{"sessionId":"c716d576-e364-4122-9a4d-1962333ff7dc","capabilities":{"browserName":"firefox","browserVersion":"57.0a1","platformName":"windows_nt","platformVersion":"6.1","pageLoadStrategy":"normal","acceptInsecureCerts":true,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"rotatable":false,"specificationLevel":0,"moz:processID":9276,"moz:profile":"C:\\Users\\alexei\\AppData\\Local\\Temp\\rust_mozprofile.KmpLScKEztfq","moz:accessibilityChecks":false,"moz:headless":false}}]
1503771502447 geckodriver::marionette TRACE <- [1,1,null,{"sessionId":"c716d576-e364-4122-9a4d-1962333ff7dc","capabilities":{"browserName":"firefox","browserVersion":"57.0a1","platformName":"windows_nt","platformVersion":"6.1","pageLoadStrategy":"normal","acceptInsecureCerts":true,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"rotatable":false,"specificationLevel":0,"moz:processID":9276,"moz:profile":"C:\\Users\\alexei\\AppData\\Local\\Temp\\rust_mozprofile.KmpLScKEztfq","moz:accessibilityChecks":false,"moz:headless":false}}]
1503771502447 webdriver::server DEBUG <- 200 OK {"value": {"sessionId":"c716d576-e364-4122-9a4d-1962333ff7dc","capabilities":{"acceptInsecureCerts":true,"browserName":"firefox","browserVersion":"57.0a1","moz:accessibilityChecks":false,"moz:headless":false,"moz:processID":9276,"moz:profile":"C:\\Users\\alexei\\AppData\\Local\\Temp\\rust_mozprofile.KmpLScKEztfq","pageLoadStrategy":"normal","platformName":"windows_nt","platformVersion":"6.1","rotatable":false,"specificationLevel":0,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000}}}}
1503771502803 webdriver::server DEBUG -> POST /session/c716d576-e364-4122-9a4d-1962333ff7dc/url {"url":"http://comp1:43833/common/mousePositionTracker.html"}
1503771502803 geckodriver::marionette TRACE -> 73:[0,2,"get",{"url":"http://comp1:43833/common/mousePositionTracker.html"}]
1503771502805 Marionette TRACE 0 -> [0,2,"get",{"url":"http://comp1:43833/common/mousePositionTracker.html"}]
1503771502811 Marionette DEBUG Received DOM event "beforeunload" for "about:blank"
1503771502883 Marionette DEBUG Received DOM event "pagehide" for "about:blank"
1503771502883 Marionette DEBUG Received DOM event "unload" for "about:blank"
1503771502969 Marionette DEBUG Received DOM event "DOMContentLoaded" for "http://comp1:43833/common/mousePositionTracker.html"
1503771502971 Marionette DEBUG Received DOM event "pageshow" for "http://comp1:43833/common/mousePositionTracker.html"
1503771502981 Marionette TRACE 0 <- [1,2,null,{}]
1503771502997 geckodriver::marionette TRACE <- [1,2,null,{}]
1503771502997 webdriver::server DEBUG <- 200 OK {"value": {}}
1503771503004 webdriver::server DEBUG -> POST /session/c716d576-e364-4122-9a4d-1962333ff7dc/element {"value":"#mousetracker","using":"css selector"}
1503771503004 geckodriver::marionette TRACE -> 68:[0,3,"findElement",{"using":"css selector","value":"#mousetracker"}]
1503771503015 Marionette TRACE 0 -> [0,3,"findElement",{"using":"css selector","value":"#mousetracker"}]
1503771503019 Marionette TRACE 0 <- [1,3,null,{"value":{"element-6066-11e4-a52e-4f735466cecf":"3d0d6af0-087b-462a-bd96-03fcb57b3a88","ELEMENT":"3d0d6af0-087b-462a-bd96-03fcb57b3a88"}}]
1503771503020 geckodriver::marionette TRACE <- [1,3,null,{"value":{"element-6066-11e4-a52e-4f735466cecf":"3d0d6af0-087b-462a-bd96-03fcb57b3a88","ELEMENT":"3d0d6af0-087b-462a-bd96-03fcb57b3a88"}}]
1503771503020 webdriver::server DEBUG <- 200 OK {"value":{"element-6066-11e4-a52e-4f735466cecf":"3d0d6af0-087b-462a-bd96-03fcb57b3a88"}}
1503771503035 webdriver::server DEBUG -> POST /session/c716d576-e364-4122-9a4d-1962333ff7dc/actions {"actions":[{"id":"default mouse","type":"pointer","parameters":{"pointerType":"mouse"},"actions":[{"duration":100,"x":95,"y":195,"type":"pointerMove","origin":{"ELEMENT":"3d0d6af0-087b-462a-bd96-03fcb57b3a88","element-6066-11e4-a52e-4f735466cecf":"3d0d6af0-087b-462a-bd96-03fcb57b3a88"}}]}]}
1503771503036 geckodriver::marionette TRACE -> 266:[0,4,"performActions",{"actions":[{"actions":[{"duration":100,"origin":{"element-6066-11e4-a52e-4f735466cecf":"3d0d6af0-087b-462a-bd96-03fcb57b3a88"},"type":"pointerMove","x":95,"y":195}],"id":"default mouse","parameters":{"pointerType":"mouse"},"type":"pointer"}]}]
1503771503046 Marionette TRACE 0 -> [0,4,"performActions",{"actions":[{"actions":[{"duration":100,"origin":{"element-6066-11e4-a52e-4f735466cecf":"3d0d6af0-087b-462a-bd96-03fcb57b3a88"},"type":"pointerMove","x":95,"y":195}],"id":"default mouse","parameters":{"pointerType":"mouse"},"type":"pointer"}]}]
1503771503171 Marionette TRACE 0 <- [1,4,null,{}]
1503771503171 geckodriver::marionette TRACE <- [1,4,null,{}]
1503771503171 webdriver::server DEBUG <- 200 OK {"value": {}}
1503771503174 webdriver::server DEBUG -> POST /session/c716d576-e364-4122-9a4d-1962333ff7dc/element {"value":"#status","using":"css selector"}
1503771503174 geckodriver::marionette TRACE -> 62:[0,5,"findElement",{"using":"css selector","value":"#status"}]
1503771503176 Marionette TRACE 0 -> [0,5,"findElement",{"using":"css selector","value":"#status"}]
1503771503178 Marionette TRACE 0 <- [1,5,null,{"value":{"element-6066-11e4-a52e-4f735466cecf":"d4586469-bad5-4bb7-8b65-d3d2d5c38421","ELEMENT":"d4586469-bad5-4bb7-8b65-d3d2d5c38421"}}]
1503771503178 geckodriver::marionette TRACE <- [1,5,null,{"value":{"element-6066-11e4-a52e-4f735466cecf":"d4586469-bad5-4bb7-8b65-d3d2d5c38421","ELEMENT":"d4586469-bad5-4bb7-8b65-d3d2d5c38421"}}]
1503771503178 webdriver::server DEBUG <- 200 OK {"value":{"element-6066-11e4-a52e-4f735466cecf":"d4586469-bad5-4bb7-8b65-d3d2d5c38421"}}
1503771503182 webdriver::server DEBUG -> GET /session/c716d576-e364-4122-9a4d-1962333ff7dc/element/d4586469-bad5-4bb7-8b65-d3d2d5c38421/text
1503771503183 geckodriver::marionette TRACE -> 68:[0,6,"getElementText",{"id":"d4586469-bad5-4bb7-8b65-d3d2d5c38421"}]
1503771503184 Marionette TRACE 0 -> [0,6,"getElementText",{"id":"d4586469-bad5-4bb7-8b65-d3d2d5c38421"}]
1503771503193 Marionette TRACE 0 <- [1,6,null,{"value":"78, 191"}]
1503771503193 geckodriver::marionette TRACE <- [1,6,null,{"value":"78, 191"}]
1503771503193 webdriver::server DEBUG <- 200 OK {"value":"78, 191"}
1503771503698 webdriver::server DEBUG -> GET /session/c716d576-e364-4122-9a4d-1962333ff7dc/element/d4586469-bad5-4bb7-8b65-d3d2d5c38421/text
1503771503698 geckodriver::marionette TRACE -> 68:[0,7,"getElementText",{"id":"d4586469-bad5-4bb7-8b65-d3d2d5c38421"}]
1503771503699 Marionette TRACE 0 -> [0,7,"getElementText",{"id":"d4586469-bad5-4bb7-8b65-d3d2d5c38421"}]
1503771503704 Marionette TRACE 0 <- [1,7,null,{"value":"78, 191"}]
1503771503704 geckodriver::marionette TRACE <- [1,7,null,{"value":"78, 191"}]
1503771503704 webdriver::server DEBUG <- 200 OK {"value":"78, 191"}
@andreastt based on your comment from July last year (https://github.com/mozilla/geckodriver/issues/789#issuecomment-314178944) it looks like that no spec issue has been filed, or?
I do not see any changes from FF or on geckodriver. I am using geckodriver 24 with FF 65.0.2 I see the same behaviour that it takes centre of any given element. We did an implementation to take the offset and click top-left corner.
I feel the solution to this problem has been left hanging in lot of places where I see similar questions. I recently had this problem with geckodriver 19 and FF 57. We also upgraded to FF 65 and gecko being 24. So, gecko is still continuing to have W3C specs intact but not the same with chromedriver (or not?).
We resolved this situation by,
WebElement ele = findElement(By.xpath("//div[contains(@class, 'columnClass')]"));
// Action to move to the element top-left corner
Actions userAction = new Actions(getDriver());
int getTopLeftY = ((ele.getSize().getHeight()/2) - ele.getSize().getHeight());
int getTopLeftX = (ele.getSize().getWidth()/2) - ele.getSize().getWidth();
userAction.moveToElement(ele, getTopLeftX, getTopLeftY).click().perform();
From this if you want to move to any point at the element. you may want to
int positionX = getTopLeftX + 100;
int positionY = getTopLeftY + 100;
Which will click at the point where you are anticipating.
@raguravindran I assume you didn't run chromedriver in w3c mode? It should fail the same way.
Also please read the documentation of geckodriver and use the following capability for now:
https://firefox-source-docs.mozilla.org/testing/geckodriver/Capabilities.html#moz-usenonspeccompliantpointerorigin
The bug is in selenium. The drivers for Chrome and Edge Dev (chromium), since version 75, both comply with the w3c. Selenium need to update their javadocs to say that it's from the centre. It would probably be too much of a faff to fix it in the selenium code because they won't know the dimensions of the element.
There is precedence for high-level Selenium client commands to run extra steps to alter the behaviour of commands. WebElement#submit() is one such example, as Selenium offered a way to submit a form but WebDriver (by the standard) doesn’t.
So it’s technically possible to mitigate the difference in behaviour for Selenium users by the clients first getting the bounding box of the element and then changing the click point using an action primitive. That said, if recent Chrome and Edge drivers are aligned with geckodriver’s behaviour perhaps it is not a priority.
int getTopLeftY = ((ele.getSize().getHeight()/2) - ele.getSize().getHeight()); int getTopLeftX = (ele.getSize().getWidth()/2) - ele.getSize().getWidth();
height/2 is wrong because of oveflow: scroll in parent and element height bigger than screen height (when a part of element hidden)
Is anyone still having issues with this particular behavior? If yes please report and provide a testcase + trace log. Otherwise I'm going to close this issue soon. Thanks.
Due to lack of response I'm going to close this issue for now. I'm happy to reopen if there are still use cases out there which do not work with the center of element approach.