SeleniumLibrary icon indicating copy to clipboard operation
SeleniumLibrary copied to clipboard

Published 20 hours ago • verified publisher icon robotframework

Add parameter to make Wait Until Page Contains case insensitive

Open apuchkov opened this issue 5 years ago • 12 comments

It would be great to be able to turn off case sensitivity for keywords like Element Text Should Be, Wait Until Element Contains, Wait Until Element Does Not Contain, Wait Until Page Contains and Wait Until Page Does Not Contain

This feature was already implemented for Element Should Contain and Element Should Not Contain in #849.

I spent lots of time trying to understand why Wait Until Page Contains fails just to find that it is a case sensitive and case on the screen was displayed differently, because of CSS style: text-transform: lowercase

I would vote to make these keywords case insensitive by default and case sensitive via parameter. I think it would make tests faster to write and less brittle.

apuchkov avatar Nov 19 '19 19:11 apuchkov

For all the keywords that verifies text from from a single element, adding the case insensitive argument sounds like good idea. Also they should not be complicated to do. But the Page Should (Not) Contain keywords are problematic, because they use xpath internally to find the text from the page. And if I recall correctly, xpath doesn't support this type of conversions. But there might be other solutions available and investigation is a good idea.

Do you have @apuchkov have an interest to provide a PR for this enhancement request?

aaltat avatar Nov 19 '19 20:11 aaltat

:+1: for this in general but :-1: for making case-insensitivity the default.

pekkaklarck avatar Nov 19 '19 20:11 pekkaklarck

I agree that keywords should stay case sensitive by default.

aaltat avatar Nov 19 '19 21:11 aaltat

Since Page Should (Not) Contain depends internally on is_text_present (xpath expression: "xpath://*[contains(., %s)]" % escape_xpath_value(text)), is it possible to alter that function to add case insensitivity and then perform a similar solution as:

https://stackoverflow.com/questions/2893551/case-insensitive-matching-in-xpath OR https://stackoverflow.com/questions/8474031/case-insensitive-xpath-contains-possible

What would be the pitfalls of this? I'm new to open source and would love to contribute in any way I can. Hope this wasn't too silly a question.

ritikasaboo avatar Dec 06 '19 14:12 ritikasaboo

Last time I did check, browsers did not support xpath 2. But that was pretty long time ago, so things might have changed. But it can be easily tested by using browser developer tools.

But if you find out a combination which works with as many browsers as possible, we can make the xpath configurable. You don't need SeleniumLibrary to test it, just use different browsers developer tools. If you happen to find a good combination (good means that works as many browser as possible), we can talk more.

aaltat avatar Dec 06 '19 14:12 aaltat

Yes, upper & lower case functions are part of Xpath 2.0

However, there is a workaround that works for Xpath 1.0:

//*[
  contains(
    translate(., 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz'),
    'test'
  )
]

I've checked this for Chrome & Firefox using developer tools, and I am able to find text via xpath case insensitively. I think this should also work for Edge/IE; will confirm on that soon.

How do you want to proceed? I'd love to contribute on this. Thanks!

ritikasaboo avatar Dec 08 '19 20:12 ritikasaboo

Does work with characters outside of the ASCII range, example with utf-8 and utf-16?

Those xpath2 functions would be good solution, did you check does the browser support them nowdays?

aaltat avatar Dec 08 '19 20:12 aaltat

Here's a sample example I ran to check a non-ASCII character:

`<html>
<head></head><body>
<p title="empire"> teXTĀ </p>
<p title="Empire"> TeXTĀ </p>
<p title="EMpire"> textĀ </p>
<p title="EMPire"> texTā </p>
</body></html>
`

xpath 1.0 expression: //p[contains(translate(.,"TEXTĀ","textā"), "textā")]

This actually identifies all 4 p-tags, which leads me to believe that the translate() function supports ASCII and non-ASCII characters.

As for Xpath 2.0, I believe it is not supported by browsers still. Search results from a couple of years ago suggest that there's no future plans to support Xpath 2.0 by browsers. Additionally, running //p[lower-case(@title)='empire'] against above web page gives no results in Chrome, which probably confirms that Xpath 2.0 is NOT supported.

Is it possible to perform string manipulation on the search text in python and then search via xpath, which means we may not need case-insensitive xpath solutions in the first place? Let me know how you want to proceed.

ritikasaboo avatar Dec 09 '19 06:12 ritikasaboo

Well, almost everything is possible in the Python side, keyword could construct the translated text inside of the keyword. Most likely Python upper() and lower() should work pretty well. Needs testing with all kinds of funny characters and therefore looking example big list of naughty strings might be useful. SeleniumLibrary has subset of blns in use.

aaltat avatar Dec 09 '19 08:12 aaltat

Do you want to use the xpath translate() solution or the Python string manipulation approach? How do you want me to proceed with providing an acceptable solution? Could you guide me through this process, if you can afford the time? I'd like contribute to this issue. Thanks, let me know.

I could perform string manipulation on the search text but it would have to cover all possibilities of case-insensitivity on the page. I think the xpath solution is the way to go? A

ritikasaboo avatar Dec 09 '19 09:12 ritikasaboo

Well, there are two factors to take in account.

  1. For the keywords that deal with element, like Wait Until Element Contains, it is easier to do conversion in Python side. For those keywords one can copy the implementation from Element Should Contain and Element Should Not Containkeywords.
  2. The Page Should (Not) Contain keywords needs research is it possible to implement ignore_case argument. So far I can think several possibilities how it can be done, but all of them requires research. Also without doing or reading the implementation, it is impossible to say which way I would prefer. Because there are always so many things to learn during the implementation, which may lead to the implementation to totally new areas. But generally code should follow the coding conventions set to the project.

For me it is always easiest to do TDD when writing implementation for any thing.

aaltat avatar Dec 09 '19 21:12 aaltat

I believe keywords such as Wait Until Element Contains already have a case-insensitive implementation that involve conversion on the Python side.

With regards to utf-8/utf-16/blns-- how exactly do you want me to map case-wise? I would need a comprehensive list of characters with case mappings to ensure the xpath translate() function covers non-ASCII. If I have this, I can attempt one kind of solution for starters and we can explore post that implementation. Please let me know/guide me on this.

So far I can think several possibilities how it can be done, but all of them requires research.

What other possible implementations come to mind? I don't think the implementation applied to Element Should Contain can be rehashed for this.

ritikasaboo avatar Dec 11 '19 10:12 ritikasaboo