[Transformer] Normalize variable name 'separators'

[ealap]

In RF, single spaces and underscores in variable names are ignored. It would be nice to have a way to normalize this depending on the preferred separator.

Given

${variable separated by space}
@{variable_separated_by_underscore}
&{variable_with mixed separator}

Normalize variable names to use underscores

${variable_separated_by_space}
@{variable_separated_by_underscore}
&{variable_with_mixed_separator}

Normalize variable names to use single space

${variable separated by space}
@{variable separated by underscore}
&{variable with mixed separator}

Totally removing the separators may be supported as well but of course this will be nearly impossible to convert back to separated formats.

[bhirsz]

It sounds useful and it could be also later extended by normalizing the case (upper case for globals etc). I only wonder if it should be enabled by default or not?

We could also try to auto detect if variables in file use underscores or spaces but that's optional for the future, we can start with enforcing either of the two (most likely underscores) and allow to configure it.

[ealap]

I'm leaning towards disabling it by default since RF doesn't really favor one style over the other.

[bhirsz]

I've created working POC (https://github.com/MarketSquare/robotframework-tidy/tree/rename_variables). There are some edge cases I need to think over so it's most likely aimed for release 3.1.

Case1: Variables in values:

*** Variables ***
${_VAR}    1
@{VARS}    0    ${ VARS}

Case1 extended: Variables in strings:

*** Keywords ***
Keyword
    Keyword Call    this is string with ${var_iable}

[ealap]

Thanks @bhirsz! I gave it a try and found the following issues

1. It seems to be adding a separator between two variables

2. It is not normalizing names on variables written as values under Variable section

[bhirsz]

Thanks for testing - yeah recognizing the variables is bit primitive, I will replace that logic with the one I used somewhere else for detecting embedded variables. Or perhaps simple regex pattern should suffice since we're not interested in position etc of the variable, we only want to sub patterns in variables. I will test it.

[bhirsz]

I realized I can't use regex pattern because the robot variables can be nested. I can use logic similar to what I did in Robocop https://github.com/MarketSquare/robotframework-robocop/blob/32c46484ca7c618dd2a7bf8445d27d542025e7bd/robocop/utils/misc.py#L291 instead

[bhirsz]

Adding more edge cases and loose thoughts (bit chaotic but I wanted to record it):

# item access
${VAR['key']}  # not sure if it's still supported in RF 4? Robotidy only supports 4+
# python objects
${OBJECT.eat('Cucumber')}
# name access
$variable
# variable in item access
${_var_[${ var2}]}

We can't cover everything so it will be responsiblity of the user to verify if syntax he/she is using still works as expected after running the transformer. The variable resolving is dynamic in Robot Framework. For example it will first check if ${DICT[2]} name is defined somewhere, then it will try to find "DICT" and access it's value with key=2. We can at least try to follow some of the rules for variable name resolving:

1. Variable starts with either of $, @, &, % (and is not preceded by escape char \), followed by { and ends with }. We also need to differentiate from inline Python evaluation (${{ eval }}).
2. Variable can contain other variables in a name.
3. The end of the variable name is when we got any non-alphanumeric character (except for separators and embeded variables).

With this approach with this variable:

${letters_second_${embedded.key()} third['key_']} 

we will only 'parse' following sub-strings:

letters_seconds_
embedded
 third

[bhirsz]

RenameVariables transformer that handles it and much more was just released in Robotidy 4.0! However I recommend waiting till the Robotidy 4.0.1 is (soon) released because I introduced regression.