SeleniumHQ
No way to to pass in information externally into test run - eg test value parameters
🐛 Bug Report
https://github.com/SeleniumHQ/selenium-ide/issues/720 2019: "As of right now, there's no way to pass in information externally into test runs in the IDE. This will change when we move to a native app. We'll be able to pass information into test runs through environment variables."
Expected behavior
Be able to supply values such as username/password combinations from a command line --params switch or from .side.yml config file.
Still no option to do this?
@JohnCarmichael3000 It doesn't seem too bad of an ask tbh. We're on native now. My only concern is that I make sure this works sensibly across both the side-runner and the IDE. What does this look like w/r/t the IDE to you?
I'm a bit new to the ide, so you'll have to forgive any oversights.
IDE reading parameters from a .side.yml or similar in the same folder as project .side file? Perhaps with a "parameters:" section and then in your .side a $() notation where for example $(password) could be replaced with actual value.
Motivation here is that I'd like to check in .side files but it's all too easy to check in secret-values when they are intermixed with your test code.
@JohnCarmichael3000 - That sounds fine for the IDE. I guess we can put in environment variable equivalents, and just use environment variables for those in the side-runner. That doesn't solve code export, but the approach to code-export needs to be rethought entirely anyway.
In my C# core based siderunner, I went with a CLI option for a "data" input file parameter like below. This allows me to run the same script against a number of targets where each environment might have different required inputs.
Then, when the script runs, if any named values are provided by the input JSON data file, then those values are used instead of whatever was being set in the script itself.
Maybe we can look at something similar for the IDE.
-d c:\temp\CCData2.json
where the json file has some or all the variable values used by the script like
{
"usersearch": "test",
"searchDisplayName": "Test2 Test2",
"category": "General Inquiry",
"date": "04/20/2024",
"attachment": "c:\\temp\\Link.png",
"username": "user",
"password": "password"
}
Hello 👋 Would it be possible in the soon future to pass environment variables to Selenium IDE/side runner?
@lesniakania - Lemme just double check, you want to stick something on process.env and use it in a test? Are you on v4?
Yes, exactly. I want to store some sensitive data in the ENV variables and be able to use it in a test (e.g. password to login to the site etc). I've also seen there is params option, but seems like it's not available to use just in the runner.
@lesniakania - I need to write new documentation for v4. That is for v3, and v4 is a bit different.
I can see the use for this. For my notes, I have the following questions:
- Are you using the side-runner for CI playback, or exporting code?
- Are you concerned about using the env variables in the ide as well as the runner?
For the first pass of this, I might just enable it in syntax and leave it up to you to figure out how to get the environment variables there. Would that work?
Thank you very much for so quick response! As for your questions:
- I'm running
selenium-side-runneron CircleCI - If there would be a way of using ENV variables both in Chrome extension and runner on CI, would be perfect, but just being able to use it on the CI, would be good enough :)
I think I would recommend instead supporting an input file as something like a JSON document with name value pairs like I mentioned above.
This gives you the ability to quickly spawn a number of parallel instances of test execution each with its own data set. I don't think the current side runner has this capacity. I've built this into my own C sharp based side runner however and would be happy to share how it works.
When we run a battery of tests performing stress loads with chrome driver we can simulate say 50 simultaneous user accounts each running separate scripts by using this input method of handing in the dynamic data to use in each test set. We store our test set data in a Git repository as well making it easy to manage and share.
@lesniakania - Ah, I understand. I'm sorry, I'm not worried about adding features to v3, as the entire extension is effectively living on borrowed time.
@coinzdude - I am fine with file helpers as long as that isn't the primary access layer. I want to ensure that our core design doesn't promote any level of degrading environment variables into plaintext files.
@coinzdude - To give a bit more context, I think these are two separate features:
- env command syntax: in your value, specify env command:
test-user+${env.CIRCLE_ID_THING}@xyz.com - load variables from path: a new command that reads a file of some format easily accessed by all languages (json?) and populates entries in the variable pool accordingly
@lesniakania - 100% no in web extension. I'll see if there's an easy way to add this in v4.
@lesniakania a little more context here. There should be no further development for the web extension. It's not long term viable as it interacts differently with the browser than chromedriver and generates incompatible scripts for headless or scriptable playback.
I also can't look into this soon. I'm not familiar with the script runner code yet.
@coinzdude @lesniakania - There's also looming webextension v3 manifest changes, which are the equivalent of the giant boot coming for the flighty cockroach that is Selenium IDE. If you're on the side-runner, I'll look at adding mechanisms for this to the side-runner, and I'd appreciate if you can give v4 a try after this weekend and let me know of any outstanding pain points. The reason this feature is so far out is I need to break fix the whole binary, write a docs site, and improve test coverage of the IDE with the next couple weekends.
EDIT: Which is why I'm asking you to try it next week. Things is broked this week on latest haha sorry.
I ported the popular JavaScript Errors Notifier requiring manifest v2 to v3 conversion and it was tedious. What was prior directly callable now requires message events to the background service and it can be quite a challenge dealing with the storage requirements changes. I would not think it is worth trying to preserve the extension.
@coinzdude - Yeah, and honestly I don't know how to lift some of the commands. Execute script, calling a fully dynamic eval string in a user browser window, is basically what the manifest changes are shifting to disallow.
@toddtarsi I also have one more use case for the environment variables or any kind of option for passing params to the tests. I was implementing tests that require file upload and to make it work, you need to pass the full path, from the root directory. But you can't really hardcode it, if you want those tests to run on different environments. Each team member have different setup, we also run those tests on CI.
@lesniakania - This is like priority number 2 or 3 for me. I noticed that there's no way in current IDE to see or edit opening window attributes (which leads to both sneaky behaviors and very bad times), so I'm going to have to add controls around that, and then I'll start taking a look at creating env pass in syntax. My goal is this weekend to put something down for both of these, but also I'm a bit optimistic and out of reality for self expectations
@lesniakania - Sorry for the delay, this should work in latest? I believe as long as you refer to it as a variable and prefix it with env:, everything should be fine:
example: "${env:NODE_ENV}"
@lesniakania - Also don't try to set stuff this way plz.
eg: store | "production" | "${env:NODE_ENV}" <-- this will not work as expected