Selenium 4 dynamic grid - add support for get downloaded files to your host path

[rdark100]

🚀 Feature Proposal

Selenium 4 Dynamic Grid - get downloaded files from eg. /home/seluser/Downloads to your host path. It should be same as get video file when you have to mount your host path (/path/on/your/host/machine:/opt/selenium/assets)

[diemol]

I think we could implement an endpoint that allows a user download the file(s) from the session. Mounting volumes might work well for local Docker deployments but it gets complicated when Grid is deployed without Docker or it gets deployed through Docker Swarm or Kubernetes.

However, this is not a priority right now, as we are focusing on the Selenium 4 release.

As usually, all contributions are welcome.

[diemol]

Actually, I am now questioning myself if we should do this or not. What is the use case, @rdark100?

[brendon-r]

I know the Selenium docs discourage using Selenium to perform file downloads but that's our most reliable way of performing our testing. We have an app that we test via downloaded files (specifically, we test Power BI charts by exporting them and comparing to the data source) and with Selenium Grid, I'd have to mess around with volume mounts to be able to get to the downloaded files. I've since switched to Selenosis mainly for this reason.

[rdark100]

I have similiar usage as @beerai . I tests exports of our tables and the export is not same as table in web browser - more columns in exported file (currency is separated, more decimal places defined, etc.)

[MartykQ]

This would be much more useful if it also worked in the other direction => sharing some files from the host machine to the container. Use case: testing any kind of uploads/imports. Right now copying files from the host machine to the container directly is needed each time

[diemol]

@MartykQ this is already possible directly from the client, see: https://www.selenium.dev/documentation/webdriver/remote_webdriver/#local-file-detector

[ablok]

We also test that file generation works correctly by downloading files and checking their contents and would like to use the dynamic grid for these tests. We don't really have another way of testing that this works e2e. File generation is dependant on some client side info (session/cookie/data) which makes it pretty difficult to easily generate a direct download as an alternative. Having an endpoint for downloading files of the client would be super useful.

[krmahadevan]

@diemol - This could be one of the use cases for having custom routes being supported in the node being asked in https://github.com/SeleniumHQ/selenium/issues/9708 which I had earlier tried to address via https://github.com/SeleniumHQ/selenium/pull/11233

I tried hacking into whatever is available by doing the following:

1. I created a custom executor which was then fed into the RemoteWebDriver for instantiation.
2. I built a custom node based on the documentation here wherein I tried intercepting the public HttpResponse executeWebDriverCommand(HttpRequest req) to filter out my custom method and then try to handle it, but I am hitting

org.openqa.selenium.UnsupportedCommandException: Unable to find handler for (GET) /DOWNLOAD
Build info: version: '4.6.0', revision: '79f1c02ae20'
System info: os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '12.5', java.version: '11.0.13'
Driver info: driver.version: unknown

I am still trying to debug as to where is this edit check being triggered.

My sample test

import com.google.common.collect.ImmutableMap;
import java.net.URL;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.remote.Command;
import org.openqa.selenium.remote.CommandExecutor;
import org.openqa.selenium.remote.CommandInfo;
import org.openqa.selenium.remote.CommandPayload;
import org.openqa.selenium.remote.HttpCommandExecutor;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.remote.Response;
import org.openqa.selenium.remote.SessionId;
import org.openqa.selenium.remote.TracedCommandExecutor;
import org.openqa.selenium.remote.http.ClientConfig;
import org.openqa.selenium.remote.http.HttpClient;
import org.openqa.selenium.remote.http.HttpMethod;
import org.openqa.selenium.remote.tracing.TracedHttpClient;
import org.openqa.selenium.remote.tracing.Tracer;
import org.openqa.selenium.remote.tracing.opentelemetry.OpenTelemetryTracer;

public class App {

  public static void main(String[] args) throws Exception {
      RemoteWebDriver rwd = null;
      try {
          CommandExecutor executor = createExecutor(new URL("http://localhost:4444"), true);
          rwd = new RemoteWebDriver(executor, new ChromeOptions());
          rwd.get("https://www.google.com");
          System.err.println("Title: " + rwd.getTitle());
          SessionId sessionId = rwd.getSessionId();
          CommandPayload commandPayload = new CommandPayload("DOWNLOAD", ImmutableMap.of("file", "docker-compose.yml"));
          Command downloadCommand = new Command(sessionId, commandPayload);
          Response response = rwd.getCommandExecutor()
              .execute(downloadCommand);
          System.err.println(response.getValue());
      }finally {
          if (rwd != null) {
              rwd.quit();
          }
      }
  }

    private static CommandExecutor createExecutor(URL remoteAddress, boolean enableTracing) {
        ClientConfig config = ClientConfig.defaultConfig().baseUrl(remoteAddress);
        if (enableTracing) {
            Tracer tracer = OpenTelemetryTracer.getInstance();
            CommandExecutor executor = new HttpCommandExecutor(
                ImmutableMap.of("DOWNLOAD", new CommandInfo("/DOWNLOAD", HttpMethod.GET)),
                config,
                new TracedHttpClient.Factory(tracer, HttpClient.Factory.createDefault()));
            return new TracedCommandExecutor(executor, tracer);
        } else {
            return new HttpCommandExecutor(config);
        }
    }
}

Here's my relevant method from my custom node implementation

  @Override
  public HttpResponse executeWebDriverCommand(HttpRequest req) {
    try {
      System.err.println("Before executeWebDriverCommand(): " + req.getUri());
      if (req.getUri().endsWith("DOWNLOAD")) {
        return new HttpResponse().setStatus(HTTP_OK)
            .setContent(Contents.asJson(ImmutableMap.of("download", "fine")));
      } else {
        return node.executeWebDriverCommand(req);
      }
    } finally {
      System.err.println("After executeWebDriverCommand()");
    }
  }

[diemol]

@krmahadevan there is no need to add custom rules. When we implement the download functionality in Grid, it would be by adding a route on the Node next to the one it is used to upload a file. Probably some rules need to be defined to figure out what can be selected on the local file system on the Node before the download is done.

[krmahadevan]

@diemol

When we implement the download functionality in Grid

Is this being worked upon ?

[diemol]

No, but it is something that will be added.

[krmahadevan]

Is there some place wherein the scope for it is defined? Or is it something as straight forward as adding method similar to uploadfile within the Node implementation? I would like to take a jab at this and hence the curiosity.

[diemol]

I think the wiring can be done, and it should not be too complicated. Ideally following the same concept of the file upload should be a good start. When we are there, we need to define how the file to be downloaded can be selected.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.