Seemingly non-deterministic failures on download testing in RSpec tests

[JamesGlover]

When running RSpec tests we see failures of the form:

  1) Cherrypicking pipeline where picking by ng for a tecan robot behaves like a cherrypicking procedure running the cherrypicking pipeline
     Failure/Error: raise StandardError, "Could not open #{file} in #{PATH} after #{timeout} seconds"
     
     StandardError:
       Could not open 238_batch_DN4V_1.gwl in /Users/jg16/projects/sanger/sequencescape/tmp/capybara after 10 seconds
     Shared Example Group: "a cherrypicking procedure" called from ./spec/features/pipelines/cherrypick/cherrypick_pipeline_spec.rb:328
     # ./spec/support/download_helper.rb:26:in `rescue in wait_for_download'
     # ./spec/support/download_helper.rb:23:in `wait_for_download'
     # ./spec/support/download_helper.rb:13:in `downloaded_file'
     # ./spec/features/shared_examples/cherrypicking.rb:341:in `block (6 levels) in <top (required)>'
     # ./spec/features/shared_examples/cherrypicking.rb:240:in `block (5 levels) in <top (required)>'
     # ./spec/features/shared_examples/cherrypicking.rb:167:in `block (4 levels) in <top (required)>'
     # ./spec/features/shared_examples/cherrypicking.rb:165:in `each'
     # ./spec/features/shared_examples/cherrypicking.rb:165:in `block (3 levels) in <top (required)>'
     # ./spec/features/shared_examples/cherrypicking.rb:163:in `block (2 levels) in <top (required)>'
     # ------------------
     # --- Caused by: ---
     # Timeout::Error:
     #   execution expired
     #   ./spec/support/download_helper.rb:24:in `sleep'

This happens because the downloaded files are saved to the project root, rather than the Capybara.save_path (./tmp/capybara). However, when runn in isolation, this failure is not reproduced.

Furthermore, even with a fixed test order, failure does not appear to be deterministic. When running:

be rspec './spec/api/dilution_plate_purpose_spec.rb[1:2:2:1]' './spec/features/generate_a_bulk_submission_template_spec.rb[1:1,1:2:1,1:3:1]' './spec/features/pipelines/cherrypick/cherrypick_pipeline_spec.rb[1:2:1:1:1,1:2:2:1:1,1:3:1:1,1:4:1:1,1:5:1:1,1:6:1:1,1:7:1:1,1:7:2:1:1,1:8:1:1,1:8:2:1:1]' './spec/features/tag_group_spec.rb[1:1,1:2]' './spec/lib/record_loader/submission_template_loader_spec.rb[1:1:3]' './spec/sample_manifest_excel/configuration_spec.rb[1:5:7]' --seed 34572 --format documentation --fail-fast

I would observe failure only about 50% ofd the time.

Observations:

• Accessing page.driver.browser on a failed test (pry session in the rescue hook of DownloadHelper.wait_for_download then up until you reach the test itself) revealed that the download directory should still be set to Capybara.save_path
• The deprecation warning seen upon setting the download directory appear to originate from within Capybara itself, however the setting is not being ignored entirely. Removing the download directory in spec/spec_helper.rb results in failure 100% of the time.