(Ruby - Cucumber)Not all scenarios are shown in report when they are run in parallel

[penknife]

Describe the bug When I run scenarios for example in 5 threads (1350 scenarios) (via parallel_tests gem), I can see only 3 threads (940 scenarios for example) in Repost Portal launch. I cannot see all scenarios which should be run. I have thoughts that a not all scenarios are run with -f ReportPortal::Cucumber::ParallelFormatter.

When I have run scenarios with default html report everything good. I have seen for request in build log such things with params /item?filter.eq.parent=5bf30a0bb9d8210001c48485&filter.eq.name=Folder:%20scenarios produced an exception:

RestClient::BadRequest: 400 Bad Request"  or 
" /Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/request.rb:377:in `block in make_headers' : warning: Overriding "Content-Type" header "multipart/form-data" with "multipart/form-data; boundary=----RubyFormBoundary6ioZJqz4TJecRnfc" due to payload (StructuredWarnings::StandardWarning)"

Expected behavior I should see all 5 threads in launch and all run scenarios

Versions: -Mac OS

• gem for report portal support have used https://github.com/reportportal/agent-ruby#formatter_modes
• Version of RP [Current version: Analysis Service: 4.3.0; API Service: 4.3.0; Index Service: 4.2.0; Jira Service: 4.3.0; Rally Service: 4.3.0; Authorization Service: 4.2.0; Service UI: 4.3.0; ]

[DzmitryHumianiuk]

@aliaksandr-trush, @abotalov, any ides?

[penknife]

Dudes DO you have any ideas?

[aliaksandr-trush]

@penknife

1. "warning" message doesn't related to the issue.

2. The issue is possible when one of the formatter threads received error and unable to continue. Could you attach more logs around "RestClient::BadRequest: 400 Bad Request"?

[penknife]

Also I have seen this warnings and errors in log

/Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/request.rb:377:in `block in make_headers' : warning: Overriding "Content-Type" header "multipart/form-data" with "multipart/form-data; boundary=----RubyFormBoundaryPkAAQ2N7fi9m1vjh" due to payload (StructuredWarnings::StandardWarning)

[Step 5/5] undefined method `key?' for nil:NilClass (NoMethodError)
[01:12:05] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:165:in block in close_child_items'" [01:12:05]	[Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:166:in block in close_child_items'
[01:12:05] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:164:in loop'" [01:12:05]	[Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:164:in loop'
[01:12:05] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:164:in close_child_items'" [01:12:05]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:179:in block in close_child_items'"
[01:12:05] [Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:164:in close_child_items' [01:12:05]	[Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:179:in block in close_child_items'
[01:12:05] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:178:in each'" [01:12:05]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:178:in close_child_items'"
[01:12:05] [Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:178:in each' [01:12:05]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_report.rb:61:in done'"
[01:12:05] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:37:in public_send'" [01:12:05]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:37:in block (2 levels) in initialize'"
[01:12:06] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:35:in loop'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:35:in block in initialize'"
[01:12:06] [Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:178:in close_child_items' [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/abstract_response.rb:223:in exception_with_response'"
[01:12:06] [Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_report.rb:61:in done' [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/abstract_response.rb:103:in return!'"
[01:12:06] [Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:37:in public_send' [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/request.rb:809:in process_result'"
[01:12:06] [Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:37:in block (2 levels) in initialize' [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/request.rb:725:in block in transmit'"
[01:12:06] [Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:35:in loop' [01:12:06]	[Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:35:in block in initialize'
[01:12:06] [Step 5/5] "/Users/runner/.rvm/rubies/ruby-2.3.7/lib/ruby/2.3.0/net/http.rb:853:in start'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/request.rb:715:in transmit'"
[01:12:06] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/request.rb:145:in execute'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/request.rb:52:in execute'"
[01:12:06] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/resource.rb:51:in get'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:165:in block (2 levels) in close_child_items'"
[01:12:06] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:198:in do_request'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:165:in block in close_child_items'"
[01:12:06] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:164:in loop'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:164:in close_child_items'"
[01:12:06] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:179:in block in close_child_items'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:178:in each'"
[01:12:06] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:178:in close_child_items'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_report.rb:61:in done'"
[01:12:06] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:37:in public_send'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:37:in block (2 levels) in initialize'"
[01:12:06] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:35:in loop'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:35:in block in initialize'"

[penknife]

@penknife

1. "warning" message doesn't related to the issue.
2. The issue is possible when one of the formatter threads received error and unable to continue. Could you attach more logs around "RestClient::BadRequest: 400 Bad Request"?

Will be it enough details?

[penknife]

Aso I have seen that for example from 4 threads in report I have 3 threads, so some threads are not shown in RP or not launch

[DzmitryHumianiuk]

@aliaksandr-trush can you assist with it? @penknife is it still actual?

[mikesalvia]

@DzmitryHumianiuk There has been some work done to work on the parallel executions, it is unfortunately in a branch. I can confirm that there are issues when running in parallel but we have worked to reduce those issues.

[mikesalvia]

https://github.com/reportportal/agent-ruby/pull/33 << work that has been done. I need to do some more fixes though.

[DzmitryHumianiuk]

@mikesalvia i see it merged. Do we only need to release it?

[abotalov]

It targeted a different branch. I'm not sure if that branch was properly tested.

[mikesalvia]

Is this still active!?

[HardNorth]

@mikesalvia No one knows. There is no Ruby contributor.

[mikesalvia]

Oh yikes!