utPLSQL-cli icon indicating copy to clipboard operation
utPLSQL-cli copied to clipboard

Excessive reporters sessions

Open jgebal opened this issue 6 years ago • 6 comments

When I run utPLSQL-cli with multiple reporters and outputs like:

time utPLSQL-cli/bin/utplsql run ${UT3_TESTER}/${UT3_TESTER_PASSWORD}@${CONNECTION_STR} \
-source_path=source -owner=ut3 \
-test_path=test -c \
-f=ut_sonar_test_reporter     -o=test_results.xml \
-f=ut_junit_reporter          -o=junit_test_results.xml \
-f=ut_tfs_junit_reporter      -o=tfs_test_results.xml \
-f=ut_sonar_test_reporter     -o=test_results1.xml \
-f=ut_junit_reporter          -o=junit_test_results1.xml \
-f=ut_tfs_junit_reporter      -o=tfs_test_results1.xml \
-f=ut_sonar_test_reporter     -o=test_results2.xml \
-f=ut_junit_reporter          -o=junit_test_results2.xml \
-f=ut_tfs_junit_reporter      -o=tfs_test_results2.xml \
-f=ut_sonar_test_reporter     -o=test_results3.xml \
-f=ut_junit_reporter          -o=junit_test_results3.xml \
-f=ut_tfs_junit_reporter      -o=tfs_test_results3.xml \
-f=ut_sonar_test_reporter     -o=test_results4.xml \
-f=ut_junit_reporter          -o=junit_test_results4.xml \
-f=ut_tfs_junit_reporter      -o=tfs_test_results4.xml \
-f=ut_documentation_reporter  -o=test_results.log -s

Ever reporter allocates separate session. This can be challenging for environments, where users have limit of concurrent sessions.

It seems that API/cli could use only 2 sessions and still be successful. session 1 - main session for running tests and collecting report outputs. session 2 (asynchronous) for collecting real-time report info and putting it into screen.

Alternatively, maybe there should be a parameter for cli to set maximum number of concurrent sessions. This could be minimum of 2 - like in the example above and max could be as many as needed. If set to 0, there is no limit, if set to 1 all is synchronous. default = 2?

jgebal avatar Jan 12 '19 17:01 jgebal

I see the potential problem, but I ask myself: Is this a real issue at the moment? Do we have real-life scenarios currently like that? I'd even wonder why anyone would use so many reporters (if not for testing purpose).

I'd also not introduce another parameter for something like that. I'm not sure we won't run into serious troubles if we use only one single connection and if we change the internal behavior of connection usage to 2, this is nothing I want a user to decide (I also see no benefit in passing that decision to the user).

pesse avatar Jan 14 '19 07:01 pesse

Well... It's probably not just hypothetical issue. I actually have higly shared DB, where developer accounts have concurrent session limit of 4. So if I want 4 reports I neet to run utPLSQL twice as 4 reports require 5 sessions.

Flexibility of max sessions as parameter would be great but isn't a must have. Realistic request:

  • Limit to 2 sessions
  • Parameter of max sessions with minimum value of 2. If less than 2 provided, fail or default to 2.

jgebal avatar Jan 14 '19 08:01 jgebal

Ok, so we have a real-life need to reduce the concurrent sessions.

Can you explain what's the benefit you see in adding increased complexity by adding a parameter for "max sessions"?

pesse avatar Jan 14 '19 08:01 pesse

Potentially faster execution time if reading reporters output takes more time. This is however only a hypothesis so probably no real benefit until proven otherwise.

jgebal avatar Jan 14 '19 08:01 jgebal

Thought about this topic again. What we could do: 1 Session to run the tests 1 Session to (live-)consume the results of the reporter that goes to screen or the first mentioned reporter 1 Session to consume all the other reporters after the run itself is finished

That way we would always use at most 3 sessions. What do you think @jgebal ?

pesse avatar Jul 10 '19 14:07 pesse

Well, actually we could reuse the session of the live-consuming reporter so it would be 2 sessions at all. At the cost of the secondary reporters only being written after the complete test-run.

pesse avatar Jul 10 '19 14:07 pesse