jitsi-meet-torture icon indicating copy to clipboard operation
jitsi-meet-torture copied to clipboard

Published 20 hours ago • verified publisher icon jitsi

Tests Failing when using torture .We tried on our self hosted server and then tried on jitsi meet (https://meet.jit.si/) and getting the same errors as below

Open rvvssvraju opened this issue 3 years ago • 5 comments 

[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   ActiveSpeakerTest>AbstractBaseTest.setupClassPrivate:188->setupClass:45->WebTestBase.ensureThreeParticipants:199->WebTestBase.ensureThreeParticipants:180->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   AudioOnlyTest>AbstractBaseTest.setupClassPrivate:188->setupClass:44->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   AvatarTest>AbstractBaseTest.setupClassPrivate:188->setupClass:51->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   ChatPanelTest>AbstractBaseTest.setupClassPrivate:188->setupClass:43->WebTestBase.ensureOneParticipant:56->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   DesktopSharingTest.testDesktopSharingStart:47->WebTestBase.ensureOneParticipant:56->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   DisplayNameTest>AbstractBaseTest.setupClassPrivate:188->setupClass:49->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   EndConferenceTest>AbstractBaseTest.setupClassPrivate:188->setupClass:36->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   EtherpadTest>AbstractBaseTest.setupClassPrivate:188->setupClass:49->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   FollowMeTest>AbstractBaseTest.setupClassPrivate:188->setupClass:48->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   InviteTest>AbstractBaseTest.setupClassPrivate:188->setupClass:47->WebTestBase.ensureOneParticipant:56->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   KickTest>AbstractBaseTest.setupClassPrivate:188->setupClass:41->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   LastNTest.testLastN:48->WebTestBase.ensureOneParticipant:68->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   LipSyncTest.testLipSync:114->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   LobbyTest>AbstractBaseTest.setupClassPrivate:188->setupClass:45->WebTestBase.ensureOneParticipant:56->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   LockRoomDigitsOnlyTest.lockRoomWithDigitsOnly:44->WebTestBase.ensureOneParticipant:68->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   LockRoomTest>AbstractBaseTest.setupClassPrivate:188->setupClass:50->WebTestBase.ensureOneParticipant:56->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   ModeratorTest>AbstractBaseTest.setupClassPrivate:188->setupClass:49->WebTestBase.ensureOneParticipant:56->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   MuteTest>AbstractBaseTest.setupClassPrivate:188->setupClass:58->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   OneOnOneTest.testFilmstripHiddenInOneOnOne:58->WebTestBase.ensureTwoParticipants:104->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   Peer2PeerTest.testSwitchToP2P:49->WebTestBase.ensureTwoParticipants:104->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   PreJoinTest.testJoinWithoutAudio:84->WebTestBase.joinFirstParticipant:347->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   PreJoinTest.testPreJoinForRoomWithoutLobby:65->WebTestBase.joinFirstParticipant:347->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   PreJoinTest.testPreJoinWhenDisplayedNameRequired:40->WebTestBase.joinFirstParticipant:347->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   SharedVideoTest>AbstractBaseTest.setupClassPrivate:188->setupClass:72->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   SinglePortTest>AbstractBaseTest.setupClassPrivate:188->setupClass:38->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   StartMutedTest>AbstractBaseTest.setupClassPrivate:188->setupClass:39->WebTestBase.ensureOneParticipant:56->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   StopVideoTest>AbstractBaseTest.setupClassPrivate:188->setupClass:57->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   SwitchVideoTest>AbstractBaseTest.setupClassPrivate:188->setupClass:57->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   TileViewTest>AbstractBaseTest.setupClassPrivate:188->setupClass:54->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   UDPTest>AbstractBaseTest.setupClassPrivate:188->setupClass:37->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   VideoLayoutTest>AbstractBaseTest.setupClassPrivate:188->setupClass:44->WebTestBase.ensureOneParticipant:56->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[INFO] 
[ERROR] Tests run: 290, Failures: 31, Errors: 0, Skipped: 256
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  45.882 s
[INFO] Finished at: 2021-06-01T19:30:15+05:30
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.20:test (default-test) on project jitsi-meet-torture: There are test failures.
[ERROR] 
[ERROR] Please refer to /home/vignesh/jitsi-meet-torture/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

rvvssvraju avatar Jun 01 '21 14:06 rvvssvraju

Same here; any updates?

dkgrieshammer avatar Nov 27 '21 20:11 dkgrieshammer

How are you running the test?

damencho avatar Nov 27 '21 23:11 damencho

How are you running the test?

@damencho I just cloned the repo & run MuteTest & TCPTest, fails against own install as well as on meet.jit.si; mvn test -Djitsi-meet.instance.url="https://meet.jit.si" -Djitsi-meet.tests.toRun="MuteTest,TCPTest"

I did that to narrow down the errors I got using the docker-hub version on local macbook as well as on an AWS Debian; any hints would be highly appreciated 🙏

Error Log:

[1] End tcpTest.
[1] Closing web.participant1
[1] Closing web.participant2
[ERROR] Tests run: 415, Failures: 2, Errors: 0, Skipped: 406, Time elapsed: 88.967 s <<< FAILURE! - in TestSuite
[ERROR] muteAfterJoinCanShareAndUnmute(org.jitsi.meet.test.MuteTest)  Time elapsed: 25.645 s  <<< FAILURE!
org.openqa.selenium.TimeoutException:
Expected condition failed: waiting for org.jitsi.meet.test.util.TestUtils$$Lambda$344/0x0000000800e89390@7dd611c8 (tried for 10 second(s) with 500 milliseconds interval)
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17:03'
System info: host: 'David-Air.local', ip: '**IP**', os.name: 'Mac OS X', os.arch: 'aarch64', os.version: '12.0.1', java.version: '17.0.1'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 96.0.4664.55, chrome: {chromedriverVersion: 96.0.4664.45 (76e4c1bb2ab46..., userDataDir: /var/folders/6g/4hb15fqd6x7...}, goog:chromeOptions: {debuggerAddress: localhost:54971}, javascriptEnabled: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: MAC, platformName: MAC, proxy: Proxy(), setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:virtualAuthenticators: true}
Session ID: b1cc9df88ce281bcdce8f04a90ea983a
	at org.jitsi.meet.test.MuteTest.muteParticipant1BeforeParticipant2JoinsAndScreenshare(MuteTest.java:281)
	at org.jitsi.meet.test.MuteTest.muteAfterJoinCanShareAndUnmute(MuteTest.java:231)

[ERROR] tcpTest(org.jitsi.meet.test.TCPTest)  Time elapsed: 17.649 s  <<< FAILURE!
org.openqa.selenium.TimeoutException:
Expected condition failed: waiting for WebParticipant[web.participant2]@424106351#isIceConnected (tried for 15 second(s) with 500 milliseconds interval)
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17:03'
System info: host: 'David-Air.local', ip: '**IP**', os.name: 'Mac OS X', os.arch: 'aarch64', os.version: '12.0.1', java.version: '17.0.1'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 96.0.4664.55, chrome: {chromedriverVersion: 96.0.4664.45 (76e4c1bb2ab46..., userDataDir: /var/folders/6g/4hb15fqd6x7...}, goog:chromeOptions: {debuggerAddress: localhost:55275}, javascriptEnabled: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: MAC, platformName: MAC, proxy: Proxy(), setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:virtualAuthenticators: true}
Session ID: 085c268efbdf155d25a6688815a7784f
	at org.jitsi.meet.test.TCPTest.tcpTest(TCPTest.java:82)

[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]   MuteTest.muteAfterJoinCanShareAndUnmute:231->muteParticipant1BeforeParticipant2JoinsAndScreenshare:281 » Timeout
[ERROR]   TCPTest.tcpTest:82->WebTestBase.ensureTwoParticipants:104->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:157 » Timeout
[INFO]
[ERROR] Tests run: 411, Failures: 2, Errors: 0, Skipped: 402
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:30 min
[INFO] Finished at: 2021-11-28T10:25:24Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.20:test (default-test) on project jitsi-meet-torture: There are test failures.

dkgrieshammer avatar Nov 28 '21 10:11 dkgrieshammer

The one muteAfterJoinCanShareAndUnmute is known to fail sometimes and we will be working on fixing it in the following weeks or we will disable it, as for the TCPTest ... not sure we run it anymore ... maybe something with it is not working it needs debugging the client ...

damencho avatar Nov 28 '21 16:11 damencho

It is expected for TCPTest to fail against meet.jit.si as we don't have ice/tcp enabled (we have turn/tls instead).

On Sun, Nov 28, 2021, 10:50 AM Дамян Минков @.***> wrote:

The one muteAfterJoinCanShareAndUnmute is known to fail sometimes and we will be working on fixing it in the following weeks or we will disable it, as for the TCPTest ... not sure we run it anymore ... maybe something with it is not working it needs debugging the client ...

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jitsi/jitsi-meet-torture/issues/387#issuecomment-981116587, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAY5BOZITL4KNH6OXCJAZIDUOJMU5ANCNFSM454ZKZZQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

bgrozev avatar Nov 29 '21 18:11 bgrozev