at_server icon indicating copy to clipboard operation
at_server copied to clipboard

Sync in Progress on both @wavi and @mospherePro on Windows

Open cconstab opened this issue 2 years ago • 34 comments

Describe the bug Both apps get stuck on Sync in Progress

To Reproduce Steps to reproduce the behavior:

  1. First install on Windows latest versions
  2. Then try to run apps
  3. My Atsigns are on Canary BUT this only started happening last week.. Mobile Apps seem fine

Expected behavior Should be able to Sync !

Screenshots image

Windows (please complete the following information):

  • PC Windows 11 Were you using an @‎application when the bug was found?
  • [e.g. @‎wavi, @‎tmosphere]

Additional context Unclear to me why this started happening last week.. My Secondary has been up for days

@info
data:{"version":"3.0.17+c3.0.18a","uptimeAsWords":"16 days 0 hours 25 minutes 3 seconds","features":[{"name":"noop:","status":"Beta","description":"The No-Op verb simply does nothing for the requested number of milliseconds. The requested number of milliseconds may not be greater than 5000. Upon completion, the noop verb sends 'ok' as a response to the client.","syntax":"^noop:(?<delayMillis>\\d+)$"},{"name":"info:","status":"Beta","description":"The Info verb returns some information about the server including uptime and some info about available features. ","syntax":"^info(:brief)?$"}]}
@

Only thing I cxan think of is the Windows version got updated ? @sachins-geekyants ?

cconstab avatar Jun 06 '22 22:06 cconstab

And the logs on the client show the problem (running @mospherePro

The reason this happened last week is that @ kevin was deleted last week and then reinstated but has not been enabled yet. This is arguably a SDK issue and a bug in @ mospherePro.

Would really like to nail and learn from this issue as much as we can @gkc / @murali-shris / @VJag on for the Arch call tomorrow.. @sachins-geekyants see what you think!

lutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@kevin"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: FINER|2022-06-06 15:50:14.352466|AtLookup|AtLookup.findSecondary received atsign: kevin
flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@kevin"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: FINER|2022-06-06 15:50:14.361467|AtLookup|AtLookup.findSecondary got answer: 1619b378-f1bb-5f63-b7bc-fe628380f7e7.canary.atsign.zone:2364 and closing connection
flutter: FINER|2022-06-06 15:50:14.399499|AtLookup|AtLookup.findSecondary received atsign: kevin
flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@kevin"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: FINER|2022-06-06 15:50:14.445507|AtLookup|AtLookup.findSecondary received atsign: kevin
flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@kevin"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: FINER|2022-06-06 15:50:14.458508|AtLookup|AtLookup.findSecondary got answer: 1619b378-f1bb-5f63-b7bc-fe628380f7e7.canary.atsign.zone:2364 and closing connection
flutter: FINER|2022-06-06 15:50:14.459507|AtLookup|AtLookup.findSecondary got answer: null and closing connection

cconstab avatar Jun 06 '22 22:06 cconstab

Outstanding oddness is why does the Mobile app work still ?

cconstab avatar Jun 06 '22 23:06 cconstab

@sachins-geekyants is creating a new release for windows with latest at_client and at_client_mobile dependency

murali-shris avatar Jun 07 '22 08:06 murali-shris

@cconstab can you try running windows app from v1.0.7.42 tag.

sachins-geekyants avatar Jun 07 '22 12:06 sachins-geekyants

In tried but Windows keeps deleting the binary

image

cconstab avatar Jun 07 '22 13:06 cconstab

@sachins-geekyants @murali-shris

At this stage I can no longer log in on Windows...

The key is correct

image

cconstab@canary-01:~$ docker service logs -f --tail 10  79b6d83f-5026-5fda-8299-5a0704bd2416_secondary
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vw6ijlu27k4l@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 10:01:58.429933|AtSecondaryServer|Closing AccessLog HiveBox
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vw6ijlu27k4l@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vw6ijlu27k4l@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 10:01:58.430038|AtSecondaryServer|Closing NotificationKeyStore HiveBox
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vw6ijlu27k4l@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vw6ijlu27k4l@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 10:01:58.441840|AtSecondaryServer|Closing Main key store HiveBox
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vw6ijlu27k4l@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vw6ijlu27k4l@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 10:01:58.453267|AtSecondaryServer|Stopping scheduled tasks
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vw6ijlu27k4l@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vw6ijlu27k4l@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 10:01:58.453366|SecondaryServerBootStrapper|secondaryServerInstance.stop() completed, and isRunning is false - exiting with status 0
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vw6ijlu27k4l@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.dnl5sqasfskr@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 21:26:34.062957|AtSecondaryServer|Closing AccessLog HiveBox
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.dnl5sqasfskr@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.dnl5sqasfskr@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 21:26:34.063285|AtSecondaryServer|Closing NotificationKeyStore HiveBox
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.dnl5sqasfskr@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.dnl5sqasfskr@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 21:26:34.075020|AtSecondaryServer|Closing Main key store HiveBox
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.dnl5sqasfskr@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.dnl5sqasfskr@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 21:26:34.086468|AtSecondaryServer|Stopping scheduled tasks
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.dnl5sqasfskr@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.dnl5sqasfskr@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 21:26:34.086531|SecondaryServerBootStrapper|secondaryServerInstance.stop() completed, and isRunning is false - exiting with status 0
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.dnl5sqasfskr@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.mxubvokkmqa6@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 21:47:21.252314|AtSecondaryServer|Closing AccessLog HiveBox
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.mxubvokkmqa6@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.mxubvokkmqa6@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 21:47:21.252452|AtSecondaryServer|Closing NotificationKeyStore HiveBox
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.mxubvokkmqa6@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.mxubvokkmqa6@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 21:47:21.261778|AtSecondaryServer|Closing Main key store HiveBox
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.mxubvokkmqa6@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.mxubvokkmqa6@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 21:47:21.271074|AtSecondaryServer|Stopping scheduled tasks
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.mxubvokkmqa6@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.mxubvokkmqa6@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 21:47:21.271142|SecondaryServerBootStrapper|secondaryServerInstance.stop() completed, and isRunning is false - exiting with status 0
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.mxubvokkmqa6@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.sczdoblr02lu@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 21:38:03.175071|AtSecondaryServer|Closing AccessLog HiveBox
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.sczdoblr02lu@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.sczdoblr02lu@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 21:38:03.175130|AtSecondaryServer|Closing NotificationKeyStore HiveBox
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.sczdoblr02lu@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.sczdoblr02lu@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 21:38:03.184325|AtSecondaryServer|Closing Main key store HiveBox
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.sczdoblr02lu@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.sczdoblr02lu@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 21:38:03.195339|AtSecondaryServer|Stopping scheduled tasks
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.sczdoblr02lu@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.sczdoblr02lu@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-05-21 21:38:03.195386|SecondaryServerBootStrapper|secondaryServerInstance.stop() completed, and isRunning is false - exiting with status 0
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.sczdoblr02lu@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:40:52.761459|InboundListener|RCVD: [_977307f7-8a5d-463b-a4e4-613af49e8656] noop:0
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:52.761505|AtSecondaryServer|inside _executeVerbCallBack: noop:0
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:52.761571|DefaultVerbExecutor|verb handler found : NoOpVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:52.761705|NoOpVerbHandler|Verb : noop  Response: Response{_data: ok, _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:40:52.761743|InboundConnectionImpl|SENT: [_977307f7-8a5d-463b-a4e4-613af49e8656] data:ok
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:58.220422|AtSecondaryServer|In _listen - clientSocket.peerCertificate : null
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:40:58.220534|InboundConnectionImpl|SENT: [_6308d506-e67e-4037-9b5f-d1f2ca6cda99] @
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:40:58.307712|InboundListener|RCVD: [_6308d506-e67e-4037-9b5f-d1f2ca6cda99] scan
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:58.307775|AtSecondaryServer|inside _executeVerbCallBack: scan
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:58.307824|DefaultVerbExecutor|verb handler found : ScanVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:58.317916|ScanVerbHandler|response.data : [custom_about_me.wavi@colin, custom_butterflies.wavi@colin, custom_callsign.wavi@colin, custom_catch-up_and_book_a_meeting.wavi@colin, custom_github.wavi@colin, custom_jeeping.wavi@colin, custom_my_first_professional_computer.wavi@colin, custom_proverb.wavi@colin, custom_quote_of_the_day.wavi@colin, discord.wavi@colin, email.wavi@colin, facebook.wavi@colin, field_order_of_self.wavi@colin, firstname.wavi@colin, followers_of_self.at_follows.wavi@colin, following_by_self.at_follows.wavi@colin, image.wavi@colin, instagram.wavi@colin, lastname.wavi@colin, linkedin.wavi@colin, location.wavi@colin, locationnickname.wavi@colin, phone.wavi@colin, privateaccount.wavi@colin, profilepic.buzz@colin, pronoun.wavi@colin, publickey, publickey@colin, signing_publickey@colin, theme.wavi@colin, theme_color.wavi@colin, twitter.wavi@colin, youtube.wavi@colin]
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:58.317999|ScanVerbHandler|keysArray : [custom_about_me.wavi@colin, custom_butterflies.wavi@colin, custom_callsign.wavi@colin, custom_catch-up_and_book_a_meeting.wavi@colin, custom_github.wavi@colin, custom_jeeping.wavi@colin, custom_my_first_professional_computer.wavi@colin, custom_proverb.wavi@colin, custom_quote_of_the_day.wavi@colin, discord.wavi@colin, email.wavi@colin, facebook.wavi@colin, field_order_of_self.wavi@colin, firstname.wavi@colin, followers_of_self.at_follows.wavi@colin, following_by_self.at_follows.wavi@colin, image.wavi@colin, instagram.wavi@colin, lastname.wavi@colin, linkedin.wavi@colin, location.wavi@colin, locationnickname.wavi@colin, phone.wavi@colin, privateaccount.wavi@colin, profilepic.buzz@colin, pronoun.wavi@colin, publickey, publickey@colin, signing_publickey@colin, theme.wavi@colin, theme_color.wavi@colin, twitter.wavi@colin, youtube.wavi@colin], 33
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:58.318053|ScanVerbHandler|Verb : scan  Response: Response{_data: ["custom_about_me.wavi@colin","custom_butterflies.wavi@colin","custom_callsign.wavi@colin","custom_catch-up_and_book_a_meeting.wavi@colin","custom_github.wavi@colin","custom_jeeping.wavi@colin","custom_my_first_professional_computer.wavi@colin","custom_proverb.wavi@colin","custom_quote_of_the_day.wavi@colin","discord.wavi@colin","email.wavi@colin","facebook.wavi@colin","field_order_of_self.wavi@colin","firstname.wavi@colin","followers_of_self.at_follows.wavi@colin","following_by_self.at_follows.wavi@colin","image.wavi@colin","instagram.wavi@colin","lastname.wavi@colin","linkedin.wavi@colin","location.wavi@colin","locationnickname.wavi@colin","phone.wavi@colin","privateaccount.wavi@colin","profilepic.buzz@colin","pronoun.wavi@colin","publickey","publickey@colin","signing_publickey@colin","theme.wavi@colin","theme_color.wavi@colin","twitter.wavi@colin","youtube.wavi@colin"], _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:40:58.318080|InboundConnectionImpl|SENT: [_6308d506-e67e-4037-9b5f-d1f2ca6cda99] data:["custom_about_me.wavi@colin","custom_butterflies.wavi@colin","custom_callsign.wavi@colin","custom_catch-up_and_book_a_meeting.wavi@colin","custom_github.wavi@colin","custom_jeeping.wavi@colin","custom_my_first_professional_computer.wavi@colin","custom_proverb.wavi@colin","custom_quote_of_the_day.wavi@colin","discord.wavi@colin","email.wavi@colin","facebook.wavi@colin","field_order_of_self.wavi@colin","firstname.wavi@colin","followers_of_self.at_follows.wavi@colin","following_by_self.at_follows.wavi@colin","image.wavi@colin","instagram.wavi@colin","lastname.wavi@colin","linkedin.wavi@colin","location.wavi@colin","locationnickname.wavi@colin","phone.wavi@colin","privateaccount.wavi@colin","profilepic.buzz@colin","pronoun.wavi@colin","publickey","publickey@colin","signing_publickey@colin","theme.wavi@colin","theme_color.wavi@colin","twitter.wavi@colin","youtube.wavi@colin"]
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:40:58.494342|InboundListener|RCVD: [_6308d506-e67e-4037-9b5f-d1f2ca6cda99] lookup:publickey@colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:58.494385|AtSecondaryServer|inside _executeVerbCallBack: lookup:publickey@colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:58.494442|DefaultVerbExecutor|verb handler found : LookupVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:58.494496|LookupVerbHandler|fromAtSign : null
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |  atSign : @colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |  key : publickey@colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:58.494503|LookupVerbHandler|lookup_key in lookupVerbHandler : public:publickey@colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:58.497076|Secondary_Util|result : MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAiE/FJPmcsByxwTSUWSTQcz/08YkI956dL10+3IT4jiEqlkBTs+/6bp4NiAK/MWBW3smJLbK+g02mXetG/URqH0kb6HgcD4Idb0wfYqjT5IGKCbPA/YsO0gINaaBLaMkyzHyg7C/jdQNiJR+1zrJeHt/EtU1R7hN4tdpW1ZagiyrmMUkNH5kqXNzM9K/C28zRctBhlpNpwbg41vJ4eMRipYepl0s5uUfAGOoLhRRbzbN+40iALcnL648U8XzwU3SWoEZHvA7b+Xwu1l13kopXyWyudOciZgQoSa5UYxbT8HNhy5ptTQE13UW1JyNXoDNhb/no0y2SliOtWmkZl4U55wIDAQAB
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:58.497392|LookupVerbHandler|Verb : lookup  Response: Response{_data: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAiE/FJPmcsByxwTSUWSTQcz/08YkI956dL10+3IT4jiEqlkBTs+/6bp4NiAK/MWBW3smJLbK+g02mXetG/URqH0kb6HgcD4Idb0wfYqjT5IGKCbPA/YsO0gINaaBLaMkyzHyg7C/jdQNiJR+1zrJeHt/EtU1R7hN4tdpW1ZagiyrmMUkNH5kqXNzM9K/C28zRctBhlpNpwbg41vJ4eMRipYepl0s5uUfAGOoLhRRbzbN+40iALcnL648U8XzwU3SWoEZHvA7b+Xwu1l13kopXyWyudOciZgQoSa5UYxbT8HNhy5ptTQE13UW1JyNXoDNhb/no0y2SliOtWmkZl4U55wIDAQAB, _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:40:58.497434|InboundConnectionImpl|SENT: [_6308d506-e67e-4037-9b5f-d1f2ca6cda99] data:MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAiE/FJPmcsByxwTSUWSTQcz/08YkI956dL10+3IT4jiEqlkBTs+/6bp4NiAK/MWBW3smJLbK+g02mXetG/URqH0kb6HgcD4Idb0wfYqjT5IGKCbPA/YsO0gINaaBLaMkyzHyg7C/jdQNiJR+1zrJeHt/EtU1R7hN4tdpW1ZagiyrmMUkNH5kqXNzM9K/C28zRctBhlpNpwbg41vJ4eMRipYepl0s5uUfAGOoLhRRbzbN+40iALcnL648U8XzwU3SWoEZHvA7b+Xwu1l13kopXyWyudOciZgQoSa5UYxbT8HNhy5ptTQE13UW1JyNXoDNhb/no0y2SliOtWmkZl4U55wIDAQAB
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:58.584915|InboundConnectionImpl|InternetAddress('10.10.0.15', IPv4):11161 Disconnected
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | WARNING|2022-06-08 19:40:59.504387|AtSecondaryServer|ServerSocket.listen called onError with 'HandshakeException: Connection terminated during handshake'
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:59.610445|AtSecondaryServer|In _listen - clientSocket.peerCertificate : null
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:40:59.610673|InboundConnectionImpl|SENT: [_fe7fc1fc-a848-4eb0-a3c3-b643fa556b35] @
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:40:59.704042|InboundListener|RCVD: [_fe7fc1fc-a848-4eb0-a3c3-b643fa556b35] scan
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:59.704110|AtSecondaryServer|inside _executeVerbCallBack: scan
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:59.704175|DefaultVerbExecutor|verb handler found : ScanVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:59.711969|ScanVerbHandler|response.data : [custom_about_me.wavi@colin, custom_butterflies.wavi@colin, custom_callsign.wavi@colin, custom_catch-up_and_book_a_meeting.wavi@colin, custom_github.wavi@colin, custom_jeeping.wavi@colin, custom_my_first_professional_computer.wavi@colin, custom_proverb.wavi@colin, custom_quote_of_the_day.wavi@colin, discord.wavi@colin, email.wavi@colin, facebook.wavi@colin, field_order_of_self.wavi@colin, firstname.wavi@colin, followers_of_self.at_follows.wavi@colin, following_by_self.at_follows.wavi@colin, image.wavi@colin, instagram.wavi@colin, lastname.wavi@colin, linkedin.wavi@colin, location.wavi@colin, locationnickname.wavi@colin, phone.wavi@colin, privateaccount.wavi@colin, profilepic.buzz@colin, pronoun.wavi@colin, publickey, publickey@colin, signing_publickey@colin, theme.wavi@colin, theme_color.wavi@colin, twitter.wavi@colin, youtube.wavi@colin]
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:59.712048|ScanVerbHandler|keysArray : [custom_about_me.wavi@colin, custom_butterflies.wavi@colin, custom_callsign.wavi@colin, custom_catch-up_and_book_a_meeting.wavi@colin, custom_github.wavi@colin, custom_jeeping.wavi@colin, custom_my_first_professional_computer.wavi@colin, custom_proverb.wavi@colin, custom_quote_of_the_day.wavi@colin, discord.wavi@colin, email.wavi@colin, facebook.wavi@colin, field_order_of_self.wavi@colin, firstname.wavi@colin, followers_of_self.at_follows.wavi@colin, following_by_self.at_follows.wavi@colin, image.wavi@colin, instagram.wavi@colin, lastname.wavi@colin, linkedin.wavi@colin, location.wavi@colin, locationnickname.wavi@colin, phone.wavi@colin, privateaccount.wavi@colin, profilepic.buzz@colin, pronoun.wavi@colin, publickey, publickey@colin, signing_publickey@colin, theme.wavi@colin, theme_color.wavi@colin, twitter.wavi@colin, youtube.wavi@colin], 33
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:59.712124|ScanVerbHandler|Verb : scan  Response: Response{_data: ["custom_about_me.wavi@colin","custom_butterflies.wavi@colin","custom_callsign.wavi@colin","custom_catch-up_and_book_a_meeting.wavi@colin","custom_github.wavi@colin","custom_jeeping.wavi@colin","custom_my_first_professional_computer.wavi@colin","custom_proverb.wavi@colin","custom_quote_of_the_day.wavi@colin","discord.wavi@colin","email.wavi@colin","facebook.wavi@colin","field_order_of_self.wavi@colin","firstname.wavi@colin","followers_of_self.at_follows.wavi@colin","following_by_self.at_follows.wavi@colin","image.wavi@colin","instagram.wavi@colin","lastname.wavi@colin","linkedin.wavi@colin","location.wavi@colin","locationnickname.wavi@colin","phone.wavi@colin","privateaccount.wavi@colin","profilepic.buzz@colin","pronoun.wavi@colin","publickey","publickey@colin","signing_publickey@colin","theme.wavi@colin","theme_color.wavi@colin","twitter.wavi@colin","youtube.wavi@colin"], _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:40:59.712158|InboundConnectionImpl|SENT: [_fe7fc1fc-a848-4eb0-a3c3-b643fa556b35] data:["custom_about_me.wavi@colin","custom_butterflies.wavi@colin","custom_callsign.wavi@colin","custom_catch-up_and_book_a_meeting.wavi@colin","custom_github.wavi@colin","custom_jeeping.wavi@colin","custom_my_first_professional_computer.wavi@colin","custom_proverb.wavi@colin","custom_quote_of_the_day.wavi@colin","discord.wavi@colin","email.wavi@colin","facebook.wavi@colin","field_order_of_self.wavi@colin","firstname.wavi@colin","followers_of_self.at_follows.wavi@colin","following_by_self.at_follows.wavi@colin","image.wavi@colin","instagram.wavi@colin","lastname.wavi@colin","linkedin.wavi@colin","location.wavi@colin","locationnickname.wavi@colin","phone.wavi@colin","privateaccount.wavi@colin","profilepic.buzz@colin","pronoun.wavi@colin","publickey","publickey@colin","signing_publickey@colin","theme.wavi@colin","theme_color.wavi@colin","twitter.wavi@colin","youtube.wavi@colin"]
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:40:59.938370|InboundListener|RCVD: [_fe7fc1fc-a848-4eb0-a3c3-b643fa556b35] lookup:publickey@colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:59.938472|AtSecondaryServer|inside _executeVerbCallBack: lookup:publickey@colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:59.938643|DefaultVerbExecutor|verb handler found : LookupVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:59.938822|LookupVerbHandler|fromAtSign : null
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |  atSign : @colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |  key : publickey@colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:59.938860|LookupVerbHandler|lookup_key in lookupVerbHandler : public:publickey@colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:59.939277|Secondary_Util|result : MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAiE/FJPmcsByxwTSUWSTQcz/08YkI956dL10+3IT4jiEqlkBTs+/6bp4NiAK/MWBW3smJLbK+g02mXetG/URqH0kb6HgcD4Idb0wfYqjT5IGKCbPA/YsO0gINaaBLaMkyzHyg7C/jdQNiJR+1zrJeHt/EtU1R7hN4tdpW1ZagiyrmMUkNH5kqXNzM9K/C28zRctBhlpNpwbg41vJ4eMRipYepl0s5uUfAGOoLhRRbzbN+40iALcnL648U8XzwU3SWoEZHvA7b+Xwu1l13kopXyWyudOciZgQoSa5UYxbT8HNhy5ptTQE13UW1JyNXoDNhb/no0y2SliOtWmkZl4U55wIDAQAB
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:40:59.939774|LookupVerbHandler|Verb : lookup  Response: Response{_data: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAiE/FJPmcsByxwTSUWSTQcz/08YkI956dL10+3IT4jiEqlkBTs+/6bp4NiAK/MWBW3smJLbK+g02mXetG/URqH0kb6HgcD4Idb0wfYqjT5IGKCbPA/YsO0gINaaBLaMkyzHyg7C/jdQNiJR+1zrJeHt/EtU1R7hN4tdpW1ZagiyrmMUkNH5kqXNzM9K/C28zRctBhlpNpwbg41vJ4eMRipYepl0s5uUfAGOoLhRRbzbN+40iALcnL648U8XzwU3SWoEZHvA7b+Xwu1l13kopXyWyudOciZgQoSa5UYxbT8HNhy5ptTQE13UW1JyNXoDNhb/no0y2SliOtWmkZl4U55wIDAQAB, _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:40:59.939813|InboundConnectionImpl|SENT: [_fe7fc1fc-a848-4eb0-a3c3-b643fa556b35] data:MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAiE/FJPmcsByxwTSUWSTQcz/08YkI956dL10+3IT4jiEqlkBTs+/6bp4NiAK/MWBW3smJLbK+g02mXetG/URqH0kb6HgcD4Idb0wfYqjT5IGKCbPA/YsO0gINaaBLaMkyzHyg7C/jdQNiJR+1zrJeHt/EtU1R7hN4tdpW1ZagiyrmMUkNH5kqXNzM9K/C28zRctBhlpNpwbg41vJ4eMRipYepl0s5uUfAGOoLhRRbzbN+40iALcnL648U8XzwU3SWoEZHvA7b+Xwu1l13kopXyWyudOciZgQoSa5UYxbT8HNhy5ptTQE13UW1JyNXoDNhb/no0y2SliOtWmkZl4U55wIDAQAB
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:00.043688|InboundConnectionImpl|InternetAddress('10.10.0.15', IPv4):16874 Disconnected
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:41:07.375174|InboundConnectionImpl|SENT: [_f90d655f-375b-4617-8653-a8eb360537f9] notification: {"id":"-1","from":"@colin","to":"@colin","key":"statsNotification.@colin","value":"353754","operation":"update","epochMillis":1654717267375}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | WARNING|2022-06-08 19:41:08.341556|AtSecondaryServer|ServerSocket.listen called onError with 'HandshakeException: Connection terminated during handshake'
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:08.442435|AtSecondaryServer|In _listen - clientSocket.peerCertificate : null
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:41:08.442555|InboundConnectionImpl|SENT: [_f02fb15c-bac0-43f0-9d91-26a138219107] @
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:41:08.530109|InboundListener|RCVD: [_f02fb15c-bac0-43f0-9d91-26a138219107] scan
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:08.530156|AtSecondaryServer|inside _executeVerbCallBack: scan
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:08.530201|DefaultVerbExecutor|verb handler found : ScanVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:08.538758|ScanVerbHandler|response.data : [custom_about_me.wavi@colin, custom_butterflies.wavi@colin, custom_callsign.wavi@colin, custom_catch-up_and_book_a_meeting.wavi@colin, custom_github.wavi@colin, custom_jeeping.wavi@colin, custom_my_first_professional_computer.wavi@colin, custom_proverb.wavi@colin, custom_quote_of_the_day.wavi@colin, discord.wavi@colin, email.wavi@colin, facebook.wavi@colin, field_order_of_self.wavi@colin, firstname.wavi@colin, followers_of_self.at_follows.wavi@colin, following_by_self.at_follows.wavi@colin, image.wavi@colin, instagram.wavi@colin, lastname.wavi@colin, linkedin.wavi@colin, location.wavi@colin, locationnickname.wavi@colin, phone.wavi@colin, privateaccount.wavi@colin, profilepic.buzz@colin, pronoun.wavi@colin, publickey, publickey@colin, signing_publickey@colin, theme.wavi@colin, theme_color.wavi@colin, twitter.wavi@colin, youtube.wavi@colin]
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:08.538837|ScanVerbHandler|keysArray : [custom_about_me.wavi@colin, custom_butterflies.wavi@colin, custom_callsign.wavi@colin, custom_catch-up_and_book_a_meeting.wavi@colin, custom_github.wavi@colin, custom_jeeping.wavi@colin, custom_my_first_professional_computer.wavi@colin, custom_proverb.wavi@colin, custom_quote_of_the_day.wavi@colin, discord.wavi@colin, email.wavi@colin, facebook.wavi@colin, field_order_of_self.wavi@colin, firstname.wavi@colin, followers_of_self.at_follows.wavi@colin, following_by_self.at_follows.wavi@colin, image.wavi@colin, instagram.wavi@colin, lastname.wavi@colin, linkedin.wavi@colin, location.wavi@colin, locationnickname.wavi@colin, phone.wavi@colin, privateaccount.wavi@colin, profilepic.buzz@colin, pronoun.wavi@colin, publickey, publickey@colin, signing_publickey@colin, theme.wavi@colin, theme_color.wavi@colin, twitter.wavi@colin, youtube.wavi@colin], 33
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:08.538900|ScanVerbHandler|Verb : scan  Response: Response{_data: ["custom_about_me.wavi@colin","custom_butterflies.wavi@colin","custom_callsign.wavi@colin","custom_catch-up_and_book_a_meeting.wavi@colin","custom_github.wavi@colin","custom_jeeping.wavi@colin","custom_my_first_professional_computer.wavi@colin","custom_proverb.wavi@colin","custom_quote_of_the_day.wavi@colin","discord.wavi@colin","email.wavi@colin","facebook.wavi@colin","field_order_of_self.wavi@colin","firstname.wavi@colin","followers_of_self.at_follows.wavi@colin","following_by_self.at_follows.wavi@colin","image.wavi@colin","instagram.wavi@colin","lastname.wavi@colin","linkedin.wavi@colin","location.wavi@colin","locationnickname.wavi@colin","phone.wavi@colin","privateaccount.wavi@colin","profilepic.buzz@colin","pronoun.wavi@colin","publickey","publickey@colin","signing_publickey@colin","theme.wavi@colin","theme_color.wavi@colin","twitter.wavi@colin","youtube.wavi@colin"], _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:41:08.538924|InboundConnectionImpl|SENT: [_f02fb15c-bac0-43f0-9d91-26a138219107] data:["custom_about_me.wavi@colin","custom_butterflies.wavi@colin","custom_callsign.wavi@colin","custom_catch-up_and_book_a_meeting.wavi@colin","custom_github.wavi@colin","custom_jeeping.wavi@colin","custom_my_first_professional_computer.wavi@colin","custom_proverb.wavi@colin","custom_quote_of_the_day.wavi@colin","discord.wavi@colin","email.wavi@colin","facebook.wavi@colin","field_order_of_self.wavi@colin","firstname.wavi@colin","followers_of_self.at_follows.wavi@colin","following_by_self.at_follows.wavi@colin","image.wavi@colin","instagram.wavi@colin","lastname.wavi@colin","linkedin.wavi@colin","location.wavi@colin","locationnickname.wavi@colin","phone.wavi@colin","privateaccount.wavi@colin","profilepic.buzz@colin","pronoun.wavi@colin","publickey","publickey@colin","signing_publickey@colin","theme.wavi@colin","theme_color.wavi@colin","twitter.wavi@colin","youtube.wavi@colin"]
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:41:08.761394|InboundListener|RCVD: [_f02fb15c-bac0-43f0-9d91-26a138219107] lookup:publickey@colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:08.761456|AtSecondaryServer|inside _executeVerbCallBack: lookup:publickey@colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:08.761513|DefaultVerbExecutor|verb handler found : LookupVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:08.761573|LookupVerbHandler|fromAtSign : null
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |  atSign : @colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |  key : publickey@colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:08.761581|LookupVerbHandler|lookup_key in lookupVerbHandler : public:publickey@colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:08.762581|Secondary_Util|result : MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAiE/FJPmcsByxwTSUWSTQcz/08YkI956dL10+3IT4jiEqlkBTs+/6bp4NiAK/MWBW3smJLbK+g02mXetG/URqH0kb6HgcD4Idb0wfYqjT5IGKCbPA/YsO0gINaaBLaMkyzHyg7C/jdQNiJR+1zrJeHt/EtU1R7hN4tdpW1ZagiyrmMUkNH5kqXNzM9K/C28zRctBhlpNpwbg41vJ4eMRipYepl0s5uUfAGOoLhRRbzbN+40iALcnL648U8XzwU3SWoEZHvA7b+Xwu1l13kopXyWyudOciZgQoSa5UYxbT8HNhy5ptTQE13UW1JyNXoDNhb/no0y2SliOtWmkZl4U55wIDAQAB
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:08.763527|LookupVerbHandler|Verb : lookup  Response: Response{_data: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAiE/FJPmcsByxwTSUWSTQcz/08YkI956dL10+3IT4jiEqlkBTs+/6bp4NiAK/MWBW3smJLbK+g02mXetG/URqH0kb6HgcD4Idb0wfYqjT5IGKCbPA/YsO0gINaaBLaMkyzHyg7C/jdQNiJR+1zrJeHt/EtU1R7hN4tdpW1ZagiyrmMUkNH5kqXNzM9K/C28zRctBhlpNpwbg41vJ4eMRipYepl0s5uUfAGOoLhRRbzbN+40iALcnL648U8XzwU3SWoEZHvA7b+Xwu1l13kopXyWyudOciZgQoSa5UYxbT8HNhy5ptTQE13UW1JyNXoDNhb/no0y2SliOtWmkZl4U55wIDAQAB, _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:41:08.763572|InboundConnectionImpl|SENT: [_f02fb15c-bac0-43f0-9d91-26a138219107] data:MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAiE/FJPmcsByxwTSUWSTQcz/08YkI956dL10+3IT4jiEqlkBTs+/6bp4NiAK/MWBW3smJLbK+g02mXetG/URqH0kb6HgcD4Idb0wfYqjT5IGKCbPA/YsO0gINaaBLaMkyzHyg7C/jdQNiJR+1zrJeHt/EtU1R7hN4tdpW1ZagiyrmMUkNH5kqXNzM9K/C28zRctBhlpNpwbg41vJ4eMRipYepl0s5uUfAGOoLhRRbzbN+40iALcnL648U8XzwU3SWoEZHvA7b+Xwu1l13kopXyWyudOciZgQoSa5UYxbT8HNhy5ptTQE13UW1JyNXoDNhb/no0y2SliOtWmkZl4U55wIDAQAB
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:08.861769|InboundConnectionImpl|InternetAddress('10.10.0.15', IPv4):3950 Disconnected
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:09.926763|AtSecondaryServer|In _listen - clientSocket.peerCertificate : null
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:41:09.926874|InboundConnectionImpl|SENT: [_9d8ff8a8-7769-461c-8940-b9dcd955c22d] @
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:41:10.013390|InboundListener|RCVD: [_9d8ff8a8-7769-461c-8940-b9dcd955c22d] scan
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:10.013445|AtSecondaryServer|inside _executeVerbCallBack: scan
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:10.013502|DefaultVerbExecutor|verb handler found : ScanVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:10.021495|ScanVerbHandler|response.data : [custom_about_me.wavi@colin, custom_butterflies.wavi@colin, custom_callsign.wavi@colin, custom_catch-up_and_book_a_meeting.wavi@colin, custom_github.wavi@colin, custom_jeeping.wavi@colin, custom_my_first_professional_computer.wavi@colin, custom_proverb.wavi@colin, custom_quote_of_the_day.wavi@colin, discord.wavi@colin, email.wavi@colin, facebook.wavi@colin, field_order_of_self.wavi@colin, firstname.wavi@colin, followers_of_self.at_follows.wavi@colin, following_by_self.at_follows.wavi@colin, image.wavi@colin, instagram.wavi@colin, lastname.wavi@colin, linkedin.wavi@colin, location.wavi@colin, locationnickname.wavi@colin, phone.wavi@colin, privateaccount.wavi@colin, profilepic.buzz@colin, pronoun.wavi@colin, publickey, publickey@colin, signing_publickey@colin, theme.wavi@colin, theme_color.wavi@colin, twitter.wavi@colin, youtube.wavi@colin]
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:10.021580|ScanVerbHandler|keysArray : [custom_about_me.wavi@colin, custom_butterflies.wavi@colin, custom_callsign.wavi@colin, custom_catch-up_and_book_a_meeting.wavi@colin, custom_github.wavi@colin, custom_jeeping.wavi@colin, custom_my_first_professional_computer.wavi@colin, custom_proverb.wavi@colin, custom_quote_of_the_day.wavi@colin, discord.wavi@colin, email.wavi@colin, facebook.wavi@colin, field_order_of_self.wavi@colin, firstname.wavi@colin, followers_of_self.at_follows.wavi@colin, following_by_self.at_follows.wavi@colin, image.wavi@colin, instagram.wavi@colin, lastname.wavi@colin, linkedin.wavi@colin, location.wavi@colin, locationnickname.wavi@colin, phone.wavi@colin, privateaccount.wavi@colin, profilepic.buzz@colin, pronoun.wavi@colin, publickey, publickey@colin, signing_publickey@colin, theme.wavi@colin, theme_color.wavi@colin, twitter.wavi@colin, youtube.wavi@colin], 33
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:10.021658|ScanVerbHandler|Verb : scan  Response: Response{_data: ["custom_about_me.wavi@colin","custom_butterflies.wavi@colin","custom_callsign.wavi@colin","custom_catch-up_and_book_a_meeting.wavi@colin","custom_github.wavi@colin","custom_jeeping.wavi@colin","custom_my_first_professional_computer.wavi@colin","custom_proverb.wavi@colin","custom_quote_of_the_day.wavi@colin","discord.wavi@colin","email.wavi@colin","facebook.wavi@colin","field_order_of_self.wavi@colin","firstname.wavi@colin","followers_of_self.at_follows.wavi@colin","following_by_self.at_follows.wavi@colin","image.wavi@colin","instagram.wavi@colin","lastname.wavi@colin","linkedin.wavi@colin","location.wavi@colin","locationnickname.wavi@colin","phone.wavi@colin","privateaccount.wavi@colin","profilepic.buzz@colin","pronoun.wavi@colin","publickey","publickey@colin","signing_publickey@colin","theme.wavi@colin","theme_color.wavi@colin","twitter.wavi@colin","youtube.wavi@colin"], _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:41:10.021683|InboundConnectionImpl|SENT: [_9d8ff8a8-7769-461c-8940-b9dcd955c22d] data:["custom_about_me.wavi@colin","custom_butterflies.wavi@colin","custom_callsign.wavi@colin","custom_catch-up_and_book_a_meeting.wavi@colin","custom_github.wavi@colin","custom_jeeping.wavi@colin","custom_my_first_professional_computer.wavi@colin","custom_proverb.wavi@colin","custom_quote_of_the_day.wavi@colin","discord.wavi@colin","email.wavi@colin","facebook.wavi@colin","field_order_of_self.wavi@colin","firstname.wavi@colin","followers_of_self.at_follows.wavi@colin","following_by_self.at_follows.wavi@colin","image.wavi@colin","instagram.wavi@colin","lastname.wavi@colin","linkedin.wavi@colin","location.wavi@colin","locationnickname.wavi@colin","phone.wavi@colin","privateaccount.wavi@colin","profilepic.buzz@colin","pronoun.wavi@colin","publickey","publickey@colin","signing_publickey@colin","theme.wavi@colin","theme_color.wavi@colin","twitter.wavi@colin","youtube.wavi@colin"]
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:41:10.247440|InboundListener|RCVD: [_9d8ff8a8-7769-461c-8940-b9dcd955c22d] lookup:publickey@colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:10.247536|AtSecondaryServer|inside _executeVerbCallBack: lookup:publickey@colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:10.247583|DefaultVerbExecutor|verb handler found : LookupVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:10.247670|LookupVerbHandler|fromAtSign : null
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |  atSign : @colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |  key : publickey@colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:10.247680|LookupVerbHandler|lookup_key in lookupVerbHandler : public:publickey@colin
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:10.248286|Secondary_Util|result : MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAiE/FJPmcsByxwTSUWSTQcz/08YkI956dL10+3IT4jiEqlkBTs+/6bp4NiAK/MWBW3smJLbK+g02mXetG/URqH0kb6HgcD4Idb0wfYqjT5IGKCbPA/YsO0gINaaBLaMkyzHyg7C/jdQNiJR+1zrJeHt/EtU1R7hN4tdpW1ZagiyrmMUkNH5kqXNzM9K/C28zRctBhlpNpwbg41vJ4eMRipYepl0s5uUfAGOoLhRRbzbN+40iALcnL648U8XzwU3SWoEZHvA7b+Xwu1l13kopXyWyudOciZgQoSa5UYxbT8HNhy5ptTQE13UW1JyNXoDNhb/no0y2SliOtWmkZl4U55wIDAQAB
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:10.248568|LookupVerbHandler|Verb : lookup  Response: Response{_data: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAiE/FJPmcsByxwTSUWSTQcz/08YkI956dL10+3IT4jiEqlkBTs+/6bp4NiAK/MWBW3smJLbK+g02mXetG/URqH0kb6HgcD4Idb0wfYqjT5IGKCbPA/YsO0gINaaBLaMkyzHyg7C/jdQNiJR+1zrJeHt/EtU1R7hN4tdpW1ZagiyrmMUkNH5kqXNzM9K/C28zRctBhlpNpwbg41vJ4eMRipYepl0s5uUfAGOoLhRRbzbN+40iALcnL648U8XzwU3SWoEZHvA7b+Xwu1l13kopXyWyudOciZgQoSa5UYxbT8HNhy5ptTQE13UW1JyNXoDNhb/no0y2SliOtWmkZl4U55wIDAQAB, _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:41:10.248628|InboundConnectionImpl|SENT: [_9d8ff8a8-7769-461c-8940-b9dcd955c22d] data:MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAiE/FJPmcsByxwTSUWSTQcz/08YkI956dL10+3IT4jiEqlkBTs+/6bp4NiAK/MWBW3smJLbK+g02mXetG/URqH0kb6HgcD4Idb0wfYqjT5IGKCbPA/YsO0gINaaBLaMkyzHyg7C/jdQNiJR+1zrJeHt/EtU1R7hN4tdpW1ZagiyrmMUkNH5kqXNzM9K/C28zRctBhlpNpwbg41vJ4eMRipYepl0s5uUfAGOoLhRRbzbN+40iALcnL648U8XzwU3SWoEZHvA7b+Xwu1l13kopXyWyudOciZgQoSa5UYxbT8HNhy5ptTQE13UW1JyNXoDNhb/no0y2SliOtWmkZl4U55wIDAQAB
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:10.323379|InboundConnectionImpl|InternetAddress('10.10.0.15', IPv4):13392 Disconnected
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:41:12.362530|InboundListener|RCVD: [_977307f7-8a5d-463b-a4e4-613af49e8656] noop:0
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:12.362595|AtSecondaryServer|inside _executeVerbCallBack: noop:0
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:12.362735|DefaultVerbExecutor|verb handler found : NoOpVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:12.362923|NoOpVerbHandler|Verb : noop  Response: Response{_data: ok, _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:41:12.362977|InboundConnectionImpl|SENT: [_977307f7-8a5d-463b-a4e4-613af49e8656] data:ok
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:41:21.612786|InboundListener|RCVD: [_f90d655f-375b-4617-8653-a8eb360537f9] noop:0
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:21.612834|AtSecondaryServer|inside _executeVerbCallBack: noop:0
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:21.612894|DefaultVerbExecutor|verb handler found : NoOpVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-08 19:41:21.612985|NoOpVerbHandler|Verb : noop  Response: Response{_data: ok, _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-08 19:41:21.613007|InboundConnectionImpl|SENT: [_f90d655f-375b-4617-8653-a8eb360537f9] data:ok
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
^C
cconstab@canary-01:~$



cconstab avatar Jun 08 '22 19:06 cconstab

If I run from Trunk I can login (it uses the keys that were aledged to be faulty) and then get stuck at Sync in progress with lots of nasty errors in the logs..

e.g.

ried calling: forEach(Closure: (dynamic) => Null)
flutter: FINER|2022-06-08 12:49:04.789382|AtLookup|AtLookup.findSecondary received atsign: jenmonroe
flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@jenmonroe"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: FINER|2022-06-08 12:49:04.802383|AtLookup|AtLookup.findSecondary got answer: 97ef7af4-e033-59b5-bf63-bddd3d738eaa.swarm0001.atsign.zone:11030 and closing connection
flutter: FINER|2022-06-08 12:49:04.845382|AtLookup|AtLookup.findSecondary got answer: 446b7a28-8028-558d-be62-b079bf387bc0.canary.atsign.zone:1101 and closing connection
flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@jenmonroe"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: FINER|2022-06-08 12:49:04.906384|AtLookup|AtLookup.findSecondary got answer: 446b7a28-8028-558d-be62-b079bf387bc0.canary.atsign.zone:1101 and closing connection
flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@jenmonroe"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.

cconstab avatar Jun 08 '22 19:06 cconstab

I have bumped this to P1 as I am concerned about this hitting other folks using desktop

@gkc FEEL FREE TO BUMP BACK DOWN IF YOU DO NOT AGREE

cconstab avatar Jun 08 '22 19:06 cconstab

@cconstab I agree, those are ugly exceptions in the logs @sachins-geekyants have you diagnosed the problem? what help do you need?

gkc avatar Jun 08 '22 20:06 gkc

Initially, we assumed the sync issue on windows might be happening because of old dependency of at_client and at_client_mobile, I tried running @colin atsign on windows using atmopshere trunk and it failed for me too even after keeping the app running for around 20 mins.

Sync kind of freezed with Exception

flutter: SEVERE|2022-06-09 13:54:06.098748|AtLookup|Exception in sending to server, Exception: No response after 10000 millis from remote secondary

flutter: SEVERE|2022-06-09 13:54:06.099694|AtLookup|Error in remote verb execution Exception: No response after 10000 millis from remote secondary

flutter: SEVERE|2022-06-09 13:54:06.100691|SyncService|Exception in sync 84a62690-47bf-4c02-b796-5db3051fa883. Reason AT0023: Exception: No response after 10000 millis from remote secondary

flutter: SEVERE|2022-06-09 13:54:14.646045|AtLookup|Exception in sending to server, ErrorCode: AT0014 - Exception: Unexpected response found 

flutter: SEVERE|2022-06-09 13:54:14.646045|AtLookup|Error in remote verb execution ErrorCode: AT0014 - Exception: Unexpected response found

flutter: SEVERE|2022-06-09 13:54:14.647099|SyncUtil|Exception occurred in processing stats verb AT0014 - ErrorCode: AT0014 - Exception: Unexpected response found

flutter: INFO|2022-06-09 13:54:14.658010|SyncService|Returning the serverCommitId 357293

flutter: SEVERE|2022-06-09 13:54:15.092906|SyncUtil|Exception occurred in processing stats verb AT0003 - Invalid syntax : Invalid syntax. stats:1,2,3 

I don't think app has control over sync. Hada call with @kalluriramkumar also regarding this issue.

@colin_logs.txt

sachins-geekyants avatar Jun 09 '22 09:06 sachins-geekyants

Apart from the sync Exception, noticed bug in the front end.

  1. Regarding periodic refresh of history screen. Currently, we start periodic refreshes after onboarding completes. Which should be started after sync completes.

  2. Exeption in getSentFileItems

flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@jenmonroe"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: FINER|2022-06-08 12:49:04.845382|AtLookup|AtLookup.findSecondary got answer: 446b7a28-8028-558d-be62-b079bf387bc0.canary.atsign.zone:1101 and closing connection
flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@jenmonroe"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@jenmonroe"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'. 

This is happening when we are not able to convert any sent file JSON string to the view model, the JSON string data might be in old format and might be missing some required fields.

But these two exceptions should not cause any issue in sync process.

sachins-geekyants avatar Jun 09 '22 09:06 sachins-geekyants

Regarding not being able to authenticate even after providing correct key, we have an open issue for it. It was observed with @barbara atsing. #585.

We don't know the reason behind it, Nitesh looked into it but could not reproduce it.

sachins-geekyants avatar Jun 09 '22 09:06 sachins-geekyants

I can reproduce every time

cconstab avatar Jun 09 '22 12:06 cconstab

@cconstab can you think of any user behaviour that might have caused to fail auth? like some combination using multiple apps/atsign or atsign being deleted and onboarded again ?

sachins-geekyants avatar Jun 09 '22 15:06 sachins-geekyants

I have tried other @signs on my desktop and same result.. Very odd

cconstab avatar Jun 09 '22 15:06 cconstab

Very odd that it did not worked with other atsign aslo.

  • Can you check the at_client and at_client_mobile version mentioned in the pubspec.yaml and pubspec.lock file ?
  • Can you try running release tag version with other atsign and still see auth issue ?

If you are running from terminal, please use trunk branch.

sachins-geekyants avatar Jun 10 '22 14:06 sachins-geekyants

Using trunk and Flutter version

Flutter 3.0.2 • channel stable • https://github.com/flutter/flutter.git Framework • revision cd41fdd495 (2 days ago) • 2022-06-08 09:52:13 -0700 Engine • revision f15f824b57 Tools • Dart 2.17.3 • DevTools 2.12.2

PS C:\Users\colin\GitHub@foundation\atmosphere_pro> flutter pub upgrade Resolving dependencies... _fe_analyzer_shared 40.0.0 analyzer 4.1.0 archive 3.3.0 args 2.3.1 asn1lib 1.1.0 async 2.8.2 (2.9.0 available)

at_backupkey_flutter 4.0.2 (was 4.0.1) at_base2e15 1.0.0 at_client 3.0.25 at_client_mobile 3.1.16 ! at_common_flutter 2.0.8 from git https://github.com/atsign-foundation/at_widgets at 265555 in at_common_flutter (overridden) at_commons 3.0.16 at_contact 3.0.5 ! at_contacts_flutter 4.0.5 from git https://github.com/atsign-foundation/at_widgets at 265555 in at_contacts_flutter (overridden) ! at_contacts_group_flutter 4.0.5 from git https://github.com/atsign-foundation/at_widgets at 265555 in at_contacts_group_flutter (overridden) at_lookup 3.0.24 ! at_onboarding_flutter 4.0.3 from git https://github.com/atsign-foundation/at_widgets at 265555 in at_onboarding_flutter (overridden) at_persistence_secondary_server 3.0.25 (was 3.0.24) at_persistence_spec 2.0.5 at_server_status 1.0.3 at_sync_ui_flutter 1.0.3 (was 1.0.2) at_utf7 1.0.0 at_utils 3.0.9 biometric_storage 3.0.1 (4.1.2 available) boolean_selector 2.1.0 carousel_slider 4.1.1 characters 1.2.0 (1.2.1 available) charcode 1.3.1 cli_dialog 0.5.0 cli_util 0.3.5 clock 1.1.0 collection 1.16.0 convert 3.0.2 (was 3.0.1) coverage 1.3.2 (was 1.0.3) cron 0.3.2 (0.5.0 available) cross_file 0.3.3+1 (was 0.3.3) crypto 3.0.2 crypton 2.0.5 (was 2.0.3) csslib 0.17.2 (was 0.17.1) cupertino_icons 0.1.3 (1.0.5 available) dart_console 1.0.0 dbus 0.7.4 (was 0.7.3) (0.7.5 available) desktop_window 0.4.0 emoji_picker_flutter 1.2.1 (was 1.1.2) encrypt 5.0.1 fake_async 1.3.0 ffi 1.2.1 (2.0.0 available) file 6.1.2 file_picker 4.6.1 (was 4.5.1) ! file_saver 0.1.0 from git https://github.com/incrediblezayed/file_saver.git at c0ea84 (overridden) file_selector 0.8.4+3 (was 0.8.4+2) file_selector_linux 0.0.2+1 file_selector_macos 0.8.2+2 (was 0.8.2+1) file_selector_platform_interface 2.0.4 file_selector_web 0.8.1+5 file_selector_windows 0.8.2+2 (was 0.8.2+1) filesystem_picker 2.0.1 (was 2.0.0) fixnum 1.0.1 flutter 0.0.0 from sdk flutter flutter_image_compress 1.1.0 flutter_keychain 2.1.0 flutter_local_notifications 9.6.0 (was 9.5.3+1) flutter_local_notifications_linux 0.5.0+1 (was 0.4.2) flutter_local_notifications_platform_interface 5.0.0 flutter_plugin_android_lifecycle 2.0.6 flutter_qr_reader 1.0.5 flutter_slidable 0.6.0 (1.3.0 available) flutter_test 0.0.0 from sdk flutter flutter_toastr 1.0.3 flutter_web_plugins 0.0.0 from sdk flutter fluttertoast 8.0.9 frontend_server_client 2.1.3 get_it 7.2.0 glob 2.0.2 google_fonts 2.3.3 (was 2.3.2) (3.0.1 available) hive 2.2.2 (was 2.1.0) html 0.15.0 http 0.13.4 http_multi_server 3.2.0 http_parser 4.0.1 (was 4.0.0) image 3.2.0 (was 3.1.3) internet_connection_checker 0.0.1+4 (was 0.0.1+3) intl 0.17.0 io 1.0.3 js 0.6.4 lints 1.0.1 (2.0.0 available) local_notifier 0.1.5 (was 0.1.4) logging 1.0.2 matcher 0.12.11 material_color_utilities 0.1.4 (0.1.5 available) meta 1.7.0 (1.8.0 available) mime 1.0.2 mocktail 0.3.0 msix 3.6.2 mutex 3.0.0 nested 1.0.0 new_version 0.3.1 (was 0.3.0) node_preamble 2.0.1 open_file 3.2.1 package_config 2.0.2 package_info_plus 1.4.2 package_info_plus_linux 1.0.5 package_info_plus_macos 1.3.0 package_info_plus_platform_interface 1.0.2 package_info_plus_web 1.0.5 package_info_plus_windows 1.0.5 path 1.8.1 (1.8.2 available) path_provider 2.0.11 (was 2.0.10) path_provider_android 2.0.14 path_provider_ios 2.0.9 path_provider_linux 2.1.7 (was 2.1.6) path_provider_macos 2.0.6 path_provider_platform_interface 2.0.4 path_provider_windows 2.0.7 (was 2.0.6) (2.1.0 available) pedantic 1.11.1 (discontinued replaced by lints) permission_handler 9.2.0 permission_handler_android 9.0.2+1 permission_handler_apple 9.0.4 permission_handler_platform_interface 3.7.0 permission_handler_windows 0.1.0 petitparser 5.0.0 (was 4.4.0) pin_code_fields 7.4.0 platform 3.1.0 plugin_platform_interface 2.1.2 pointycastle 3.6.0 pool 1.5.0 process 4.2.4 provider 6.0.3 (was 6.0.2) pub_semver 2.1.1 qr 2.1.0 (3.0.1 available) qr_flutter 4.0.0 receive_sharing_intent 1.4.5 rxdart 0.27.4 (was 0.27.3) share_plus 4.0.7 (was 4.0.4) share_plus_linux 3.0.0 share_plus_macos 3.0.1 (was 3.0.0) share_plus_platform_interface 3.0.3 (was 3.0.2) share_plus_web 3.0.1 (was 3.0.0) share_plus_windows 3.0.1 (was 3.0.0) shared_preferences 2.0.15 shared_preferences_android 2.0.12 shared_preferences_ios 2.1.1 shared_preferences_linux 2.1.1 shared_preferences_macos 2.0.4 shared_preferences_platform_interface 2.0.0 shared_preferences_web 2.0.4 shared_preferences_windows 2.1.1 shelf 1.3.0 shelf_packages_handler 3.0.0 shelf_static 1.1.0 shelf_web_socket 1.0.1 sky_engine 0.0.99 from sdk flutter source_map_stack_trace 2.1.0 source_maps 0.10.10 source_span 1.8.2 (1.9.0 available) stack_trace 1.10.0 stream_channel 2.1.0 string_scanner 1.1.0 (1.1.1 available) term_glyph 1.2.0 test 1.21.1 test_api 0.4.9 test_core 0.4.13 timezone 0.8.0 typed_data 1.3.1 (was 1.3.0) url_launcher 6.1.3 (was 6.1.2) url_launcher_android 6.0.17 url_launcher_ios 6.0.17 (was 6.0.16) url_launcher_linux 3.0.1 url_launcher_macos 3.0.1 url_launcher_platform_interface 2.0.5 url_launcher_web 2.0.11 url_launcher_windows 3.0.1 uuid 3.0.6 vector_math 2.1.2 video_compress 3.1.0 video_thumbnail 0.4.6 (0.5.0 available) vm_service 8.3.0 (was 7.5.0) watcher 1.0.1 web_socket_channel 2.2.0 webkit_inspection_protocol 1.1.0 (was 1.0.1) webview_flutter 3.0.4 webview_flutter_android 2.8.11 (was 2.8.8) webview_flutter_platform_interface 1.9.1 (was 1.8.1) webview_flutter_wkwebview 2.8.1 (was 2.7.5) win32 2.6.1 (was 2.5.2) (2.7.0 available) xdg_directories 0.2.0+1 xml 6.1.0 (was 5.3.1) yaml 3.1.1 zxing2 0.1.0 Warning: You are using these overridden dependencies: ! at_common_flutter 2.0.8 from git https://github.com/atsign-foundation/at_widgets at 265555 in at_common_flutter ! at_contacts_flutter 4.0.5 from git https://github.com/atsign-foundation/at_widgets at 265555 in at_contacts_flutter ! at_contacts_group_flutter 4.0.5 from git https://github.com/atsign-foundation/at_widgets at 265555 in at_contacts_group_flutter ! at_onboarding_flutter 4.0.3 from git https://github.com/atsign-foundation/at_widgets at 265555 in at_onboarding_flutter ! file_saver 0.1.0 from git https://github.com/incrediblezayed/file_saver.git at c0ea84 Changed 50 dependencies!

cconstab avatar Jun 10 '22 23:06 cconstab

App still hangs in the sync phase when using @ Colin

I see this in the client logs ....

flutter: FINER|2022-06-10 16:15:17.729350|SyncService|syncing to local: localCommitId 331775 serverCommitId 368353

flutter: FINER|2022-06-10 16:15:17.729350|SyncService|** syncBuilder sync:from:331775:limit:10:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})


flutter: FINER|2022-06-10 16:15:17.746926|SyncService|** syncBuilder sync:from:331775:limit:10:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})


flutter: SEVERE|2022-06-10 16:15:29.200970|AtLookup|Exception in sending to server, Exception: No response after 10000 millis from remote secondary 

flutter: SEVERE|2022-06-10 16:15:29.200970|AtLookup|Error in remote verb execution Exception: No response after 10000 millis from remote secondary

flutter: SEVERE|2022-06-10 16:15:29.200970|SyncService|Exception in sync 5445f1ce-84e4-49e0-920a-188de7402bc0. Reason AT0023: Exception: No response after 10000 millis from remote secondary

flutter: FINER|2022-06-10 16:15:32.236639|Monitor|received response on monitor: notification: {"id":"-1","from":"@colin","to":"@colin","key":"statsNotification.@colin","value":"368353","operation":"update","epochMillis":1654902932387}

flutter: FINER|2022-06-10 16:15:32.237638|SyncService|got stats notification in sync: 368353

and this in the secondary

79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:14:19.645313|InboundListener|RCVD: [_e869c813-72f0-4d9d-8d5c-4ff46c35dc7e] stats:3:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:19.645510|AtSecondaryServer|inside _executeVerbCallBack: stats:3:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:19.645666|DefaultVerbExecutor|verb handler found : StatsVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:19.645770|StatsVerbHandler|In statsVerbHandler statID : :3, regex : (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:14:19.645817|StatsVerbHandler|addStatToResult for id : 3, regex: (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:19.645903|RegexUtil|In commitID getMetrics...regex : (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:19.665471|StatsVerbHandler|Verb : stats  Response: Response{_data: [{"id":"3","name":"lastCommitID","value":"368353"}], _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:14:19.665581|InboundConnectionImpl|SENT: [_e869c813-72f0-4d9d-8d5c-4ff46c35dc7e] data: [{"id":"3","name":"lastCommitID","value":"368353"}]
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:14:26.950292|InboundConnectionImpl|SENT: [_5e5714a3-2654-4472-8d06-1440d5edff2b] notification: {"id":"-1","from":"@colin","to":"@colin","key":"statsNotification.@colin","value":"368353","operation":"update","epochMillis":1654902866950}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:14:31.071564|InboundListener|RCVD: [_e869c813-72f0-4d9d-8d5c-4ff46c35dc7e] stats:3:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:31.071667|AtSecondaryServer|inside _executeVerbCallBack: stats:3:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:31.071731|DefaultVerbExecutor|verb handler found : StatsVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:31.071814|StatsVerbHandler|In statsVerbHandler statID : :3, regex : (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:14:31.071847|StatsVerbHandler|addStatToResult for id : 3, regex: (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:31.071869|RegexUtil|In commitID getMetrics...regex : (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:31.091578|StatsVerbHandler|Verb : stats  Response: Response{_data: [{"id":"3","name":"lastCommitID","value":"368353"}], _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:14:31.091696|InboundConnectionImpl|SENT: [_e869c813-72f0-4d9d-8d5c-4ff46c35dc7e] data: [{"id":"3","name":"lastCommitID","value":"368353"}]
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:14:38.022413|InboundListener|RCVD: [_e869c813-72f0-4d9d-8d5c-4ff46c35dc7e] stats:3:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:38.022465|AtSecondaryServer|inside _executeVerbCallBack: stats:3:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:38.022516|DefaultVerbExecutor|verb handler found : StatsVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:38.022563|StatsVerbHandler|In statsVerbHandler statID : :3, regex : (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:14:38.022577|StatsVerbHandler|addStatToResult for id : 3, regex: (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:38.022588|RegexUtil|In commitID getMetrics...regex : (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:38.040308|StatsVerbHandler|Verb : stats  Response: Response{_data: [{"id":"3","name":"lastCommitID","value":"368353"}], _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:14:38.040393|InboundConnectionImpl|SENT: [_e869c813-72f0-4d9d-8d5c-4ff46c35dc7e] data: [{"id":"3","name":"lastCommitID","value":"368353"}]
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:14:47.384164|InboundConnectionImpl|SENT: [_5e5714a3-2654-4472-8d06-1440d5edff2b] notification: {"id":"-1","from":"@colin","to":"@colin","key":"statsNotification.@colin","value":"368353","operation":"update","epochMillis":1654902887384}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:14:49.474420|InboundListener|RCVD: [_e869c813-72f0-4d9d-8d5c-4ff46c35dc7e] stats:3:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:49.474498|AtSecondaryServer|inside _executeVerbCallBack: stats:3:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:49.474580|DefaultVerbExecutor|verb handler found : StatsVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:49.474699|StatsVerbHandler|In statsVerbHandler statID : :3, regex : (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:14:49.474777|StatsVerbHandler|addStatToResult for id : 3, regex: (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:49.474808|RegexUtil|In commitID getMetrics...regex : (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:49.487068|StatsVerbHandler|Verb : stats  Response: Response{_data: [{"id":"3","name":"lastCommitID","value":"368353"}], _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:14:49.487182|InboundConnectionImpl|SENT: [_e869c813-72f0-4d9d-8d5c-4ff46c35dc7e] data: [{"id":"3","name":"lastCommitID","value":"368353"}]
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:14:53.116573|InboundListener|RCVD: [_5e5714a3-2654-4472-8d06-1440d5edff2b] noop:0
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:53.116654|AtSecondaryServer|inside _executeVerbCallBack: noop:0
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:53.116709|DefaultVerbExecutor|verb handler found : NoOpVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:14:53.116855|NoOpVerbHandler|Verb : noop  Response: Response{_data: ok, _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:14:53.116875|InboundConnectionImpl|SENT: [_5e5714a3-2654-4472-8d06-1440d5edff2b] data:ok
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:15:00.945327|InboundListener|RCVD: [_e869c813-72f0-4d9d-8d5c-4ff46c35dc7e] stats:3:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:00.945397|AtSecondaryServer|inside _executeVerbCallBack: stats:3:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:00.945466|DefaultVerbExecutor|verb handler found : StatsVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:00.945529|StatsVerbHandler|In statsVerbHandler statID : :3, regex : (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:15:00.945559|StatsVerbHandler|addStatToResult for id : 3, regex: (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:00.945577|RegexUtil|In commitID getMetrics...regex : (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:00.959765|StatsVerbHandler|Verb : stats  Response: Response{_data: [{"id":"3","name":"lastCommitID","value":"368353"}], _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:15:00.960036|InboundConnectionImpl|SENT: [_e869c813-72f0-4d9d-8d5c-4ff46c35dc7e] data: [{"id":"3","name":"lastCommitID","value":"368353"}]
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:15:11.953148|InboundConnectionImpl|SENT: [_5e5714a3-2654-4472-8d06-1440d5edff2b] notification: {"id":"-1","from":"@colin","to":"@colin","key":"statsNotification.@colin","value":"368353","operation":"update","epochMillis":1654902911953}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:15:12.285575|InboundListener|RCVD: [_e869c813-72f0-4d9d-8d5c-4ff46c35dc7e] stats:3:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:12.285651|AtSecondaryServer|inside _executeVerbCallBack: stats:3:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:12.285706|DefaultVerbExecutor|verb handler found : StatsVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:12.285752|StatsVerbHandler|In statsVerbHandler statID : :3, regex : (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:15:12.285768|StatsVerbHandler|addStatToResult for id : 3, regex: (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:12.285776|RegexUtil|In commitID getMetrics...regex : (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:12.299220|StatsVerbHandler|Verb : stats  Response: Response{_data: [{"id":"3","name":"lastCommitID","value":"368353"}], _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:15:12.299292|InboundConnectionImpl|SENT: [_e869c813-72f0-4d9d-8d5c-4ff46c35dc7e] data: [{"id":"3","name":"lastCommitID","value":"368353"}]
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:15:15.333553|InboundListener|RCVD: [_e869c813-72f0-4d9d-8d5c-4ff46c35dc7e] stats:3:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:15.333639|AtSecondaryServer|inside _executeVerbCallBack: stats:3:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:15.333795|DefaultVerbExecutor|verb handler found : StatsVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:15.333900|StatsVerbHandler|In statsVerbHandler statID : :3, regex : (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:15:15.333922|StatsVerbHandler|addStatToResult for id : 3, regex: (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:15.333946|RegexUtil|In commitID getMetrics...regex : (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:15.350150|StatsVerbHandler|Verb : stats  Response: Response{_data: [{"id":"3","name":"lastCommitID","value":"368353"}], _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:15:15.350294|InboundConnectionImpl|SENT: [_e869c813-72f0-4d9d-8d5c-4ff46c35dc7e] data: [{"id":"3","name":"lastCommitID","value":"368353"}]
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:15:15.437540|InboundListener|RCVD: [_e869c813-72f0-4d9d-8d5c-4ff46c35dc7e] stats:3:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:15.437650|AtSecondaryServer|inside _executeVerbCallBack: stats:3:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:15.437719|DefaultVerbExecutor|verb handler found : StatsVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:15.437779|StatsVerbHandler|In statsVerbHandler statID : :3, regex : (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:15:15.437812|StatsVerbHandler|addStatToResult for id : 3, regex: (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:15.437839|RegexUtil|In commitID getMetrics...regex : (.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:15.456456|StatsVerbHandler|Verb : stats  Response: Response{_data: [{"id":"3","name":"lastCommitID","value":"368353"}], _type: null, _isError: false, _errorMessage: null}
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:15:15.456539|InboundConnectionImpl|SENT: [_e869c813-72f0-4d9d-8d5c-4ff46c35dc7e] data: [{"id":"3","name":"lastCommitID","value":"368353"}]
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:15:17.972958|InboundListener|RCVD: [_e869c813-72f0-4d9d-8d5c-4ff46c35dc7e] sync:from:331775:limit:10:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:17.973014|AtSecondaryServer|inside _executeVerbCallBack: sync:from:331775:limit:10:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:17.973091|DefaultVerbExecutor|verb handler found : SyncProgressiveVerbHandler
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | INFO|2022-06-10 23:15:18.146841|InboundListener|RCVD: [_e869c813-72f0-4d9d-8d5c-4ff46c35dc7e] sync:from:331775:limit:10:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:18.146900|AtSecondaryServer|inside _executeVerbCallBack: sync:from:331775:limit:10:(.mospherepro|atconnections|[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12})
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    |
79b6d83f-5026-5fda-8299-5a0704bd2416_secondary.1.vqrbfbqc7uxa@canary-02.us-central1-b.c.secondaries.internal    | FINER|2022-06-10 23:15:18.146973|DefaultVerbExecutor|verb handler found : SyncProgressiveVerbHandler

cconstab avatar Jun 10 '22 23:06 cconstab

I am now blocked as both atmosphere and wavi will not sync and the atsign @ colin is active on them both so I cannot reset the atsign

cconstab avatar Jun 10 '22 23:06 cconstab

Just to re-confirm - this is still a problem only on the desktop apps - ie not the Android or iOS apps?

gkc avatar Jun 10 '22 23:06 gkc

Some application clues

flutter: FINER|2022-06-10 16:51:04.537647|AtLookup|AtLookup.findSecondary got answer: null and closing connection 

[ERROR:flutter/lib/ui/ui_dart_state.cc(198)] Unhandled Exception: Exception: Unable to find secondary address for atSign:kevin
#0      CacheableSecondaryAddressFinder._updateCache (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:92:9)
<asynchronous suspension>
#1      CacheableSecondaryAddressFinder.findSecondary (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:47:7)
<asynchronous suspension>
#2      AtClientValidation.isAtSignExists (package:at_client/src/util/at_client_validation.dart:51:7)
<asynchronous suspension>

flutter: FINER|2022-06-10 16:51:04.544647|AtLookup|AtLookup.findSecondary got answer: null and closing connection

[ERROR:flutter/lib/ui/ui_dart_state.cc(198)] Unhandled Exception: Exception: Unable to find secondary address for atSign:kevin
#0      CacheableSecondaryAddressFinder._updateCache (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:92:9)
<asynchronous suspension>
#1      CacheableSecondaryAddressFinder.findSecondary (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:47:7)
<asynchronous suspension>
#2      AtClientValidation.isAtSignExists (package:at_client/src/util/at_client_validation.dart:51:7)
<asynchronous suspension>

flutter: FINER|2022-06-10 16:51:04.559645|AtLookup|AtLookup.findSecondary received atsign: kevin

flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@kevin"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: FINER|2022-06-10 16:51:04.618683|AtLookup|AtLookup.findSecondary received atsign: kevin 

flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@kevin"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: FINER|2022-06-10 16:51:04.626684|AtLookup|AtLookup.findSecondary got answer: null and closing connection

[ERROR:flutter/lib/ui/ui_dart_state.cc(198)] Unhandled Exception: Exception: Unable to find secondary address for atSign:kevin
#0      CacheableSecondaryAddressFinder._updateCache (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:92:9)
<asynchronous suspension>
#1      CacheableSecondaryAddressFinder.findSecondary (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:47:7)
<asynchronous suspension>
#2      AtClientValidation.isAtSignExists (package:at_client/src/util/at_client_validation.dart:51:7)
<asynchronous suspension>

flutter: FINER|2022-06-10 16:51:04.666684|AtLookup|AtLookup.findSecondary received atsign: kevin 

flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@kevin"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: FINER|2022-06-10 16:51:04.673684|AtLookup|AtLookup.findSecondary got answer: null and closing connection

[ERROR:flutter/lib/ui/ui_dart_state.cc(198)] Unhandled Exception: Exception: Unable to find secondary address for atSign:kevin
#0      CacheableSecondaryAddressFinder._updateCache (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:92:9)
<asynchronous suspension>
#1      CacheableSecondaryAddressFinder.findSecondary (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:47:7)
<asynchronous suspension>
#2      AtClientValidation.isAtSignExists (package:at_client/src/util/at_client_validation.dart:51:7)
<asynchronous suspension>

flutter: FINER|2022-06-10 16:51:04.716679|AtLookup|AtLookup.findSecondary received atsign: kevin 

flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@kevin"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: FINER|2022-06-10 16:51:04.725252|AtLookup|AtLookup.findSecondary got answer: null and closing connection

[ERROR:flutter/lib/ui/ui_dart_state.cc(198)] Unhandled Exception: Exception: Unable to find secondary address for atSign:kevin
#0      CacheableSecondaryAddressFinder._updateCache (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:92:9)
<asynchronous suspension>
#1      CacheableSecondaryAddressFinder.findSecondary (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:47:7)
<asynchronous suspension>
#2      AtClientValidation.isAtSignExists (package:at_client/src/util/at_client_validation.dart:51:7)
<asynchronous suspension>

flutter: FINER|2022-06-10 16:51:04.776682|AtLookup|AtLookup.findSecondary received atsign: kevin 

flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@kevin"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: FINER|2022-06-10 16:51:04.835709|AtLookup|AtLookup.findSecondary got answer: null and closing connection 

[ERROR:flutter/lib/ui/ui_dart_state.cc(198)] Unhandled Exception: Exception: Unable to find secondary address for atSign:kevin
#0      CacheableSecondaryAddressFinder._updateCache (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:92:9)
<asynchronous suspension>
#1      CacheableSecondaryAddressFinder.findSecondary (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:47:7)
<asynchronous suspension>
#2      AtClientValidation.isAtSignExists (package:at_client/src/util/at_client_validation.dart:51:7)
<asynchronous suspension>

flutter: FINER|2022-06-10 16:51:04.835709|AtLookup|AtLookup.findSecondary got answer: null and closing connection

[ERROR:flutter/lib/ui/ui_dart_state.cc(198)] Unhandled Exception: Exception: Unable to find secondary address for atSign:kevin
#0      CacheableSecondaryAddressFinder._updateCache (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:92:9)
<asynchronous suspension>
#1      CacheableSecondaryAddressFinder.findSecondary (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:47:7)
<asynchronous suspension>
#2      AtClientValidation.isAtSignExists (package:at_client/src/util/at_client_validation.dart:51:7)
<asynchronous suspension>

flutter: FINER|2022-06-10 16:51:04.844248|AtLookup|AtLookup.findSecondary received atsign: kevin

flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@kevin"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: FINER|2022-06-10 16:51:04.897620|AtLookup|AtLookup.findSecondary received atsign: kevin 

flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@kevin"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: FINER|2022-06-10 16:51:04.942621|AtLookup|AtLookup.findSecondary got answer: null and closing connection 

[ERROR:flutter/lib/ui/ui_dart_state.cc(198)] Unhandled Exception: Exception: Unable to find secondary address for atSign:kevin
#0      CacheableSecondaryAddressFinder._updateCache (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:92:9)
<asynchronous suspension>
#1      CacheableSecondaryAddressFinder.findSecondary (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:47:7)
<asynchronous suspension>
#2      AtClientValidation.isAtSignExists (package:at_client/src/util/at_client_validation.dart:51:7)
<asynchronous suspension>

flutter: FINER|2022-06-10 16:51:04.944622|AtLookup|AtLookup.findSecondary received atsign: kevin

flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@kevin"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: FINER|2022-06-10 16:51:04.972133|AtLookup|AtLookup.findSecondary got answer: null and closing connection 

[ERROR:flutter/lib/ui/ui_dart_state.cc(198)] Unhandled Exception: Exception: Unable to find secondary address for atSign:kevin
#0      CacheableSecondaryAddressFinder._updateCache (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:92:9)
<asynchronous suspension>
#1      CacheableSecondaryAddressFinder.findSecondary (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:47:7)
<asynchronous suspension>
#2      AtClientValidation.isAtSignExists (package:at_client/src/util/at_client_validation.dart:51:7)
<asynchronous suspension>

flutter: FINER|2022-06-10 16:51:04.995720|AtLookup|AtLookup.findSecondary received atsign: kevin 

flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@kevin"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: FINER|2022-06-10 16:51:05.041719|AtLookup|AtLookup.findSecondary received atsign: kevin 

flutter: exeption in getSentFileItems : NoSuchMethodError: Class 'String' has no instance method 'forEach'.
Receiver: "@kevin"
Tried calling: forEach(Closure: (dynamic) => Null)
flutter: FINER|2022-06-10 16:51:05.049720|AtLookup|AtLookup.findSecondary got answer: null and closing connection

[ERROR:flutter/lib/ui/ui_dart_state.cc(198)] Unhandled Exception: Exception: Unable to find secondary address for atSign:kevin
#0      CacheableSecondaryAddressFinder._updateCache (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:92:9)
<asynchronous suspension>
#1      CacheableSecondaryAddressFinder.findSecondary (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:47:7)
<asynchronous suspension>
#2      AtClientValidation.isAtSignExists (package:at_client/src/util/at_client_validation.dart:51:7)
<asynchronous suspension>

flutter: FINER|2022-06-10 16:51:05.094758|AtLookup|AtLookup.findSecondary got answer: null and closing connection 

[ERROR:flutter/lib/ui/ui_dart_state.cc(198)] Unhandled Exception: Exception: Unable to find secondary address for atSign:kevin
#0      CacheableSecondaryAddressFinder._updateCache (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:92:9)
<asynchronous suspension>
#1      CacheableSecondaryAddressFinder.findSecondary (package:at_lookup/src/cache/cacheable_secondary_address_finder.dart:47:7)
<asynchronous suspension>
#2      AtClientValidation.isAtSignExists (package:at_client/src/util/at_client_validation.dart:51:7)
<asynchronous suspension>

cconstab avatar Jun 10 '22 23:06 cconstab

I have spent hours on this and see so many errors it is unclear to me what is going on

cconstab avatar Jun 11 '22 02:06 cconstab

Just to double check I deleted @ Colin on my Android phone and reinstalled and it worked just fine

cconstab avatar Jun 11 '22 04:06 cconstab

@cconstab : I have tested with atmosphere pro with trunk branch code (onboarded the atsign for 3 to 4 times) on windows desktop version and unable to replicate the issue. The sync completes successfully. Attaching the video. Can you please try with atmosphere pro trunk branch code (if in case you are using other version)

https://user-images.githubusercontent.com/16645698/173192144-54b8c573-85f6-4a94-a12c-e1738575ecf7.mp4

Apologies, my laptop frozen and accidentally ticket was closed. Reopened the issue.

sitaram-kalluri avatar Jun 11 '22 14:06 sitaram-kalluri

@cconstab @sachins-geekyants : What is the version of atmosphere pro used for testing (Is it different from what we have in trunk branch)?

sitaram-kalluri avatar Jun 11 '22 14:06 sitaram-kalluri

Interesting as I am using the trunk branch and it fails for me!

cconstab avatar Jun 11 '22 15:06 cconstab

As does wavi!

cconstab avatar Jun 11 '22 15:06 cconstab

OK after @kalluriramkumar observation I wondered if Starlink my new ISP had anything to do with this ?? It simply cannot right ???

Well I swapped to SierraTel and things just worked !

Now to figure out what on earth is going on !!

NETWORKS!

So I moved back to Starlink and guess what everythig still works OMG this is odd..

Eve odder @ wavi is also unstuck

cconstab avatar Jun 11 '22 19:06 cconstab

Ok I had the same synch problem on my MAC and so I wanted to figure out what was going on without changing ISP..

The solution ?

I chnaged the MTU on the WIFI interface to 1280 (normally set to 1500)

And then magically it worked !

Once working again I cannot get it to fail even when I move back to 1500..

BUT this is likely because we are not moving as much data and the frame is not full ??

This needs some more tests in a Dart program sending large chunks of data..

If anyone else hits this problem, this is the work around to get them up..

BUT this is super ugly !

This also explains why ANdroid was working as it likely has a lower MTU set....

My MTU is set correctly on the network... Here's some proof..

cconstab@cally:~$ ping google.com -s 1498 -M do
PING google.com (142.250.68.78) 1498(1526) bytes of data.
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
^C
--- google.com ping statistics ---
3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2025ms

cconstab@cally:~$

MTU discovery finds 1500 and so the stack should adjust accordingly...

We hit this same issue on the server/docker side but I think this is showing that either fragmentation or MTU discovery sis not working..

More investigation and test code to be written this week...

Also explains why nothing was obvious in the logs... Ho hum.. @kalluriramkumar this is all thanks to your video above THANK YOU !!!

Out of interest what is your MTU on your machine ?

cconstab avatar Jun 12 '22 17:06 cconstab

@cconstab : MTU in my machine is 1500.

sitaram@sitaram-ThinkPad-E14:~$ ping google.com -s 1498 -M do
PING google.com(maa03s46-in-x0e.1e100.net (2404:6800:4007:82b::200e)) 1498 data bytes
ping: local error: message too long, mtu: 1500
ping: local error: message too long, mtu: 1500
ping: local error: message too long, mtu: 1500
^C
--- google.com ping statistics ---
3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2032ms

sitaram-kalluri avatar Jun 13 '22 02:06 sitaram-kalluri