community
community copied to clipboard
open-telemetry
OTEL calendar event series for Prometheus workgroup deleted?
As a facilitator for the Prometheus workgroup weekly meeting 8-9am PT on Wednesdays, I found the event series deleted from the OTEL calendar. A bunch of us who meet regularly to discuss our work on Prometheus-OTEL interoperability found the event had disappeared. @mtwo @tedsuo any ideas on who deleted the series? :-(
Solution: I'd like to restore the series to ensure everyone can continue participation regularly. I think I have access to the calendar and will setup a new series if I have access using the info on my personal calendar.
I doubt that any one person did this, more likely some kind of bug or automated process. In the last two weeks, a number of series have vanished once each and I've had to recreate them. None have had to be recreated multiple times, and some other meetings haven't vanished but our mailing list refuses to forward their invites, which leads me to think that it's some kind of issue with our calendar or one of its integrations. I don't think that this is related to the Zoom problems that we've been having, though I'm not certain about this.
Given these issues and the Zoom issues (the Zoom issues have somewhat known cause at least), I'm considering creating a new calendar (the current one is tied to a defunct Google account) and reverting our Zoom setup to the old process (multiple accounts).
Anyways, let me know if you're able to recreate the series. If not, I can produce it ASAP!
two more data points:
- we had one Java meeting occurrence go missing ("Java + Instrumentation Office Hours", last Thu 6pm Pacific Time), but future meetings were (and still are) there
- JVM metrics folks reported last night that two occurrences of the bi-weekly APAC/EU timezone meeting went missing but they could see future meetings. I can see the missing meeting occurrences when I go look now, so I'm not sure if someone re-created them (@anuraaga?) or if they just went missing temporarily.
I was wondering if it's possible that someone accidentally deleted them thinking they were deleting them from their personal calendar(?)
To clarify I didn't add any calendar entries - don't have access to the calendar to do so in the first place and to prevent accidental edits seems to be a good thing :)
Two more meetings just went missing: JVM metrics on Feb 7 and Feb 21 (but Mar 7 is still there). I will wait a day to re-create them, just curious if they will come back by themselves somehow.
Two more meetings just went missing: JVM metrics on Feb 7 and Feb 21 (but Mar 7 is still there). I will wait a day to re-create them, just curious if they will come back by themselves somehow.
I found these two occurrences in the OpenTelemetry Public Calendar trash and was able to restore them from there
Interestingly the zoom links were missing from the two restored occurrences, maybe pointing to the Zoom integration as @mtwo suspected above
Thanks for the updates. @SergeyKanzhelev mentioned that Kubernetes was able to get their own GSuite organization, which makes managing the calendar somewhat easier. Apparently they were running into some kind of public calendar limitations, which might explain some of the weirdness that we're now experiencing with the calendar invites mailing list.
I'll reach out to Amye and find out if we can get this. Regardless of the outcome, I'll revert back to our older Zoom setup. May take a week or two, however.
google workspace-owned calendars will make it easier to get access to the audit logs to provide transparency on why events got deleted or rescheduled.
I managed to get audit logs for the current calendar. It shows that calendar itself deleted events, which as I was told indicates it was done thru integration. I'm still following up on whether audit logs can give more than just fact that it was done thru integration, but since we only have one integration with Zoom as far as I know, it make Zoom the biggest suspect. I can share audit logs for deleted events if anybody is interested.
@mtwo, should we cancel the integraiton with Zoom? I glanced over and didn't find any obvious solution - I don't understand the logic zoom uses to delete events (why it even deletes those???)
I don't understand either :(
I've started migrating some Zoom meetings back to the old instant meetings format, though there will be one or two left on the single Zoom ID format until we get additional Zoom accounts. I'll ping Amye for these over Slack once Slack recovers from its outage.
@mtwo have you disabled zoom to calendar integration?
I cannot check now as I'm locked out of the accounts - unusual login activity is detected and zoom wants the code from the e-mail and only @amye has access to those e-mails. @amye can you help checking whether integration was disabled?
This is a Zoom thing that they've started doing. I can set your emails to go to an Otel-Zoom-admins group for you instead
Maybe you can set up eight google group or lists.io mailing list as a recipient of these e-mails? So we can have multiple admins?
I'll open a servicedesk ticket for this, we cannot track through GH effectively :)
We took care of this in Servicedesk as that's where staff can track, github is not an effective tracker. [email protected] is live. Austin, Sergey, Ted, Morgan have access to that.
I haven't heard of this happening in a while, I'm going to close this issue, but please open a new issue and link to this if anyone continues to see this.