arcade icon indicating copy to clipboard operation
arcade copied to clipboard

TimelineRecords kusto table is missing dnceng-public records

Open ulisesh opened this issue 2 years ago • 13 comments

We don't have any rows from dnceng-public in TimelineRecords, whatever services adds the rows needs to monitor our new AzDo instance

ulisesh avatar Sep 07 '22 23:09 ulisesh

FYI to avoid accidental double-work, @epananth was asking me about this the other day.

garath avatar Sep 07 '22 23:09 garath

I think it's this? https://github.com/dotnet/arcade-services/blob/main/src/Telemetry/AzureDevOpsTimeline/.config/settings.json#L5-L8

riarenas avatar Sep 07 '22 23:09 riarenas

More generally this is the project. It will need to be taught how to look at more than one Organization.

garath avatar Sep 07 '22 23:09 garath

@epananth and I talked about how we implemented the multi AzDO org support in Helix Services, because she said she needed to do the same in Arcade-Services. I'm assuming this work is related to that discussion.

missymessa avatar Sep 07 '22 23:09 missymessa

Good, thanks everyone!

cc @Chrisboh as some reports are missing data

ulisesh avatar Sep 07 '22 23:09 ulisesh

I'm probably the last person to touch this code (having it get all timelines instead of just the latest). Let me know if I can help at all.

garath avatar Sep 07 '22 23:09 garath

I think this should be part of the epic in charge of moving to the new instance

ulisesh avatar Sep 07 '22 23:09 ulisesh

I was talking to @Chrisboh about this a few mins ago. We need to update this to support both orgs, so that CI council dashboards work.

epananth avatar Sep 07 '22 23:09 epananth

Yes, I was tracking this as a part of dashboard work, but I can use this instead

epananth avatar Sep 07 '22 23:09 epananth

Feel free to take this issue to track the work or close it as a dupe if there is another one already 😄

ulisesh avatar Sep 07 '22 23:09 ulisesh

As I think about this issue more, I'm starting to think this is the root cause for the Queue Insights alert as well: https://github.com/dotnet/arcade/issues/10710.

@dkurepa can probably confirm it best, but if we're not ingesting builds from dnceng-public, the matrix of truth doesn't have information from those builds, and then queue insights won't record some of the needed entries for queue insights to work correctly.

riarenas avatar Sep 08 '22 18:09 riarenas

I have the code changes, I am currently testing it. Once that is successful I will have my PR for this

epananth avatar Sep 08 '22 22:09 epananth

@garath gave me some suggestions to test this, trying that now

epananth avatar Sep 08 '22 22:09 epananth

This is now fixed.

alexperovich avatar Oct 03 '22 21:10 alexperovich