manager icon indicating copy to clipboard operation
manager copied to clipboard

change: [M3-8450] - Optimize event query timeframe: Reduce historical data fetch from 90 days to 7 days

Open jaalah-akamai opened this issue 1 year ago โ€ข 4 comments

Description ๐Ÿ“

This change optimizes Cloud Manager's event querying by reducing the historical data range. Specifically, it limits the event query timeframe from the current 90 days to a more focused 7-day period. This adjustment aims to improve query performance and reduce unnecessary data retrieval while still providing relevant recent event information.

Changes ๐Ÿ”„

  • Previously we fetched the first 100 events but returned almost 500 results/records. These events pull from the last 90 days. Now we're returning events from the last 7 days up to a max of 25 results per query.

Target release date ๐Ÿ—“๏ธ

8/19

Preview ๐Ÿ“ท

Before After
Screenshot 2024-08-15 at 1 00 23โ€ฏPM created-filter

How to test ๐Ÿงช

[!note] We are working to have this fully covered via E2E tests in an upcoming PR cc: @jdamore-linode

Prerequisites

  • Checkout branch and observe you network tab

Verification steps

  • Go to http://localhost:3000/events
  • Observe that initial load returns at most 25 events
  • As you scroll observe subsequent queries fetch next 25
  • Ensure there aren't any redundant events
  • Ensure you're able to reach the end of your events list (90 days worth of events)

As an Author I have considered ๐Ÿค”

Check all that apply

  • [x] ๐Ÿ‘€ Doing a self review
  • [ ] โ” Our contribution guidelines
  • [x] ๐Ÿค Splitting feature into small PRs
  • [x] โž• Adding a changeset
  • [ ] ๐Ÿงช Providing/Improving test coverage
  • [ ] ๐Ÿ” Removing all sensitive information from the code and PR description
  • [ ] ๐Ÿšฉ Using a feature flag to protect the release
  • [ ] ๐Ÿ‘ฃ Providing comprehensive reproduction steps
  • [ ] ๐Ÿ“‘ Providing or updating our documentation
  • [ ] ๐Ÿ•› Scheduling a pair reviewing session
  • [ ] ๐Ÿ“ฑ Providing mobile support
  • [ ] โ™ฟ Providing accessibility support

jaalah-akamai avatar Aug 14 '24 18:08 jaalah-akamai

Coverage Report: โœ…
Base Coverage: 86.04%
Current Coverage: 86.04%

github-actions[bot] avatar Aug 14 '24 18:08 github-actions[bot]

This seems to be working better with the changes from 37e29a3

If we proceed with ordering events by created instead of id, I believe we'll need to assess the implications this has on useMarkEventsAsSeen and generatePollingFilter to account for the fact that we order by created instead of id. I'm realizing this because I have some events that are not are not marked as seen, but they should be.

https://github.com/user-attachments/assets/471be689-b58a-476c-98fa-1630af4498c5

The API appears to use id rather than created to compute seen, so we might have some issues making this work as expected. Going to keep thinking ๐Ÿง 

bnussman-akamai avatar Aug 15 '24 17:08 bnussman-akamai

The API appears to use id rather than created to compute seen, so we might have some issues making this work as expected. Going to keep thinking ๐Ÿง 

We may have to loop through all events and mark them as seen individually. We may be able to optimize this by keeping track of the highest id we have marked as seen and avoiding sending requests for lower ids.

Edit: realized we can accomplish the same with a single request by marking the highest id as seen.

hkhalil-akamai avatar Aug 15 '24 22:08 hkhalil-akamai

@bnussman-akamai Updates are working very nice. Both caches stay in sync for in-progress events and initial query fetches < 100. I think that part of our work is sound ๐Ÿ”ฅ

@jdamore-linode Are we just trying to get Makes initial fetch to events endpoint test working?

jaalah-akamai avatar Aug 23 '24 02:08 jaalah-akamai

I'm wondering if this needs to be updated

https://github.com/linode/manager/blob/8176f7a7b5bf75b16d0b18d25255203e7e174f20/packages/manager/src/features/Images/ImagesLanding/ImagesLanding.tsx#L195-L201

I don't have lots of context ImagesLanding's use of events

bnussman-akamai avatar Aug 29 '24 16:08 bnussman-akamai

closing as rebase is difficult. moved to https://github.com/linode/manager/pull/10899

abailly-akamai avatar Sep 06 '24 03:09 abailly-akamai