maxun icon indicating copy to clipboard operation
maxun copied to clipboard
Published 1 week ago • verified publisher icon getmaxun

Performance Lag When Selecting Elements in "Create List" on Maoyan Leaderboard Page

Open sunyuan1111 opened this issue 1 year ago • 4 comments

Hello,

I'm experiencing performance issues with Maxun when attempting to scrape the Maoyan movie leaderboard page (https://www.maoyan.com/board/4).

Steps to Reproduce:

  1. Navigate to https://www.maoyan.com/board/4.
  2. In Maxun, click on "Create List."
  3. Attempt to select multiple elements on the page for data extraction.

Observed Behavior: After selecting an element, the interface becomes sluggish, making it difficult to select multiple elements consecutively.

Is there a solution to this issue? Thank you for your assistance!

sunyuan1111 avatar Dec 19 '24 06:12 sunyuan1111

Thanks for reporting. Can you tell us:

  1. Your operating system
  2. System specs (RAM etc.)

It will help us understand the issue better and roll out a fix. @sunyuan1111

amhsirak avatar Dec 19 '24 17:12 amhsirak

Thanks for reporting. Can you tell us:

  1. Your operating system
  2. System specs (RAM etc.)

It will help us understand the issue better and roll out a fix. @sunyuan1111

Response:

Hello, thank you for your response!

Here is my system information:

  1. Operating System: Windows 11 Home Edition (Version 23H2)
  2. System Specifications:
    • Device Name: sunyuan
    • Processor: AMD Ryzen 5 5600H with Radeon Graphics, 3.30 GHz
    • Memory (RAM): 16.0 GB (15.9 GB usable)
    • System Type: 64-bit operating system, x64-based processor
    • Additional Information:
      • Installation Date: 2023/9/21
      • OS Build: 22631.4602
      • Windows Feature Experience Pack: 1000.22700.1055.0

While using Maxun, I noticed the following issues:

  1. When attempting to scrape the Maoyan leaderboard page, after selecting an element, the interface becomes sluggish, making it difficult to select multiple elements consecutively.
  2. The logs indicate that the remote browser is frequently retrieved from the pool, but it seems there’s no resource release happening.

Questions:

  1. Does the complexity of the page significantly affect Maxun's performance? Are there any optimization suggestions for dynamic pages?
  2. Is it possible to adjust Maxun's resource scheduling mechanism or optimize the selector engine to resolve the sluggish behavior?

If needed, I can provide more detailed logs or configuration information. Looking forward to your guidance!

Thank you! QQ_1734663746799 QQ_1734663782964

backend-1 | 2024-12-20T03:03:17.472Z debug: Remote browser with id: 650a8bb4-2aa9-47de-afe2-7f3a29afb66d retrieved from the pool backend-1 | 2024-12-20T03:03:17.505Z debug: Remote browser with id: 650a8bb4-2aa9-47de-afe2-7f3a29afb66d retrieved from the pool backend-1 | 2024-12-20T03:03:17.568Z debug: Remote browser with id: 650a8bb4-2aa9-47de-afe2-7f3a29afb66d retrieved from the pool backend-1 | 2024-12-20T03:03:17.602Z debug: Remote browser with id: 650a8bb4-2aa9-47de-afe2-7f3a29afb66d retrieved from the pool backend-1 | 2024-12-20T03:03:17.645Z debug: Remote browser with id: 650a8bb4-2aa9-47de-afe2-7f3a29afb66d retrieved from the pool backend-1 | 2024-12-20T03:03:17.674Z debug: Remote browser with id: 650a8bb4-2aa9-47de-afe2-7f3a29afb66d retrieved from the pool backend-1 | 2024-12-20T03:03:18.009Z debug: Remote browser with id: 650a8bb4-2aa9-47de-afe2-7f3a29afb66d retrieved from the pool backend-1 | 2024-12-20T03:03:18.040Z debug: Remote browser with id: 650a8bb4-2aa9-47de-afe2-7f3a29afb66d retrieved from the pool backend-1 | 2024-12-20T03:03:18.085Z debug: Remote browser with id: 650a8bb4-2aa9-47de-afe2-7f3a29afb66d retrieved from the pool backend-1 | 2024-12-20T03:03:18.132Z debug: Remote browser with id: 650a8bb4-2aa9-47de-afe2-7f3a29afb66d retrieved from the pool backend-1 | 2024-12-20T03:03:18.178Z debug: Remote browser with id: 650a8bb4-2aa9-47de-afe2-7f3a29afb66d retrieved from the pool backend-1 | 2024-12-20T03:03:18.214Z debug: Remote browser with id: 650a8bb4-2aa9-47de-afe2-7f3a29afb66d retrieved from the pool backend-1 | 2024-12-20T03:03:18.263Z debug: Remote browser with id: 650a8bb4-2aa9-47de-afe2-7f3a29afb66d retrieved from the pool backend-1 | 2024-12-20T03:03:18.291Z debug: Remote browser with id: 650a8bb4-2aa9-47de-afe2-7f3a29afb66d retrieved from the pool backend-1 | 2024-12-20T03:03:18.334Z debug: Remote browser with id: 650a8bb4-2aa9-47de-afe2-7f3a29afb66d retrieved from the pool backend-1 | 2024-12-20T03:03:18.365Z debug: Remote browser with id: 650a8bb4-2aa9-47de-afe2-7f3a29afb66d retrieved from the pool

sunyuan1111 avatar Dec 20 '24 03:12 sunyuan1111

@sunyuan1111 Thanks for such a detailed response. We are looking into this and will get back as soon as possible.

amhsirak avatar Dec 20 '24 07:12 amhsirak

@sunyuan1111 we have identified the problem. Improvements will be available from v0.0.6. We are actively working on this. Thank you!

amhsirak avatar Dec 22 '24 19:12 amhsirak