amoro [Improvement][AMS]: Improve list content of optimizing page

Search before asking

[X] I have searched in the issues and found no similar issues.

What would you like to be improved?

Add resource (name CPU / Memory)column in optimizing list page which indicates the resource consumption of related table runtime
Modify column of 'Optimizing Status' name to Action and Action Stage which is related to common actions including optimizing, expiring, cleaning and others
Add sort function for column 'Occupation'
Add filter widget of checkbox for Action after filter text boxes
Remove file count and file size columns to provide more rooms for table name
Support aggregating view of rows in optimizing list(maybe another issue)

Quick view:

How should we improve?

For 1 - 3, the meanings are simple. Let's skip.

Add filter widget of checkbox for Action after filter text boxes We can add a checkbox next to the table filter text box that lists all supported actions for multiple selections
Support aggregating view of rows in optimizing list Previously, the optimizing page only displayed a limited set of actions: minor optimizing, major optimizing, and full optimizing, with only one action executing at a time. However, in the future, different actions such as 'expire' and 'clean' will be introduced, and it may be possible to have multiple processes occurring simultaneously on a single table. The optimizing list is presented with one row per table, and for this, an aggregate display on the table is needed. By clicking to expand, one can see all processes for that table. The expanded view maintains the same columns as before, with aggregated columns showing the maximum or sum as appropriate

Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Subtasks

No response

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Aug 29 '24 03:08 majin1102

i can submit this PR

Aug 29 '24 08:08 Liammcl

Two interfaces to be added:

Retrieve All Available Actions

GET /optimize/actions

Description

This endpoint is used to fetch all available action types in the system. These action types include but are not limited to minor, major, clean, etc.

Successful Response

Code: 200 OK
Content Example:
```
[
  "minor",
  "major",
  "clean"
]
```

Error Response

Code: 500 Internal Server Error Returned when there is a server error and the list of actions cannot be retrieved.

Get All Active Processes for a Specific Table

GET /optimizerGroups/{optimizerGroup}/{tableId}/processes

Description

This endpoint retrieves information about all active processes for a specified optimizer group and table ID. Each process returned includes details such as table name, action type, duration, file count, file size, resource allocation, quota, and quota occupation.

URL Parameters

optimizerGroup (string): The unique identifier for the optimizer group.
tableId (string): The unique identifier for the table.

Successful Response

Code: 200 OK

Content Example:

[
  {
    "tableName": "my_table",
    "action": "minor",
    "duration": 3600,
    "fileCount": 150,
    "fileSize": 204800,
    "resource": "4c 8g",
    "quota": 0.5,
    "quotaOccupation": 1.2
  },
  {
    "tableName": "my_table",
    "action": "major",
    "duration": 7200,
    "fileCount": 300,
    "fileSize": 409600,
    "resource": "8c 16g",
    "quota": 0.7,
    "quotaOccupation": 1.5
  }
]

Error Responses

Code: 400 Bad Request Returned when the provided optimizerGroup or tableId parameters are invalid.
Code: 404 Not Found Returned when the specified optimizer group or table does not exist.
Code: 500 Internal Server Error Returned when there is a server error and the process information cannot be retrieved.

Notes

The resource field describes the resource allocation for the process, formatted as a string representing CPU cores and memory, e.g., "4c 8g" for 4 CPU cores and 8 GB of memory.
The quota field represents the fraction of the resource quota that is allocated to the table, with a value ranging from 0 to 1.
The quotaOccupation field indicates the current occupation of the quota, which can exceed 1 if the process is using more resources than initially allocated.

These API descriptions provide sufficient information for both frontend and backend developers to understand and implement the calling and response handling of the interfaces.

Sep 02 '24 06:09 majin1102

Modification for interface /optimizerGroups/{optimizerGroup}/tables

add action filter in request message which is a string list
add sortField of string type in request message
add resource column and field for response message

Sep 02 '24 06:09 majin1102

1

Sep 02 '24 07:09 zhangmo8

Discussed with @zhoujinsong

Let's change 'Optimizing Status' column to 'Status' which use format as action.status( idle keeps ) like:

minor_optimizing.pending
major_optimizing.running
full_optimizing.commtting
expire-data.running
idle

And the colored dots keep as idle grey, pending yellow and others green.

Aggregate column logicals from processes to tables

Duration. pick the max duration of all status
File Count, File Size, Resource, Occupy pick the sum of all processes data
Quota, All quotas under a table are equal

Sep 02 '24 09:09 majin1102

LGTM.

This change is necessary before we support more process types.

Sep 02 '24 09:09 zhoujinsong

Let's change 'Optimizing Status' column to 'Status' which use format as action.status( idle keeps ) like:

minor_optimizing.pending

major_optimizing.running

full_optimizing.commtting

expire-data.running

idle

Are these states string too long? @majin1102 @zhoujinsong @zhangmo8 WDYT?? Additionally, it is recommended to optimize the length of the Table column, as catalog.db.table does not display fully。

Sep 03 '24 03:09 Aireed

Can make a mouse hover to display all tooltips to view the completion name

Sep 03 '24 05:09 zhangmo8

@chouchouji

Sep 11 '24 02:09 zhangmo8

@chouchouji

I will complete it in recent days.

Sep 18 '24 01:09 chouchouji

Can make a mouse hover to display all tooltips to view the completion name

Agree with that

Sep 18 '24 11:09 majin1102

@majin1102 Hello, I am working on this issue about frontend section. I has completed some tasks. But there are some questions about backend and ui I want to ask for you.

[ ] How to display action? I set it multiple select in filter row
[ ] Where is the resource field more suitable? I put it between Quota and Occupation
[ ] The format of sortField, this field looks like Please tell me the format and correctness of key(duration, resource, quotaOccupation) and value(ascend, descend). Does it support multiple sorters?
[ ] I don't understand the fifth, Can you tell me how to do it?

Additionally, I add tooltip for table name. If other fields are also changed to this, I am pleased to do it.

Sep 18 '24 12:09 chouchouji

@majin1102 Hello, I am working on this issue about frontend section. I has completed some tasks. But there are some questions about backend and ui I want to ask for you.

[ ] How to display action? I set it multiple select in filter row

[ ] Where is the resource field more suitable? I put it between Quota and Occupation

[ ] The format of sortField, this field looks like Please tell me the format and correctness of key(duration, resource, quotaOccupation) and value(ascend, descend). Does it support multiple sorters?

[ ] I don't understand the fifth, Can you tell me how to do it?

Additionally, I add tooltip for table name. If other fields are also changed to this, I am pleased to do it.

Thanks for your contribution.

multiple select is good to me, I wonder how it would be like if we select more than the occupation of the box
I think the sort field is exclusive, so I believe there should be only one sort field and sort order?
the Resource position is good to me, but the columns are too many to display. we may need to discuss which column should keep for displaying more on table name.
The formation is like below. I suggest to pull this feature out as an individual improvement issue. Let's just focus the work you have done? :

Sep 20 '24 07:09 majin1102

I have updated the description and make it simpler, please take a look. @chouchouji

Sep 20 '24 09:09 majin1102

I have updated the description and make it simpler, please take a look. @chouchouji

Thanks for your reply!

it will be like this image
I set single sort in that pr.
I add scroll function, If it is too long, the table can be scrolled.

Sep 23 '24 09:09 chouchouji

I have updated the description and make it simpler, please take a look. @chouchouji

Thanks for your reply!

it will be like this image

I set single sort in that pr.

I add scroll function, If it is too long, the table can be scrolled.

Wonderful! I will check the PR later tomorrow

Sep 26 '24 14:09 majin1102

I have updated the description and make it simpler, please take a look. @chouchouji

Thanks for your reply!

it will be like this image

I set single sort in that pr.

I add scroll function, If it is too long, the table can be scrolled.

Wonderful! I will check the PR later tomorrow

OK! I will optimize code and prepare for it.

Sep 27 '24 03:09 chouchouji

@majin1102 I had resolved conflicts, please review it. https://github.com/apache/amoro/pull/3201

Oct 01 '24 09:10 chouchouji

@majin1102 I had resolved conflicts, please review it. #3201

Hi, chouchouji

Thanks for your contribution. I left some comments, and I have spiltted the original issue into more subtasks so that front and backend could better follow up(For now some backend work is somehow complicated to complete in one issue). I suggest you make this PR clean to this subtask: https://github.com/apache/amoro/issues/3234

And of course, the work you have done will not be wasted, we shall push sub-sequencial issues to catch up.

Oct 08 '24 06:10 majin1102

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

Apr 07 '25 00:04 github-actions[bot]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'

Apr 21 '25 00:04 github-actions[bot]