Update Admin Logs page to use Tabulator

[hoxbro]

Fixes #3623 Fixes #3435

The primary focus of this PR has been to replace a Terminal with Tabulator to be able to filter existing logs. This is done by subclassing Tabulator and adding a write function that takes the log message and streams it to the Tabulator data.

More filters have been added to work with the respective columns. Suggestions for alternative column names are welcome!

At last, an option for downloading the filtered log file has been added.

Example of the new log system:

(I have moved the logs tab to the front locally for easier access during development)

Code used

# Run with `panel serve tabulator_log.py --admin`

import panel as pn
import param

class Example(param.Parameterized):
    i = param.Number()

    @param.depends("i", watch=True)
    def test(self):
        return 42


e = Example()
pn.Column(e).servable()

https://user-images.githubusercontent.com/19758978/178973411-fc9921bb-4c65-4ada-b361-850a6ef91bf2.mp4

Problems

Three problems were discovered during the implementation, all somewhat related to streaming data:

1. A rediscover of #3435 of show_index=False with streaming data.
2. Calculation of max_page or self.page with empty data, which was fixed by always having them to be at least 1.
3. When streaming data and reloading the admin page, it does not show previous logs. This is solved by triggering the log_terminal.value in log_component(). I don't know if this is by design or a bug.

https://user-images.githubusercontent.com/19758978/178973852-81d932fd-b1f6-4a05-bb69-a04cff281e91.mp4

Notes:

• As it is (and was before this PR), the filters are synced between multiple users.
• I think that when having rollover enabled in streaming, all data will be kept in memory as the rollover here is a view and not a copy.

https://github.com/holoviz/panel/blob/9ca3a472ca0a590843e6e3d03959b56bcce5098c/panel/widgets/tables.py#L589-L590

• I haven't converted the datetime column to a datetime type as this gave some problems when showing the correct value in the Tabulator.
• I'm not sure if wrapping the write functionality in a try/except block is correct.

[codecov[bot]]

Codecov Report

Merging #3694 (797e22d) into master (e39e409) will increase coverage by 1.25%. The diff coverage is 21.31%.

@@            Coverage Diff             @@
##           master    #3694      +/-   ##
==========================================
+ Coverage   82.41%   83.66%   +1.25%     
==========================================
  Files         205      210       +5     
  Lines       27863    30245    +2382     
==========================================
+ Hits        22962    25304    +2342     
- Misses       4901     4941      +40     

Flag	Coverage Δ
ui-tests	33.19% <11.47%> (?)
unitexamples-tests	76.99% <21.31%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
panel/io/admin.py	0.00% <0.00%> (ø)
panel/widgets/tables.py	89.16% <100.00%> (+1.22%)	:arrow_up:
panel/auth.py	39.34% <0.00%> (-2.11%)	:arrow_down:
panel/viewable.py	66.77% <0.00%> (-0.65%)	:arrow_down:
panel/config.py	59.55% <0.00%> (-0.34%)	:arrow_down:
panel/tests/test_param.py	99.66% <0.00%> (-0.23%)	:arrow_down:
panel/io/state.py	68.25% <0.00%> (ø)
panel/io/pyodide.py	0.00% <0.00%> (ø)
panel/tests/test_depends.py	100.00% <0.00%> (ø)
... and 22 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

[philippjfr]

Had some minor comments, otherwise looks good to me! This will be much more useful than the terminal.

[philippjfr]

As it is (and was before this PR), the filters are synced between multiple users.

We should probably fix this separately later.

I think that when having rollover enabled in streaming, all data will be kept in memory as the rollover here is a view and not a copy.

Yes, that is intended.

I haven't converted the datetime column to a datetime type as this gave some problems when showing the correct value in the Tabulator.

Can you elaborate? Would definitely be nice.

[hoxbro]

Can you elaborate? Would definitely be nice.

The datetime will appear in the logging but only as a string.

An example of the behavior mentioned can be seen here:

https://user-images.githubusercontent.com/19758978/183616164-9eba7585-0eaa-48ca-8209-946fbe9bbfed.mp4

Also the .csv will not show the number:

"datetime","level","app","session","message"
"1660044574463","DEBUG","panel.viewable.Location","139986823043728","finished processing events {'pathname': '/3694_admin_tabulator_log', 'href': 'http://localhost:5006/3694_admin_tabulator_log', 'hostname': 'localhost', 'protocol': 'http:', 'port': '5006'}"
"1660044574455","DEBUG","panel.viewable.Location","139986823043728","received events {'pathname': '/3694_admin_tabulator_log', 'href': 'http://localhost:5006/3694_admin_tabulator_log', 'hostname': 'localhost', 'protocol': 'http:', 'port': '5006'}"
"1660044574388","INFO","panel.io.server","139986823043728","rendered"
"1660044574102","INFO","panel.io.server","139986823043728","created"
"1660044574008","INFO","panel.io.server","139986823043728","launching"

[philippjfr]

Thanks, my question was more about what issues you faced formatting the timestamp as a datetime properly?

[hoxbro]

I did not have any issue converting the string to datetime. The issues were with how Tabulator handled the datetime objects.

[hoxbro]

The picture here should be updated with the new interface.

The only way I can see how this can be done is by replacing it with a new URL, so the current site does not show the new interface.

[philippjfr]

The only way I can see how this can be done is by replacing it with a new URL, so the current site does not show the new interface.

The current site should continue to show it until we release the new version, all you have to do is update examples/assets/admin_logs.png.