cylc-uiserver
cylc-uiserver copied to clipboard
cat-log: support large files
Currently we cap the log at 5000 lines.
This is to protect the browser, the GUI could have multiple views (incl logs) open simultaneously. Long log files cause issues with Cylc Review (which also hits this problem). Past a hard-coded limit Cylc Review will only offer a "raw" file view which exists outside of the regular UI and scales better.
Two options for the Cylc log view:
- Paginate the log files.
- We can load the logs in X line chunks.
- E.G. to start reading from line 5000 we could do
tail -n +5000
. - This would involve adding an "offset" argument to the log subscription.
- Ideally we would be able to query the number of lines in the file so the GUI could display this, however, I'm not sure we can do that without reading the file?
- Offer the ability to open long log files in another browser tab ala Cylc Review.
- This would mean building a light-weight web app to display the log lines in a simple
<code>
block. - This app would need to perform the same authentication as the regular UI (a couple of lines of code which bung the token into a cookie).
- It would make the problem easier if we added a REST endpoint for accessing the job logs just for this view (no need for a GraphQL client in the new app).
- This would mean building a light-weight web app to display the log lines in a simple
Pull requests welcome!