Improve server startup by preloading only those projects that need metrics showing on the homepage

[dmeehan1968]

For large Junie logs, the load time is increasingly getting longer. The principle reasons for preloading are:

1. Metrics so that projects costs over time can be included in the homepage chart
2. Loading of issues and tasks for quick navigation between pages

Given that there are issues with Worker usage in Bun, which is the simple option to speed preload times, it might be better to avoid preloading until navigation. This would require some kind of suspense/loading UI to cover the time take when drilling in.

Projects, issues, tasks and events are already lazy loaded, and the preload was principally designed around compiling metrics, but this also has the side effect of preloading the entire tree (although events are dumped if the only reason was for metrics).

Either another 'preload' checkbox could be used to determine whether to preload, or the existing checkbox to include the project in the cost chart could be used. These values are currently saved to cookies, so it would be necessary to store the selected projects in a file somewhere on the users system, such as the Junie logs directory. This could be a simple hidden dot file, containing a JSON structure listing the selected projects. This does make the project selection 'server side' rather than 'client side' but its essentially a single user app due to the locality of the Junie logs.

Of course its also always possible to defer any loading until handling a page request, and the server caches what it has lazy loaded so performance improves from that point forward. The 'hasMetrics' logic would need to be reworked, because without a preload, these would appear to always be false.

[dmeehan1968]

There are some additional problems with this. In the homepage/projects table, we should the last updated, issue count and LLM's for each project.

• last updated comes from the issue logs, and is the creation date of an issue, not the timestamp of the most recent task/event. Most people wouldn't notice this discrepancy, because I don't think many people would come out of an issue, do other works in the same project, and then issue a follow up task, but that workflow would skew the accuracy of the last updated. Reading the issues on page load wouldn't be a massive overhead, but reading the tasks/events to get a more accurate last updated would likely require a full preload as existing.
• issue count requires loading the issue logs, but this is relatively quick even across many projects. The downside at the moment is that the homepage is SSR, so nothing can be sent until the issues have been loaded, so the first load of the page would have a lag.
• LLMs come from the event logs so would require a full preload to be accurate

The homepage could be enhanced to render the table without data in SSR, and then rely on XHR to fetch the projects and populate once the data is available. If the preload only went as far as loading the projects and issues, this would be a minor delay, covered by a loading state, and projects could still be ordered by name/last updated according to client settings. We could then initiate an XHR to obtain the LLM's for each project which would be fulfilled after a full loading of tasks/events.

Whilst all of this is possible in the existing codebase, its probably beyond scope. We really need a better client side approach instead of the existing vanilla JS, perhaps using React or HTMX.