[Feature] adding advanced search

[Prahitha]

Filtering by log content and source code. So I was wondering if it'd be a good idea to add the information in a CSV/JSON. The CSV would have 4 columns - tool, test, output/log, source code. The search query can be entered in the search box and will be used to search within the CSV using JS

[hzeller]

Searching the test result CSV with expressions makes a lot of sense (and is probably much easier than parsing the HTML table). So there we can have expressions such as tool = "UhdmYosys" and ExitCode = 134 and quickly filter them.

Having logs or source code in a CSV might be pretty nasty as CSV is mostly meant for single item things with little spaces and certainly no newlines - otherwise a lot of (non-standardized) escaping is needed. I'd probably look if there is a format available more suited to directly ingest with JS.

The logs and source code is huge (many megabytes), so searching there without indexing might be pretty slow in JS, so this might be the time for some server-side solution, though that is not easy as right now we just have static content serving from github. I'd look into some indexing solution that can preprocess large amount of texts at the time we generate it and then have the JS load that index and provide search there. I don't know what is the fulltext search of choice these days.

Having some proof-of-concept to test if the amount of data is usable is probably good.

It might be good to consider the use case in question: what are we looking for, and maybe the task gets easier. I could imagine searching for things to be useful on a particular file itself, say we look at the logs and we want to just show all the lines with a particular word, e.g. someerror. If on top of the log file 'subwindow' (on the bottom left), there was a search box, that allows me to search-as-I-type narrow down the lines that are matched, I think that can be very useful. For the source code ... maybe, but probably less useful.

Somewhat unrelated to the original issue, but I think important for all these things: it should be possible to create a link out of it: with all search criteria added for the general table or the log files, this should result in a link that I can copy/paste e.g. in a mail or bug so that I can point other people directly to my finding. After all, we do the searching and narrowing down of the output for the very reason that we can look at it in a team and work on finding solutions to problems ("Check out how Verible is crashing with this message"). So most important - create a link that allows to re-create the resulting page for everyone.

[Prahitha]

Thank you for taking the time to look into this and giving a detailed response. Would ElasticLunr be a better choice?