Backend tool search always returning [] on main

[ahmedhamidawan]

Discovered this on main, the search has somehow broken recently.

Additional points (for panel search, just jotting down notes here):

• Do not show Discover Tools button in Workflow Editor Toolbox
• Have filtering.ts results prioritize (order) full word matches (for e.g.: Tool has word "seq" in description, if search === "seq" show it at top)

[ahmedhamidawan]

Something peculiar going on where

https://usegalaxy.org/api/tools?q=cut1

(the backend tool search on main) is always returning an empty array

It doesn't seem like a problem on the client because I ran a local dev server against main, and it's just the API that's only returning []

[ahmedhamidawan]

Possible problems here (or observations on what could have caused this):

@uwwint : The server creates whoosh indices on startup and writes them into the following directory: database/tool_search_index We need to check what size the indices are.

Getting [] on a query most likely means one of two things:

• The query has an error, it is caught and [] is returned
• OR the search indices are broken

[natefoo]

I don't see any errors associated with /api/tools?q= requests in the logs. Here's sizes of the indexes:

[ndc@galaxy-main1 ~]$ ls -lR /srv/galaxy/main/var/tool_search_index
/srv/galaxy/main/var/tool_search_index:
total 0
drwxr-xr-x. 2 g2main G-803372 25 Nov 20 02:55 all_tools
drwxr-xr-x. 2 g2main G-803372 25 Nov 20 02:55 default
drwxr-xr-x. 2 g2main G-803372 25 Nov 20 02:55 microgalaxy
drwxr-xr-x. 2 g2main G-803372 25 Nov 20 02:55 ontology:edam_operations
drwxr-xr-x. 2 g2main G-803372 25 Nov 20 02:55 ontology:edam_topics
drwxr-xr-x. 2 g2main G-803372 25 Nov 20 02:55 singlecell
drwxr-xr-x. 2 g2main G-803372 25 Nov 20 02:55 vgp

/srv/galaxy/main/var/tool_search_index/all_tools:
total 4
-rw-r--r--. 1 g2main G-803372 3809 Nov 20 02:55 _MAIN_0.toc

/srv/galaxy/main/var/tool_search_index/default:
total 4
-rw-r--r--. 1 g2main G-803372 3809 Nov 20 02:55 _MAIN_0.toc

/srv/galaxy/main/var/tool_search_index/microgalaxy:
total 4
-rw-r--r--. 1 g2main G-803372 3809 Nov 20 02:55 _MAIN_0.toc

'/srv/galaxy/main/var/tool_search_index/ontology:edam_operations':
total 4
-rw-r--r--. 1 g2main G-803372 3809 Nov 20 02:55 _MAIN_0.toc

'/srv/galaxy/main/var/tool_search_index/ontology:edam_topics':
total 4
-rw-r--r--. 1 g2main G-803372 3809 Nov 20 02:55 _MAIN_0.toc

/srv/galaxy/main/var/tool_search_index/singlecell:
total 4
-rw-r--r--. 1 g2main G-803372 3809 Nov 20 02:55 _MAIN_0.toc

/srv/galaxy/main/var/tool_search_index/vgp:
total 4
-rw-r--r--. 1 g2main G-803372 3809 Nov 20 02:55 _MAIN_0.toc

[jdavcs]

Those are not indexes (also ref. discussion on admin slack). Here's what's in the default index directory on test:

[ic4f@galaxy-test ~]$ ll /srv/galaxy/test/var/tool_search_index/default/
total 17048
-rw-r--r--. 1 g2test G-803372  2897788 Mar 26  2025 MAIN_4hkjub3824vjl57w.seg
-rw-r--r--. 1 g2test G-803372    64046 Mar 26  2025 MAIN_6khezvssbmgdw75b.seg
-rw-r--r--. 1 g2test G-803372 14480193 Nov 27  2024 MAIN_92yw762xxh36yp9w.seg
-rwxr-xr-x. 1 g2test G-803372        0 Nov 27  2024 MAIN_WRITELOCK
-rw-r--r--. 1 g2test G-803372     5202 Mar 26  2025 _MAIN_3.toc

[ahmedhamidawan]

Fixed in https://github.com/galaxyproject/galaxy/pull/21426 and other deployment fixes by @natefoo, @mvdbeek and @jdavcs