slow-running queries

[iburunat]

When running helm-do-ag on a folder with many org files (~5k files, not code, but text), I am experiencing long retrieval times (in the order of 4 minutes).

Usually it retrieves one result very quickly. After 4-5 minutes I see the rest.

Is this behaviour normal? Thanks for your feedback!

[toothbrush]

Hey @iburunat, i just had the same issue. I went digging and for me the issue was that helm-ag actually will rewrite a query such as module Statsd into (?=.*module.*)(?=.*Statsd.*) (depending on some circumstances). I found that out by noticing my CPU being pegged during a search and having a look at what ps aux said my Emacs was actually executing. By poking around in the helm-ag source, i found that in some circumstances this rewrite gets done by helm-ag--join-patterns:

https://github.com/emacsorphanage/helm-ag/blob/c16d7a7e72a6834bf94832c2b1cad8f6c920108c/helm-ag.el#L989-L996

My first attempt was to just use Ripgrep, because that rewrite isn't done for Ripgrep as mentioned in the README:

https://github.com/emacsorphanage/helm-ag/blob/c16d7a7e72a6834bf94832c2b1cad8f6c920108c/README.md?plain=1#L279-L285

However i failed to get helm-ag correctly configured to use ripgrep (i guess that's a separate issue - rg works well for me on the CLI).

My next idea was to convince helm-ag to treat ag as if it has a "basic" RE2 capacity rather than PCRE with lookahead, which is extremely slow in some cases. I did that by looking at what helm-ag--set-command-features does and overriding the default helm-ag--command-features for Ag (which is '(pcre ag)) with '(re2 ag) to trigger a different code path in helm-ag--join-patterns. I also had to override the helm-ag--set-command-features function with a no-op so that it doesn't override my tweaks.

This is extremely hacky but in the absence of a knob to tweak helm-ag's behaviour, this seemed the easiest way to unblock myself. Hope it helps you! Here's my final init.el snippet:

(use-package helm-ag
  :config
  (setq helm-ag--command-features '(re2 ag))

  (defun helm-ag--set-command-features ()
    "Intentional no-op."))