Two level agenda TODO hierarchy

[kyzyl]

I'm trying to get an agenda/super-agenda view which collects TODOs into blocks, one block per org-agenda-files file, and then within each block, display the TODOs, flattened by org outline path. For example, the following outline:

# file1.org
* PROJ Things to do
** TODO First
** TODO Second
** PROJ Sub-thing
*** TODO Third
*** TODO Fourth

# file2.org
* PROJ New Items
** TODO Fifth
** TODO Sixth

Should turn into something like:

file1
  Things to do:
    TODO First
    TODO Second 
    PROJ Sub-thing

  Things to do/Sub-thing:
    TODO Third
    TODO Fourth
-------------------------------------------
file2
  New Items:
    TODO Fifth
    TODO Sixth
------------------------------------------
... blocks from additional files ...
------------------------------------------

So far I haven't been able to figure out how to do this, and I'm not sure if it's because I don't know how things work, or it's not really possible. I can get the TODOs organized more or less as desired by adding the following org-ql-block call to org-agenda-custom-commands:

(org-ql-block '(and (category "file1") (todo) (ancestors (todo "PROJ")))
    ((org-ql-block-header "file1")
        (org-super-agenda-groups '((:auto-outline-path t)))))

However this requires adding new, hard-coded block for every file, removing code when files are deleted/renamed etc. It would be much better if there was a way to generate a block for every file currently in org-agenda-files. I also tried doing a stored view with org-ql-search, but I wasn't able to get the two level "per file/category, per project" hierarchy working correctly. If I got the hierarchy/indentation correct, items were always out of order and so it was not intelligible.

Any suggestions? Thanks!

[alphapapa]

However this requires adding new, hard-coded block for every file, removing code when files are deleted/renamed etc. It would be much better if there was a way to generate a block for every file currently in org-agenda-files.

Well, you could write a Lisp function to output the appropriate org-agenda-custom-commands/org-ql-block forms for a list of files. Should be as simple as (dolist (file (org-agenda-files)) ...), then splice that into org-agenda-custom-commands as appropriate.

I also tried doing a stored view with org-ql-search, but I wasn't able to get the two level "per file/category, per project" hierarchy working correctly. If I got the hierarchy/indentation correct, items were always out of order and so it was not intelligible.

Are you not using org-super-agenda to group the results? org-ql is designed to integrate with it.

Anyway, this sort of thing should be easier in the future when the branch demonstrated here is merged: https://github.com/alphapapa/org-ql/issues/331

[kyzyl]

Thanks. That was my original approach, but I couldn't get it to work. After reading your comment I decided to return to that approach and did get it working. I just needed to be more careful about quoting/backquoting of the org-ql-block forms such that everything evaluates correctly when spliced into the org-agenda-custom-commands.

I did also try piecing together an org-ql-views view with the :super-groups and whatnot required. However like I said, after trying many things, I wasn't able to get both of (a) the per-file, out-line grouped items, and (b) the out-line grouped items actually displayed in order. If I got (a) the items would always appear out of order, and if I got them to appear in-order with :auto-outline-groups, I could not also make them be segregated by file/category. Hence the 'two level hierarchy' in the question title here. If you have an idea of what a query + super-grouping would look like to do this I can try it out, I'm probably just being dense.

Two semi-related questions:

1. Another issue with the org-ql-views custom view was that it didn't seem to want to display the unicode glyphs I use, for example in my org-fancy-priorities-list. Is this intentional, or a known issue?
2. Is there a reason that org-ql-block i.e. org-ql-search-block does not take similar arguments to org-ql-search? For example, if I could specify the list of files to query, then I in my use-case above I wouldn't need to use a category selector in the query. This would prevent searching through all of org-agenda-files for each block, when I know up-front that I only want to query in a particular file.

[alphapapa]

Thanks. That was my original approach, but I couldn't get it to work. After reading your comment I decided to return to that approach and did get it working. I just needed to be more careful about quoting/backquoting of the org-ql-block forms such that everything evaluates correctly when spliced into the org-agenda-custom-commands.

Yes, the answer to problems like "this backquote/splice form isn't working" is usually to try again carefully until it works. :)

I did also try piecing together an org-ql-views view with the :super-groups and whatnot required. However like I said, after trying many things, I wasn't able to get both of (a) the per-file, out-line grouped items, and (b) the out-line grouped items actually displayed in order. If I got (a) the items would always appear out of order, and if I got them to appear in-order with :auto-outline-groups, I could not also make them be segregated by file/category. Hence the 'two level hierarchy' in the question title here. If you have an idea of what a query + super-grouping would look like to do this I can try it out, I'm probably just being dense.

The order changing is likely related to https://github.com/alphapapa/org-super-agenda/issues/207 and/or https://github.com/alphapapa/org-super-agenda/pull/242

Two semi-related questions:

1. Another issue with the org-ql-views custom view was that it didn't seem to want to display the unicode glyphs I use, for example in my `org-fancy-priorities-list`. Is this intentional, or a known issue?

I can't speak to how org-fancy stuff works, because I don't use it. You'll have to consult its maintainer.

2. Is there a reason that `org-ql-block` i.e. `org-ql-search-block` does not take similar arguments to `org-ql-search`? For example, if I could specify the list of files to query, then I in my use-case above I wouldn't need to use a category selector in the query. This would prevent searching through all of `org-agenda-files` for each block, when I know up-front that I only want to query in a particular file.

org-ql-block was designed to integrate with org-agenda. What you're suggesting would seem to mean simply using org-agenda buffers as a container for multiple, independent org-ql-based searches. That's an interesting idea, and I'd suggest looking at the various issues that already discuss changing the arguments to org-ql-block, as well as the aforementioned #331.

[gsingh93]

I'm not great with elisp and somewhat new to org-agenda, so figuring out the splicing part of this was tricky for me. I eventually figured it out, so to save the next person who needs it some time here's a basic example to get you started:

(let ((out nil))
  (dolist (file (org-agenda-files) out)
    (push
     `(org-ql-block '(todo "TODO")
                    ((org-agenda-files '(,file))
                     (org-ql-block-header ,file)))
     out))

    (setq org-agenda-custom-commands
          `(("c" "Org QL Block Agenda"
             (,@out)))))