How to ignore the file-tree structure in the results with org-ql-select?

[maikol-solis]

Suppose you have this test.org file

#+TITLE: Test

#+begin_src emacs-lisp
(org-ql-select
  nil
  '(and (planning :from "2021-08-21" :to "2021-08-30")
        (todo))
  :action '(list  (org-entry-get (point) "TODO")
                  (concat "[["
                          (substring-no-properties (org-get-heading t t))
                          "]]")
                  (org-entry-get (point) "DEADLINE"))
  :sort '(deadline))
#+end_src

#+RESULTS:
| TODO | [[Task 1]] | <2021-08-23 Mon> |
| TODO | [[Task 2]] | <2021-08-27 Fri> |
| TODO | [[Task 3]] | <2021-08-24 Tue> |
| TODO | [[Task 4]] | <2021-08-23 Mon> |


* H1
** TODO Task 1
DEADLINE: <2021-08-23 Mon>
** TODO Task 2
DEADLINE: <2021-08-27 Fri>
* H2
** TODO Task 3
DEADLINE: <2021-08-24 Tue>
** TODO Task 4
DEADLINE: <2021-08-23 Mon>

According to the manual the results should be sorted by deadline, but in this case they are showing as appearance in the file. Notice how the deadline dates are completely mixed.

Did I make something wrong?

I'm in this commit 94f9e6f3031b32cf5e2149beca7074807235dcb0

Thanks for the help.

[alphapapa]

The :sort argument only works if you use element or element-with-markers as the :action argument. If you want to use the built-in sorting and change the output format, you should map over the elements returned by org-ql-select.

[maikol-solis]

Thanks for you help.

I forgot to say this example is based in this one https://github.com/alphapapa/org-ql/blob/master/examples.org#listing-bills-coming-due

Could you please give me a little example about how to change the :action part to make this work?

I read the docs, and It isn't clear to me how to interact with element or element-with-markers.

Thanks again.

[alphapapa]

Just use :action 'element-with-markers, and then see the results, which are org-element elements. You can use the org-element functions to retrieve their properties.

[maikol-solis]

Oh I see. element-with-markers gives the whole structure of the todos

#+begin_src emacs-lisp
(org-ql-select
 nil
   '(and (planning :from "2021-08-21" :to "2021-08-30")
        (todo))
  :action 'element-with-markers
  :sort '(date priority))
#+end_src

#+RESULTS:
| headline | (:raw-value Task 1 :begin 1321 :end 1335 :pre-blank 0 :contents-begin 1336 :contents-end 1336 :level 2 :priority nil :tags nil :todo-keyword TODO :todo-type todo :post-blank 1 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 1321 :deadline (timestamp (:type active :raw-value <2021-08-23 Mon>  :year-start 2021 :month-start 8 :day-start 23 :hour-start nil :minute-start nil :year-end 2021 :month-end 8 :day-end 23 :hour-end nil :minute-end nil :begin 1346 :end 1362 :post-blank 0)) :ID 20210821T164914 :title (Task 1) :org-marker #<marker at 1333 in test.org> :org-hd-marker #<marker at 1333 in test.org>) |
| headline | (:raw-value Task 4 :begin 1498 :end 1512 :pre-blank 0 :contents-begin 1513 :contents-end 1513 :level 2 :priority nil :tags nil :todo-keyword TODO :todo-type todo :post-blank 1 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 1498 :deadline (timestamp (:type active :raw-value <2021-08-23 Mon>  :year-start 2021 :month-start 8 :day-start 23 :hour-start nil :minute-start nil :year-end 2021 :month-end 8 :day-end 23 :hour-end nil :minute-end nil :begin 1523 :end 1539 :post-blank 0)) :title (Task 4) :org-marker #<marker at 1510 in test.org> :org-hd-marker #<marker at 1510 in test.org>)                     |
| headline | (:raw-value Task 3 :begin 1456 :end 1470 :pre-blank 0 :contents-begin 1471 :contents-end 1471 :level 2 :priority nil :tags nil :todo-keyword TODO :todo-type todo :post-blank 1 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 1456 :deadline (timestamp (:type active :raw-value <2021-08-24 Tue>  :year-start 2021 :month-start 8 :day-start 24 :hour-start nil :minute-start nil :year-end 2021 :month-end 8 :day-end 24 :hour-end nil :minute-end nil :begin 1481 :end 1497 :post-blank 0)) :title (Task 3) :org-marker #<marker at 1468 in test.org> :org-hd-marker #<marker at 1468 in test.org>)                     |
| headline | (:raw-value Task 2 :begin 1409 :end 1423 :pre-blank 0 :contents-begin 1424 :contents-end 1424 :level 2 :priority nil :tags nil :todo-keyword TODO :todo-type todo :post-blank 1 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 1409 :deadline (timestamp (:type active :raw-value <2021-08-27 Fri>  :year-start 2021 :month-start 8 :day-start 27 :hour-start nil :minute-start nil :year-end 2021 :month-end 8 :day-end 27 :hour-end nil :minute-end nil :begin 1434 :end 1450 :post-blank 0)) :title (Task 2) :org-marker #<marker at 1421 in test.org> :org-hd-marker #<marker at 1421 in test.org>)                     |

I assume that I can format this better? With another src block I guess?

[alphapapa]

The value returned by org-ql-select is not intended to be usefully printed as-is. You can post-process that value however you like, e.g. with mapcar.

[maikol-solis]

Thank you very much @alphapapa for your time. Now I understand a little more the logic with this function. I will create my own function to parse the output. I think it is safe to close the issue to reduce the noise in the bug tracker. Thanks again. Best.

[maikol-solis]

@alphapapa Hacking a little the example, and with the help of the amazing community in the doom's discord, I did this,

#+begin_src emacs-lisp
;;; Code:
(let ((tbl (org-ql-select
             nil
             '(and
               (todo))
             :action 'element-with-markers
             :sort '(date priority)
             )))
  (append '( (TODO P Task Deadline "\n|-\n") )
          (mapcar (lambda (x)
                    `(,(org-element-property :todo-keyword x)
                      ,(char-to-string (org-element-property :priority x))
                      ,(concat "[[id:" (org-element-property :ID x) "][" (org-element-property :raw-value x) "]]")
                      ,(ts-format "%a %d-%m" (ts-parse-org-element (org-element-property :deadline x)))))
                  tbl)))
#+end_src

#+RESULTS:
| TODO | P | Task   | Deadline  |
|------+---+--------+-----------|
| TODO | A | [[id:20210823T082131][Task 2]] | Fri 27-08 |
| TODO | B | [[id:20210823T082128][Task 1]] | Mon 23-08 |

* TODO [#B] Task 1
DEADLINE: <2021-08-23 Mon>
:PROPERTIES:
:id:       20210823T082128
:END:
* TODO [#A] Task 2
DEADLINE: <2021-08-27 Fri>
:PROPERTIES:
:ID:       20210823T082131
:END:

To people that doesn't know how to use the org-element API (like me) It could be extremely useful.

If you think it's relevant, you can include it in the examples.

Thanks

[alphapapa]

Yeah, something like that could be helpful. Thanks.

A few tips:

• Instead of using the string "\n|-\n" to insert the line, you can use the symbol hline, which Org converts to a line.
• Your end result is very close to an org-ql dynamic block: https://github.com/alphapapa/org-ql#dynamic-block I wonder if you didn't notice that in the docs. :)

[maikol-solis]

• Instead of using the string "\n|-\n" to insert the line, you can use the symbol hline, which Org converts to a line.

Ok perfect. I didn't know this existed. I can change the example accordingly

• Your end result is very close to an org-ql dynamic block: https://github.com/alphapapa/org-ql#dynamic-block I wonder if you didn't notice that in the docs. :)

YES! In fact, I use extensively the org-ql dynamic block, but I noticed that only works for the current buffer. I saw there is a couple of PRs to add a file or scope parameter to this. Meanwhile, I tried to see if I can replicate roughly the behavior of the dynamic block but using org-ql-select. My plan is to eventually replace the buffer parameter from nil to org-agenda-files and have all my todos gathered in only one place. I included the ID part to jump correctly to the headline, no matter where is located in my agenda-files.

It could be useful also to people using org-roam if they extend the search to their roam directory.

Thanks.

[alphapapa]

Oh, I see. :) Yes, I will get to that :file argument at some point, but it requires a little care, and there's a related PR that needs to be merged first. So many Emacs projects, so little time... :) Thanks.

[maikol-solis]

Yes sure, no problem. You've done an impressive work with this and other packages. Thanks again.