Unexpected behaviour from evil-org-open-below inside org code blocks

[IsaacDempsey]

Description :octocat:

Unexpected behaviour from evil-org-open-below inside org code blocks.

Reproduction guide :beetle:

• Start Emacs
• Open org file.
• Create code block.
• Write a sentence inside the code block.
• Place cursor anywhere in that sentence.
• Run evil-org-open-below

Observed behaviour: :eyes: :broken_heart:

• HELM Completion-at-point menu appears.
• When an item is selected from the menu it gets copied into the new line.
• All text in the code block is indented.

Expected behaviour: :heart: :smile:

• A new line is created below the current line.
• The cursor is moved to this new line.

System Info :computer:

• OS: darwin
• Emacs: 26.3
• Spacemacs: 0.300.0
• Spacemacs branch: develop (rev. c3f13d039)
• Graphic display: t
• Distribution: spacemacs
• Editing style: vim
• Completion: helm
• Layers:

(emacs-lisp git helm markdown multiple-cursors
						(org :variables org-enable-org-journal-support t)
						syntax-checking treemacs auto-completion csv html yaml javascript react python ruby go deft spell-checking)

• System configuration features: JPEG RSVG IMAGEMAGICK GLIB NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES THREADS LCMS2

[k-jell]

I experience the same problem. It seems like this only happens on develop and not on master though.

[duianto]

Confirmed

With the cursor in a bash, org src block

  #+begin_src bash
    This is a long line of text.
  #+end_src

Pressing either:

• o (which calls: evil-org-open-below)
• O (which calls: evil-org-open-above)

opens: HELM Completion At Point

The messages buffer shows:

Setting up indent for shell type sh Indentation variables are now local. Indentation setup for shell type sh

But with a emacs-lisp, org src block

  #+begin_src emacs-lisp
    This is a long line of text.
  #+end_src

o opens a newline below as expected, while: O opens HELM Completion At Point

master branch

It doesn't happen on the master branch, because Spacemacs uses a local version of evil-org.el, which binds the keys like this: https://github.com/syl20bnr/spacemacs/blob/c7a103a772d808101d7635ec10f292ab9202d9ee/layers/%2Bemacs/org/local/evil-org/evil-org.el#L136-L137 o calls the anonymous function, but O is rebound in the org/init-evil-org function (which is defined after the local bindings) to call: evil-open-above https://github.com/syl20bnr/spacemacs/blob/d46eacd83842815b24afcb2e1fee5c80c38187c5/layers/%2Bemacs/org/packages.el#L55-L56

Pressing o or O in a python, org src block, opens a newline as expected.

And the messages buffer shows:

Can’t guess python-indent-offset, using defaults: 4

Windows 1903

#### System Info :computer:
- OS: windows-nt
- Emacs: 26.3
- Spacemacs: 0.300.0
- Spacemacs branch: develop (rev. 1f6e39ea8)
- Graphic display: t
- Distribution: spacemacs
- Editing style: vim
- Completion: helm
- Layers:
```elisp
(autohotkey auto-completion command-log dap emacs-lisp git helm html imenu-list javascript
            (markdown :variables markdown-live-preview-engine 'vmd markdown-command "vmd")
            multiple-cursors nim
            (org :variables org-agenda-files
                 '("~/org/notes.org"))
            (shell :variables shell-default-shell 'shell shell-default-height 30 shell-default-position 'bottom)
            spell-checking
            (syntax-checking :variables syntax-checking-enable-by-default nil)
            treemacs version-control)
```
- System configuration features: XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS THREADS LCMS2

[duianto]

This seems to be what's causing the HELM Completion At Point buffer to open: https://github.com/syl20bnr/spacemacs/blob/1f6e39ea8ab3e7b7cb0a89889dec307523dfc6b4/layers/%2Bemacs/org/packages.el#L130

The variable can be disabled from the dotspacemacs-configuration-layers section near the top of your .spacemacs:

(org :variables org-src-tab-acts-natively nil)

Or from the dotspacemacs/user-config section, like this:

(setq org-src-tab-acts-natively nil)

Then restart Spacemacs.

---

org-src-tab-acts-natively was enabled by default in this commit: org: enable org-src-tab-acts-natively https://github.com/syl20bnr/spacemacs/commit/b09fe8553d853555f5c54a6797fb3911a07866d0

[IsaacDempsey]

Thank you so much @duianto - issue is fixed!

[alexalekseyenko]

@duianto Is this the behavior we want/expect? It seems odd that opening a new line should trigger completion, particularly when there isn't any text to complete.

[lebensterben]

Well, I'm not able to find a way to determine whether the cursor is within a block. (org-at-block-p returns non-nil only on the line of #+BEGIN_SRC)

Otherwise I can fix it.

[duianto]

It's unexpected.

It seems to be caused by some setting or package in Spacemacs, because there are no completion suggestions with Emacs, org-plus-contrib and evil-org.

o and O just creates a newline.

---

It doesn't seem to be caused by the variable: org-src-tab-acts-natively because it's been changed upstream, to t by default in org 9.4.

org-src-tab-acts-natively is a variable defined in ‘org-src.el’. Its value is t

You can customize this variable.

This variable was introduced, or its default value was changed, in version 9.4 of the Org package that is part of Emacs 27.2.

Documentation: If non-nil, the effect of TAB in a code block is as if it were issued in the language major mode buffer.

---

It not helm specific, because a similar thing happens with ivy.

o shows "completions" in the minibuffer:

O (shift o) shows them in the src block:

[alexalekseyenko]

It looks like org.el contains the following logic:

 ((and (eq type 'src-block)
		  org-src-tab-acts-natively
		  (> (line-beginning-position)
		     (org-element-property :post-affiliated element))
		  (< (line-beginning-position)
		     (org-with-point-at (org-element-property :end element)
		       (skip-chars-backward " \t\n")
		       (line-beginning-position))))
	     (org-babel-do-key-sequence-in-edit-buffer (kbd "TAB")))

I'm still new to this, so it's not entirely clear to me why

(org-babel-do-key-sequence-in-edit-buffer (kbd "TAB"))

is the most sensible thing to do in this situation, but given that that's how it's written, your suggestion to disable org-src-tab-acts-natively seems like our best bet.

The completion behavior is likely related to https://github.com/syl20bnr/spacemacs/issues/4478. It seems unexpected given that company-minimum-prefix-length is non-zero.

[lebensterben]

o -> evil-org-open-below, every line of it are fine except the last two:

(when evil-auto-indent
  (indent-according-to-mode))

This opens completion-at-point in my setup.

indent-according-to-mode also looks fine except the very last form

(funcall indent-line-function)

In org-mode, it's set to org-indent-line, which does trigger completion.

As pointed out previously, when org-src-tab-acts-natively is nil, the auto-completion won't be triggered.

---

		  (> (line-beginning-position)
		     (org-element-property :post-affiliated element))

This returns true if the cursor is after the beginning of src-block.

		  (< (line-beginning-position)
		     (org-with-point-at (org-element-property :end element)
		       (skip-chars-backward " \t\n")
		       (line-beginning-position))))

This returns true when the cursor is before the last character of src-block

So in other word, as long as the cursor is within the source block and the org-src-tab-acts-natively is turned on, it will execute

(org-babel-do-key-sequence-in-edit-buffer (kbd "TAB"))

That is trigger the binding of TAB in the correspondent major mode.

[lebensterben]

One way to circumvent this is to bind o to evil-insert-newline-below.

A even better solution is, use the method similar to how org-indent-line determine whether it's within a src-block, when it is, call evil-insert-newline-below, and when it's not, call evil-org-open-below.

[duianto]

The reason why it isn't happening without Spacemacs seems to be because, TAB is bound to different commands in the org-edit-src-code buffer:

• In Emacs, org-plus-contrib, evil-org: evil-jump-forward
• In Spacemacs: indent-for-tab-command

This can be seen by evaluating (org-edit-src-code) from the Eval: prompt (S-M-;), in the org bash src block. Now a *Org Src file.org[ bash]* buffer opens, and pressing TAB shows:

• In Emacs, org-plus-contrib and evil-org

TAB (translated from ) runs the command evil-jump-forward (found in evil-motion-state-map), which is an interactive compiled Lisp function in ‘evil-commands.el’.

It is bound to TAB.

(evil-jump-forward &optional COUNT)

Go to newer position in jump list. To go the other way, press C-o.

• In Spacemacs (where completion-at-point is called)

TAB (translated from ) runs the command indent-for-tab-command (found in global-map), which is an interactive Lisp function in ‘indent.el’.

It is bound to TAB.

(indent-for-tab-command &optional ARG)

Probably introduced at or before Emacs version 24.1.

Indent the current line or region, or insert a tab, as appropriate. This function either inserts a tab, or indents the current line, or performs symbol completion, depending on ‘tab-always-indent’. The function called to actually indent the line or insert a tab is given by the variable ‘indent-line-function’.

If a prefix argument is given, after this function indents the current line or inserts a tab, it also rigidly indents the entire balanced expression which starts at the beginning of the current line, to reflect the current line’s indentation.

In most major modes, if point was in the current line’s indentation, it is moved to the first non-whitespace character after indenting; otherwise it stays at the same position relative to the text.

If ‘transient-mark-mode’ is turned on and the region is active, this function instead calls ‘indent-region’. In this case, any prefix argument is ignored.

---

The org-src-tab-acts-natively variable had the default value t in both tests.

org-src-tab-acts-natively is a variable defined in ‘org-src.el’. Its value is t

[lebensterben]

Well, I think we need to fix this on spacemacs's end because org/evil-org are working perfectly fine.

That's I propose to bind o to evil-insert-newline-below when it's within src block.

[alexalekseyenko]

Assuming that the specified behavior, (kbd "TAB"), is correct, then the triggering of the completion is the issue, right? If so, would it make sense to reinvestigate/reopen either or both of the following? https://github.com/syl20bnr/spacemacs/issues/4478 https://github.com/syl20bnr/spacemacs/issues/8222

[lebensterben]

Agree

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Please let us know if this issue is still valid!

[fran-penedo]

I've looked into this and I'm quite confused as to how plain org mode plus evil deal with this. evil-jump-forward doesn't change indentation, so it would have to be set manually for each new line?

The issue here is indent.el doesn't split indenting and completion, so there's no way to call one without the other. I'm not sure how feasible it would be to provide our own without

       ;; If the text was already indented right, try completion.
       ((and (eq tab-always-indent 'complete)
             (eq old-point (point))
             (eq old-tick (buffer-chars-modified-tick)))
        (completion-at-point))

as well as changing org-indent-line to call it instead of sending TAB. In the meantime, since I don't use completions for empty strings I've changed sh-mode's completion function:

  (defun my-sh-completion-at-point-function ()
    (save-excursion
      (skip-chars-forward "[:alnum:]_")
      (let ((end (point))
            (_ (skip-chars-backward "[:alnum:]_"))
            (start (point)))
        (cond
         ((eq (char-before) ?$)
          (list start end (sh--vars-before-point)))
         ; don't provide completions on empty string
         ((or (not (char-after)) (eq (char-after) $\n)) nil)
         ((sh-smie--keyword-p)
          (list start end #'sh--cmd-completion-table))))))

  (add-hook 'sh-mode-hook
            #'(lambda ()
                (remove-hook 'completion-at-point-functions #'sh-completion-at-point-function t)
                (add-hook 'completion-at-point-functions #'my-sh-completion-at-point-function nil t)))

[lebensterben]

@fran-penedo

I think you're talking about a different issue here...

[fran-penedo]

@lebensterben What? Opening new lines in shell source code blocks in org-mode trigger completion at point, what part of my comment doesn't talk about this?

[lebensterben]

@fran-penedo

Because I think now the issue is evil-org-open-below

https://github.com/syl20bnr/spacemacs/issues/13465#issuecomment-741712718

[fran-penedo]

@lebensterben Yes, which at some point ends up calling indent-line-function, which is set to org-indent-line, which delegates indentation logic to the underlying major mode via indent-for-tab-command, which overloads TAB to indent when it needs to indent and autocomplete when it doesn't. Thus my comment.

From what I can understand, all those steps are needed and make sense except hardcoding the overloading of TAB (which is done in emacs source code). Since we're unlikely to get that to change, I'm not sure how we could fix this in spacemacs. Pure org mode plus evil seem to bypass it by using evil-insert-newline-below and not indenting? In my previous comment I proposed two workarounds, but they're far from ideal: redefining the completion hooks for each mode that autocompletes empty strings to disable it; or redefining indent-for-tab-command to only indent and use it instead of calling TAB in org-indent-line. Perhaps we can come up with a better solution?

[behaghel]

I don't know if it's useful comment but what's happening is that the sending of TAB in the org-src-edit buffer is hitting the normal mode instead of the insert mode of evil. That's what's creating the discrepancy in mapping as highlighted by @duianto here: https://github.com/syl20bnr/spacemacs/issues/13465#issuecomment-741840311

I couldn't find a better way than this to work around it:

(evil-set-initial-state 'prog-mode 'insert)

I wish there would be something less radical.

[PatrickCPE]

Some default hook was overriding setq org-src-tab-acts-natively to 1 when I loaded org-mode every time.

The following line in your .spacemacs user-config() will fix this for anyone who encounters it: (add-hook 'org-mode-hook (lambda () (setq org-src-tab-acts-natively nil)))