Duplicate entries in cmp completion

[anandkumar89]

🐛 Describe the bug

I always find duplicate entires in autocomplete suggestion.

Config

require('obsidian').setup({
	workspaces = {
		{
			name = "Notes",
			path = "~/Library/Mobile Documents/iCloud~md~obsidian/Documents/Notes"
		},
	},
	templates = {
		subdir = "Meta/Templates"
	},
	daily_notes = {
		folder = "Meta/Daily Document",
		date_format = "%Y-%m-%d",
		template = "dailyNotes.md"
	},
	completion = {
		nvim_cmp = true,
		min_chars = 2,
	}

})

Environment

NVIM v0.9.5
Build type: Release
LuaJIT 2.1.1692716794

   system vimrc file: "$VIM/sysinit.vim"
  fall-back for $VIM: "/usr/local/share/nvim"

and

nvim --headless -c 'lua require("obsidian").info()' -c q
Obsidian.nvim v3.7.4 (68da7299505a74dcd0f45292c202197840615425)
Status:
   buffer directory: nil
   working directory: /Users/anand.kumar/Library/CloudStorage/Dropbox/Apps/Overleaf
Workspaces:
  ✓ active workspace: Workspace(name='Notes', path='/Users/anand.kumar/Library/Mobile Documents/iCloud~md~obsidia
n/Documents/Notes', root='/Users/anand.kumar/Library/Mobile Documents/iCloud~md~obsidian/Documents/Notes')
Dependencies:
    plenary.nvim: 4f71c0c4a196ceb656c824a70792f3df3ce6bb6d
    nvim-cmp: 04e0ca376d6abdbfc8b52180f8ea236cbfddf782
    telescope.nvim: dc1ea28cc2a02cd1cebb9d80e967807d668f78f8
Integrations:
   picker: TelescopePicker()
   completion: enabled (nvim-cmp)
Tools:
   rg: ripgrep 14.1.0
Environment:
  operating system: Darwin
Config:
  notes_subdir: nil
Killing job%          

[epwalsh]

Hey @anandkumar89, can you pull the latest commit and run :ObsidianDebug when you see the duplicates? Let me know what the output is. I'm guessing obsidian.nvim's cmp sources are getting registered multiple times for some reason. This will confirm that.

[anandkumar89]

https://github.com/epwalsh/obsidian.nvim/assets/12571250/6e1e8c7e-f979-4eb6-82eb-79bf9d61a1ef

Here's output to :ObsidianDebug

Obsidian.nvim v3.7.5 (f2eeb0a1e06f24e70bd36bf22eb45a7e21e2d63a)
Status:
  • buffer directory: /Users/anand.kumar/Library/Mobile Documents/iCloud~md~obsidian/Documents/Notes/Meta/Daily Document
  • working directory: /Users/anand.kumar/Library/CloudStorage/Dropbox/Apps/Overleaf
Workspaces:
  ✓ active workspace: Workspace(name='Notes', path='/Users/anand.kumar/Library/Mobile Documents/iCloud~md~obsidian/Documents/Notes', root='/Users/anand.kumar/Library/Mobile Documents/iCloud~md~obsidian/Documents/Notes')
Dependencies:
  ✓ plenary.nvim: 8aad4396840be7fc42896e3011751b7609ca4119
  ✓ nvim-cmp: 97dc716fc914c46577a4f254035ebef1aa72558a
  ✓ telescope.nvim: c2b8311dfacd08b3056b8f0249025d633a4e71a8
Integrations:
  ✓ picker: TelescopePicker()
  ✓ completion: enabled (nvim-cmp) ✓ refs, ✓ tags, ✓ new
    all sources:
      • obsidian
      • obsidian_new
      • obsidian_tags
      • buffer
      • cmp_zotcite
      • path
      • nvim_lsp
Tools:
  ✓ rg: ripgrep 14.1.0
Environment:
  • operating system: Darwin
Config:
  • notes_subdir: nil

newbie here. Can't make much if sources are registered multiple times (likely twice). And apologies for late response, I should get GitHub app for timely notifications.

[epwalsh]

Thanks @anandkumar89, the sources are only registered once, so I'm at a loss here. Could you post your nvim-cmp config as well?

[anandkumar89]

added little bit more context just in case other plugins could be messing around.

local vim = vim 
local Plug = vim.fn['plug#']

vim.call('plug#begin', '~/.config/nvim/plugged')
Plug 'junegunn/vim-easy-align'
Plug 'jeffkreeftmeijer/vim-numbertoggle'

Plug 'nvim-telescope/telescope.nvim'

-- LSP 
Plug 'neovim/nvim-lspconfig'
Plug 'VonHeikemen/lsp-zero.nvim'
Plug 'williamboman/mason.nvim'
Plug 'williamboman/mason-lspconfig.nvim'
Plug 'hrsh7th/nvim-cmp'
    Plug 'hrsh7th/cmp-buffer'
    Plug 'hrsh7th/cmp-path'    
    Plug 'hrsh7th/cmp-nvim-lsp'
    Plug 'onsails/lspkind.nvim'

-- markdown, obsidian specific plugins
Plug 'epwalsh/obsidian.nvim'
Plug 'jalvesaq/zotcite' 
 	Plug 'preservim/vim-markdown'
	Plug 'nvim-lua/plenary.nvim'
	Plug 'jalvesaq/cmp-zotcite'
vim.call('plug#end')


require('obsidian').setup({
	workspaces = {
		{
			name = "Notes",
			path = "~/Library/Mobile Documents/iCloud~md~obsidian/Documents/Notes"
		},
	},
	templates = {
		subdir = "Meta/Templates"
	},
	daily_notes = {
		folder = "Meta/Daily Document",
		date_format = "%Y-%m-%d",
		template = "dailyNotes.md"
	},
	completion = {
		nvim_cmp = true,
		min_chars = 2,
	}

})


local cmp = require('cmp')
cmp.setup {
	-- global config goes here
	sources = cmp.config.sources({
		{ name = 'path'},
		{ name = 'cmp_zotcite'},
		{ name = 'buffer' },
		{ name = 'nvim_lsp'},
	}),
	mapping = cmp.mapping.preset.insert({
	     	['<C-b>'] = cmp.mapping.scroll_docs(-4),
	     	['<C-f>'] = cmp.mapping.scroll_docs(4),
	     	['<C-Space>'] = cmp.mapping.complete(),
	     	['<CR>'] = cmp.mapping.confirm({ select = true }), -- Accept currently selected item. Set `select` to `false` to only confirm explicitly selected items.
	}),
	formatting = {
		format = lspkind.cmp_format({
				mode 		= 'symbol',
				preset 		= 'default',
				show_labelDetails = true,
				maxwidth	= 50,
				ellipsis_char= '..',
		})	
    },
}

[sebszyller]

Similar issue to OP; looks like the gist of it is that, in my case, the cmp suggests three things: the file path, the title of the note in lower case (in the alias format i.e. path-to-md.md|title), the title of the note in regular case (also in the alias format).

I figured that use_path_only would solve this but instead of having only one entry, it's just three entries with an identical float.

Also, it looks like the current behaviour is working as intended as per the README:

The desired behaviour would be to have only the first item instead of the first two.

This issue might me related to #579

[epwalsh]

Hey @sebszyller thanks for the additional info. I took another look and this to try to reduce duplicates, but was unsuccessful. The reason for this is that nvim-cmp handles the filtering and sorting based on the sortText field that we provide, which is often different from the newText field (which represents the actual text to be inserted). Obsidian.nvim provides cmp entries that are unique pairs of (sortText, newText), which leaves the possibility for duplicate newText items that you're seeing. If we were to only show suggestions with unique newText, the overall quality of completion would decrease substantially since the sortText provided for each entry might not rank the entry as high as it should be. Hence we err on the side of caution and provide nvim-cmp with all of the possible sortTexts that we can for a given newText.

Maybe we can request nvim-cmp to allow mulitple sortTexts for each entry. Otherwise to improve this we'd have to implement our own code to find the best sortText associated with each newText, but there's risk there that our algorithm doesn't match up with nvim-cmp's, which could lead to weird results.

[sebszyller]

Alright, thanks for investigating. Trying to upstream it to nvim-cmp seems challenging to say the least. Also, rumour has it that native cmp is coming to neovim soon, so might be a waste of cycles too.

Seems like a better idea to try to customise the behaviour in this plugin. Worth noting that the default behaviour in the obsidian app doesn't have any duplicate entries.

I can have a look at it on my end but could use some pointers around the codebase.

[sebszyller]

Right, so for my workflow, using use_path_only and changing the insertion condition: https://github.com/epwalsh/obsidian.nvim/blob/7507dee26f18610d5f928ff2b7e5fcb658074e52/lua/cmp_obsidian.lua#L244 to and gives me what I want (with a small tweak to the newText field).

Now the things is -- do you want to maintain any completion customisation? This could be gated behind a conf flag e.g. "reduce_cmp_duplicates" or sth more elegant if there're more use cases.

[epwalsh]

Hey @sebszyller so that is exactly what I tried as well (changing or to and). It seems that might be okay for your use-case but it did not work well with my own configuration.

I just had another idea for a solution though... stay tuned

[epwalsh]

Hey @sebszyller can you try out the branch in this PR to see if that works well for you? https://github.com/epwalsh/obsidian.nvim/pull/639

[sebszyller]

Yeah, with use_path_only it works fine. The entries are deduplicated. My only caveat is that use_path_only changes the format of newText to vault_name/file_name.md -- so in my patch, I'm adjusting this behaviour too s.t. newText is just file_name.md. But this is unrelated to this issue/PR.

[epwalsh]

My only caveat is that use_path_only changes the format of newText to vault_name/file_name.md

@sebszyller you can customize this by setting wiki_link_func / markdown_link_func to a custom function. E.g.:

  ---@param opts { path: string, label: string, id: string|integer|?, anchor: obsidian.note.HeaderAnchor|?, block: obsidian.note.Block|? }
  ---@return string 
  wiki_link_func = function(opts)
    ---@type string
    local header_or_block = ""
    if opts.anchor then
      header_or_block = opts.anchor.anchor
    elseif opts.block then
      header_or_block = string.format("#%s", opts.block.id)
    end
    return string.format("[[%s%s]]", vim.fs.basename(opts.path), header_or_block)
  end,