Telescope will not open files with unicode characters in their name

[GiorgosAthanasopoulos]

Description

When i try to use telescope in projects that contain filenames that are unicode (for example they may use emojis or greek letters in my case) the search and select functionality dont work correctly. In the search function the name will not be the actual representation but instead it replaces unicode characters with escape sequences (e.g. instead of 😂 it will be \xxx). When i try to select the file anyway it will open a new file with the filename being the original filename if instead we replace the unicode character with their escaope sequence, So for a file named 😂.lua we will get \xxx.lua where xxx is the escape sequence/code for 😂.

(NOTE: both nvim and my terminal are configured to use nerd fonts, so there shouldn't be a problem with that. I can also say that this is a telescope problem because prime also encountered it in his review of 😂lang which used emojis in filenames).

Neovim version

NVIM v0.9.5
Build type: Release
LuaJIT 2.1.1702233742

Operating system and version

gnu+linux (kernel version 6.7.1-zen1-1-zen)

Telescope version / branch / rev

"telescope.nvim": { "branch": "master", "commit": "2f3857c25bbd00ed7ac593c9d4071906369e4d20" },

checkhealth telescope

==============================================================================
telescope: require("telescope.health").check()

Checking for required plugins ~
- OK plenary installed.
- OK nvim-treesitter installed.

Checking external dependencies ~
- OK rg: found ripgrep 14.1.0
- OK fd: found fd 9.0.0

===== Installed extensions ===== ~

Telescope Extension: `fzf` ~
- OK lib working as expected
- OK file_sorter correctly configured
- OK generic_sorter correctly configured

Steps to reproduce

1. Open telescope for git/dir files.
2. Search for a file with unicode characters in it. It will not show the actual character (emoji for example or greek letter) but an escape sequence (e.g. \300)
3. Open/select the file.

Expected behavior

Expected the correct file to open with its contents.

Actual behavior

A new empty buffer is opened with the name being the original but instead of unicode characters u get the escape codes. For example: Lets say that the 😂 emoji is number \300. If i have a file named 😂.lua It will instead open 300.lua, which does not exist and its empty.

Minimal config

vim.cmd [[set runtimepath=$VIMRUNTIME]]
vim.cmd [[set packpath=/tmp/nvim/site]]
local package_root = '/tmp/nvim/site/pack'
local install_path = package_root .. '/packer/start/packer.nvim'
local function load_plugins()
  require('packer').startup {
    {
      'wbthomason/packer.nvim',
      {
        'nvim-telescope/telescope.nvim',
        requires = {
          'nvim-lua/plenary.nvim',
          { 'nvim-telescope/telescope-fzf-native.nvim', run = 'make' },
        },
      },
      -- ADD PLUGINS THAT ARE _NECESSARY_ FOR REPRODUCING THE ISSUE
    },
    config = {
      package_root = package_root,
      compile_path = install_path .. '/plugin/packer_compiled.lua',
      display = { non_interactive = true },
    },
  }
end
_G.load_config = function()
  require('telescope').setup()
  require('telescope').load_extension('fzf')
  -- ADD INIT.LUA SETTINGS THAT ARE _NECESSARY_ FOR REPRODUCING THE ISSUE
end
if vim.fn.isdirectory(install_path) == 0 then
  print("Installing Telescope and dependencies.")
  vim.fn.system { 'git', 'clone', '--depth=1', 'https://github.com/wbthomason/packer.nvim', install_path }
end
load_plugins()
require('packer').sync()
vim.cmd [[autocmd User PackerComplete ++once echo "Ready!" | lua load_config()]]

[jamestrew]

If you're talking about :Telescope git_files, this is the default git behavior.

You can set git config core.quotepath false to change this behavior.

But looking more deeply, I think we can actually clean up Telescope's usage of git ls-files as well. I'll try to work something together.

Aside: :Telescope find_files (which uses rg or fd) seems to work fine.