UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9e in position 127: invalid start byte

[putraadr]

I'm getting an error running this program. reading from internet they say it's a problem with my locale or language. i already try installing a language pack, update-locale etc.. I'm stuck how can i fix this?

putraadr@ptrsoft:~/files/efibootmgr-gui$ sudo python3 efibootmgr_gui.py 
DEBUG:root:Running: findmnt --noheadings --output SOURCE,FSTYPE --mountpoint /efi
DEBUG:root:Running: findmnt --noheadings --output SOURCE,FSTYPE --mountpoint /boot/efi
DEBUG:root:Device path /dev/sda1 split into /dev/sda and 1
INFO:root:Detected ESP on disk /dev/sda part 1
Traceback (most recent call last):
  File "efibootmgr_gui.py", line 503, in <module>
    main()
  File "efibootmgr_gui.py", line 496, in main
    win = EFIWindow(f"--disk {disk} --part {part}")
  File "efibootmgr_gui.py", line 429, in __init__
    self.store.refresh()
  File "efibootmgr_gui.py", line 259, in refresh
    boot = run_efibootmgr()
  File "efibootmgr_gui.py", line 17, in run_efibootmgr
    output = subprocess.run(["efibootmgr", "-v"], check=True, capture_output=True, text=True).stdout.strip().split('\n')
  File "/usr/lib/python3.8/subprocess.py", line 491, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
  File "/usr/lib/python3.8/subprocess.py", line 1024, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "/usr/lib/python3.8/subprocess.py", line 1904, in _communicate
    stdout = self._translate_newlines(stdout,
  File "/usr/lib/python3.8/subprocess.py", line 901, in _translate_newlines
    data = data.decode(encoding, errors)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9e in position 127: invalid start byte

[Elinvention]

Could you run these two commands? python -c 'import locale; print(locale.getpreferredencoding(False))' and python -c 'import locale; print(locale.getpreferredencoding(True))'

[putraadr]

I run the command and it returns nothing

[Elinvention]

What about cat /etc/lsb-release /etc/locale.conf, python --version?

[putraadr]

cat /etc/lsb-release /etc/locale.conf :

DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.2 LTS"
cat: /etc/locale.conf: Tidak ada berkas atau direktori seperti itu

python --version :

Python 2.7.18

[Elinvention]

Your distribution (Ubuntu 20.04) is still using python 2 as a default... try again with python 3, like this python3 --version, python3 -c 'import locale; print(locale.getpreferredencoding(False))', python3 -c 'import locale; print(locale.getpreferredencoding(True))'. Also it looks like you are missing /etc/locale.conf, but that could be normal on Ubuntu. Try following this guide, maybe it solves your issue.

[putraadr]

Hello. Sorry for replying late.. I'm frustated with so many error in many projects and decided try to live with f12 boot menu everytime i power up my computer. I cant get this working to this day...

python3 -c 'import locale; print(locale.getpreferredencoding(True))' showing : UTF-8 python3 -c 'import locale; print(locale.getpreferredencoding(False))' showing : UTF-8 python3 --version showing : Python 3.8.10

I did try the guide you mention and it makes no effect. I really dont have any idea what causes this problem 😭

[Elinvention]

What if you run env LANG=C python3 efibootmgr_gui.py? It should run fine. The thing is that your system is setup to output non UTF-8 encoded text. Maybe if you post the output of locale we can get why.

[putraadr]

it is now 0x95...

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x95 in position 142: invalid start byte

[Elinvention]

I'm clueless... according to python your system uses an UTF-8 encoded locale, then it fails... :thinking: Still, I'd like to know what the locale command tells you.

[Elinvention]

Also you don't need to use sudo to run this program.

[pavkamlc]

I have the same problem on updated ARCH:

DEBUG:root:Running: findmnt --noheadings --output SOURCE,FSTYPE --mountpoint /efi DEBUG:root:Running: findmnt --noheadings --output SOURCE,FSTYPE --mountpoint /boot/efi DEBUG:root:Running: findmnt --noheadings --output SOURCE,FSTYPE --mountpoint /boot DEBUG:root:Device path /dev/sda1 split into /dev/sda and 1 INFO:root:Detected ESP on disk /dev/sda part 1 Traceback (most recent call last): File "/usr/bin/efibootmgr-gui", line 503, in main() File "/usr/bin/efibootmgr-gui", line 496, in main win = EFIWindow(f"--disk {disk} --part {part}") File "/usr/bin/efibootmgr-gui", line 429, in init self.store.refresh() File "/usr/bin/efibootmgr-gui", line 259, in refresh boot = run_efibootmgr() File "/usr/bin/efibootmgr-gui", line 17, in run_efibootmgr output = subprocess.run(["efibootmgr", "-v"], check=True, capture_output=True, text=True).stdout.strip().split('\n') File "/usr/lib/python3.9/subprocess.py", line 507, in run stdout, stderr = process.communicate(input, timeout=timeout) File "/usr/lib/python3.9/subprocess.py", line 1134, in communicate stdout, stderr = self._communicate(input, endtime, timeout) File "/usr/lib/python3.9/subprocess.py", line 2017, in _communicate stdout = self._translate_newlines(stdout, File "/usr/lib/python3.9/subprocess.py", line 1011, in _translate_newlines data = data.decode(encoding, errors) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 102: invalid start byte

[Elinvention]

@pavkamlc please run locale and report back.

[pavkamlc]

@pavkamlc please run locale and report back.

My locales is cs_CZ.UTF-8, but when I switch on terminal to LANG=C, the result is the same...

[pavel_mlcoch@notebook ~]$ locale LANG=cs_CZ.UTF-8 LC_CTYPE="cs_CZ.UTF-8" LC_NUMERIC="cs_CZ.UTF-8" LC_TIME="cs_CZ.UTF-8" LC_COLLATE="cs_CZ.UTF-8" LC_MONETARY="cs_CZ.UTF-8" LC_MESSAGES="cs_CZ.UTF-8" LC_PAPER="cs_CZ.UTF-8" LC_NAME="cs_CZ.UTF-8" LC_ADDRESS="cs_CZ.UTF-8" LC_TELEPHONE="cs_CZ.UTF-8" LC_MEASUREMENT="cs_CZ.UTF-8" LC_IDENTIFICATION="cs_CZ.UTF-8" LC_ALL=

Maybe problem is in parsing subprocess output?

[pavel_mlcoch@notebook ~]$ efibootmgr -v BootCurrent: 0000 Timeout: 10 seconds BootOrder: 0000,0001 Boot0000* Notebook Upgrade Bay BBS(CDROM,�,0x0)....................................................................... Boot0001* Notebook Hard Drive BBS(HD,�,0x0)....................................................................... Boot0002* USB Hard Drive 1 - Kingston DataTraveler SE9 BBS(HD,�,0x900)....................................................................... Boot0003* Notebook Ethernet BBS(128,�,0x0)........................A.............................................. Boot0004* Notebook Ethernet BBS(128,�,0x0)........................A.............................................. Boot0005* grub_uefi HD(1,GPT,a68cc155-6e48-4d51-b574-f39602047639,0x800,0xfa000)/File(\EFI\grub_uefi\grubx64.efi)

[Elinvention]

Thanks. You might be right, it could be that efibootmgr outputs the content of the nvram directly and that content is not utf-8. I'll look into that.

[Elinvention]

@pavkamlc could you redirect the output of efibootmgr -v in a file, so that I can properly see the bytes. Like efibootmgr -v > efibootmgr.log and attach the file here on github.

However as far as I understand, those descriptions should be UTF-16 encoded. Maybe efibootmgr doesn't make a good job at changing encodings.

[Elinvention]

Closing this stale issue. Feel free to reopen if the problem persists.