Bitwarden CLI: Umlauts / Umlaute

[superfliege]

Steps To Reproduce

I would like to create entries using the Bitwarden CLI. Here I see that unfortunately umlauts ä,ö,ü,ß do not work. These are passed correctly to “bw create item”, but then only a question mark is stored here.

I create an entry via the Bitwarden CLI with the username "Töst". After that, in the Bitwarden client (whether browser or app), the username is "T?st".

Is there a possibility to create items with umlauts with the Bitwarden CLI?

My OS is Windows 10 and i use the PowerShell 5.x.

Expected Result

Töst

Actual Result

T?st

Screenshots or Videos

No response

Additional Context

No response

Operating System

Windows

Operating System Version

10

Shell

PowerShell

Build Version

1.18.1

Issue Tracking Info

• [x] I understand that work is tracked outside of Github. A PR will be linked to this issue should one be opened to address it, but Bitwarden doesn't use fields like "assigned", "milestone", or "project" to track progress.

[djsmith85]

@superfliege: Linking the separate discussions on the Community Forums and Reddit

I have not vetted this, but I assume this has to do with Powershell itself and which encoding/system code-page is being used. Special characters usually need to be surrounded by quotes and possibly also escaping . Additionally could you please try with an updated version of the CLI (1.18.1 is from Sep 2021)

[3vent-Horiz0n]

My setup

PS> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.3.1
PSEdition                      Core
GitCommitId                    7.3.1
OS                             Microsoft Windows 10.0.19044
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

PS> bw -v
2023.1.0

I had a similar problem when getting an item from my vault and @djsmith85 is right. With PowerShell you are running into encoding problems. Why? Because PowerShell...

First of all: When working "manually" with the Bitwarden CLI you could always switch to cmd.exe. You should not have the problem there. If you prefer PowerShell or you are using Bitwarden CLI as part of an automation process like I do, you need to configure your PowerShell OutputEncoding at runtime.

There are two cases. One is sending data, the other one is receiving data.

Case 1 - Sending

This is the problem @superfliege is describing in this issue. As far as I can tell this is only an issue with Windows PowerShell (Version 5.1 or lower), not PowerShell Core (Version 6 or higher).

Problem

When sending data via PowerShell all Umlaut characters get converted into question marks.

PS> $NewBwFolder = bw get template folder | ConvertFrom-Json
PS> $NewBwFolder.Name = "Földer1"
PS> $NewBwFolder | ConvertTo-Json | bw encode | bw create folder

This results in a folder named "F?lder1".

Solution

Set $OutputEncoding correctly

PS> $OutputEncoding = [System.Text.Utf8Encoding]::new($false)
PS> $NewBwFolder = bw get template folder | ConvertFrom-Json
PS> $NewBwFolder.Name = "Földer1"
PS> $NewBwFolder | ConvertTo-Json | bw encode | bw create folder

Here the result is a folder called "Földer1"

Case 2 - Receiving

For automation tasks you most likely want to read items from you Bitwarden Vault. A similar issue is happening here:

Problem

I created a test item in my vault called "Umlaut Test Ä" with password "T3ü7r2"

# get item by id
PS> $BwItem = bw get item abdb2080-1e22-418d-b4da-af8c00d372a7 | ConvertFrom-Json
PS> $BwItem.name
Umlaut Test Ä
PS> $BwItem.login.password
T3├╝7r2

Solution

In the script put the following command:

PS> [Console]::OutputEncoding = New-Object System.Text.UTF8Encoding
PS> $BwItem = bw get item abdb2080-1e22-418d-b4da-af8c00d372a7 | ConvertFrom-Json
PS> $BwItem.name
Umlaut Test Ä
PS> $BwItem.login.password
T3ü7r2

---

The changes to $OutputEncoding is temporary. You need to set it once per script. I guess you could set it globally. I hope this is useful for everyone struggling with PowerShell encoding.

---

Sources:

1. Encoding settings for sending data: encoding - Use Powershell to import website with Chinese domain - Stack Overflow
2. Encoding settings for receiving data: https://stackoverflow.com/questions/42785077/utf8-encoding-changes-data-format/42787047#42787047

[djsmith85]

@3vent-Horiz0n Awesome post explaining all the possible variations 🙌.

@superfliege I'm closing this as @3vent-Horiz0n has provided some excellent details and solutions regarding your issue.

[bwbug]

Another solution is to make sure that your script files are saved in the format "UTF-8 with BOM", as BOM-less scripts cause PowerShell 5+ and earlier to default to Windows-1252 encoding. Detailed information about the problem and various solution approaches is available here:

https://learn.microsoft.com/en-us/powershell/scripting/dev-cross-plat/vscode/understanding-file-encoding