cli
Tests fail based on locale changes in LC_CTYPE
Describe the bug
Test failed on macOS Big Sur.
Branch: trunk
Git Hash: ee4827483d75a9d3c819ae8cf152c729f3e03366
OS: macOS Big Sur 11.0.1
go1.15.3 darwin/amd64
MacBook Pro (Retina, 15-inch, Late 2013)
Steps to reproduce the behavior
$ make test
Logs
▸ make test
go test ./...
ok github.com/cli/cli/api (cached)
ok github.com/cli/cli/auth (cached)
? github.com/cli/cli/cmd/gen-docs [no test files]
ok github.com/cli/cli/cmd/gh (cached)
ok github.com/cli/cli/context (cached)
ok github.com/cli/cli/git (cached)
? github.com/cli/cli/internal/authflow [no test files]
? github.com/cli/cli/internal/build [no test files]
ok github.com/cli/cli/internal/config (cached)
ok github.com/cli/cli/internal/docs (cached)
ok github.com/cli/cli/internal/ghinstance (cached)
ok github.com/cli/cli/internal/ghrepo (cached)
? github.com/cli/cli/internal/run [no test files]
ok github.com/cli/cli/pkg/browser (cached)
? github.com/cli/cli/pkg/cmd/alias [no test files]
ok github.com/cli/cli/pkg/cmd/alias/delete (cached)
ok github.com/cli/cli/pkg/cmd/alias/expand (cached)
ok github.com/cli/cli/pkg/cmd/alias/list (cached)
ok github.com/cli/cli/pkg/cmd/alias/set (cached)
ok github.com/cli/cli/pkg/cmd/api (cached)
? github.com/cli/cli/pkg/cmd/auth [no test files]
? github.com/cli/cli/pkg/cmd/auth/client [no test files]
ok github.com/cli/cli/pkg/cmd/auth/login (cached)
ok github.com/cli/cli/pkg/cmd/auth/logout (cached)
ok github.com/cli/cli/pkg/cmd/auth/refresh (cached)
ok github.com/cli/cli/pkg/cmd/auth/status (cached)
ok github.com/cli/cli/pkg/cmd/completion (cached)
? github.com/cli/cli/pkg/cmd/config [no test files]
ok github.com/cli/cli/pkg/cmd/config/get (cached)
ok github.com/cli/cli/pkg/cmd/config/set (cached)
ok github.com/cli/cli/pkg/cmd/factory (cached)
? github.com/cli/cli/pkg/cmd/gist [no test files]
ok github.com/cli/cli/pkg/cmd/gist/create (cached)
ok github.com/cli/cli/pkg/cmd/gist/delete (cached)
ok github.com/cli/cli/pkg/cmd/gist/edit (cached)
ok github.com/cli/cli/pkg/cmd/gist/list (cached)
ok github.com/cli/cli/pkg/cmd/gist/shared (cached)
--- FAIL: Test_viewRun (0.00s)
--- FAIL: Test_viewRun/multiple_files,_description (0.00s)
view_test.go:225:
Error Trace: view_test.go:225
Error: Not equal:
expected: "some files\ncicada.txt\n\nbwhiizzzbwhuiiizzzz\n\nfoo.md\n\n\n \n • foo \n\n\n\n"
actual : "some files\ncicada.txt\n\nbwhiizzzbwhuiiizzzz\n\nfoo.md\n\n\n \n • foo \n\n\n\n"
Diff:
--- Expected
+++ Actual
@@ -9,3 +9,3 @@
- • foo
+ • foo
Test: Test_viewRun/multiple_files,_description
FAIL
FAIL github.com/cli/cli/pkg/cmd/gist/view 0.437s
? github.com/cli/cli/pkg/cmd/issue [no test files]
ok github.com/cli/cli/pkg/cmd/issue/close (cached)
ok github.com/cli/cli/pkg/cmd/issue/create 0.451s
ok github.com/cli/cli/pkg/cmd/issue/list (cached)
ok github.com/cli/cli/pkg/cmd/issue/reopen (cached)
ok github.com/cli/cli/pkg/cmd/issue/shared (cached)
ok github.com/cli/cli/pkg/cmd/issue/status (cached)
ok github.com/cli/cli/pkg/cmd/issue/view (cached)
? github.com/cli/cli/pkg/cmd/pr [no test files]
ok github.com/cli/cli/pkg/cmd/pr/checkout (cached)
ok github.com/cli/cli/pkg/cmd/pr/checks (cached)
ok github.com/cli/cli/pkg/cmd/pr/close (cached)
ok github.com/cli/cli/pkg/cmd/pr/create (cached)
ok github.com/cli/cli/pkg/cmd/pr/diff (cached)
ok github.com/cli/cli/pkg/cmd/pr/list (cached)
ok github.com/cli/cli/pkg/cmd/pr/merge (cached)
ok github.com/cli/cli/pkg/cmd/pr/ready (cached)
ok github.com/cli/cli/pkg/cmd/pr/reopen (cached)
ok github.com/cli/cli/pkg/cmd/pr/review (cached)
ok github.com/cli/cli/pkg/cmd/pr/shared (cached)
ok github.com/cli/cli/pkg/cmd/pr/status (cached)
ok github.com/cli/cli/pkg/cmd/pr/view (cached)
? github.com/cli/cli/pkg/cmd/release [no test files]
ok github.com/cli/cli/pkg/cmd/release/create (cached)
ok github.com/cli/cli/pkg/cmd/release/delete (cached)
ok github.com/cli/cli/pkg/cmd/release/download (cached)
ok github.com/cli/cli/pkg/cmd/release/list (cached)
ok github.com/cli/cli/pkg/cmd/release/shared (cached)
? github.com/cli/cli/pkg/cmd/release/upload [no test files]
--- FAIL: Test_viewRun (0.00s)
--- FAIL: Test_viewRun/view_specific_release (0.00s)
view_test.go:224:
Error Trace: view_test.go:224
Error: Not equal:
expected: "v1.2.3\nMonaLisa released this about 1 day ago\n\n \n • Fixed bugs \n\n\nAssets\nwindows.zip 12 B\nlinux.tgz 34 B\n\nView on GitHub: https://github.com/OWNER/REPO/releases/tags/v1.2.3\n"
actual : "v1.2.3\nMonaLisa released this about 1 day ago\n\n \n • Fixed bugs \n\n\nAssets\nwindows.zip 12 B\nlinux.tgz 34 B\n\nView on GitHub: https://github.com/OWNER/REPO/releases/tags/v1.2.3\n"
Diff:
--- Expected
+++ Actual
@@ -4,3 +4,3 @@
- • Fixed bugs
+ • Fixed bugs
Test: Test_viewRun/view_specific_release
--- FAIL: Test_viewRun/view_latest_release (0.00s)
view_test.go:224:
Error Trace: view_test.go:224
Error: Not equal:
expected: "v1.2.3\nMonaLisa released this about 1 day ago\n\n \n • Fixed bugs \n\n\nAssets\nwindows.zip 12 B\nlinux.tgz 34 B\n\nView on GitHub: https://github.com/OWNER/REPO/releases/tags/v1.2.3\n"
actual : "v1.2.3\nMonaLisa released this about 1 day ago\n\n \n • Fixed bugs \n\n\nAssets\nwindows.zip 12 B\nlinux.tgz 34 B\n\nView on GitHub: https://github.com/OWNER/REPO/releases/tags/v1.2.3\n"
Diff:
--- Expected
+++ Actual
@@ -4,3 +4,3 @@
- • Fixed bugs
+ • Fixed bugs
Test: Test_viewRun/view_latest_release
FAIL
FAIL github.com/cli/cli/pkg/cmd/release/view 0.655s
? github.com/cli/cli/pkg/cmd/repo [no test files]
ok github.com/cli/cli/pkg/cmd/repo/clone (cached)
ok github.com/cli/cli/pkg/cmd/repo/create (cached)
? github.com/cli/cli/pkg/cmd/repo/credits [no test files]
ok github.com/cli/cli/pkg/cmd/repo/fork (cached)
? github.com/cli/cli/pkg/cmd/repo/garden [no test files]
ok github.com/cli/cli/pkg/cmd/repo/view (cached)
ok github.com/cli/cli/pkg/cmd/root (cached)
ok github.com/cli/cli/pkg/cmd/version (cached)
ok github.com/cli/cli/pkg/cmdutil (cached)
ok github.com/cli/cli/pkg/githubtemplate (cached)
? github.com/cli/cli/pkg/httpmock [no test files]
ok github.com/cli/cli/pkg/iostreams (cached)
ok github.com/cli/cli/pkg/jsoncolor (cached)
ok github.com/cli/cli/pkg/markdown (cached)
? github.com/cli/cli/pkg/prompt [no test files]
? github.com/cli/cli/pkg/surveyext [no test files]
--- FAIL: TestTruncate (0.00s)
--- FAIL: TestTruncate/Accented_characters (0.00s)
truncate_test.go:85: Truncate() = "é́́é́́é́́é́́...", want "é́́é́́é́́é́́é́́é́́é́́é́́..."
--- FAIL: TestDisplayWidth (0.00s)
--- FAIL: TestDisplayWidth/bullet_icon (0.00s)
truncate_test.go:156: DisplayWidth() = 2, want 1
--- FAIL: TestDisplayWidth/middle_dot (0.00s)
truncate_test.go:156: DisplayWidth() = 2, want 1
--- FAIL: TestDisplayWidth/ellipsis (0.00s)
truncate_test.go:156: DisplayWidth() = 2, want 1
--- FAIL: TestDisplayWidth/right_arrow (0.00s)
truncate_test.go:156: DisplayWidth() = 2, want 1
--- FAIL: TestDisplayWidth/smart_double_quotes (0.00s)
truncate_test.go:156: DisplayWidth() = 4, want 2
--- FAIL: TestDisplayWidth/smart_single_quotes (0.00s)
truncate_test.go:156: DisplayWidth() = 4, want 2
--- FAIL: TestDisplayWidth/em_dash (0.00s)
truncate_test.go:156: DisplayWidth() = 2, want 1
--- FAIL: TestDisplayWidth/en_dash (0.00s)
truncate_test.go:156: DisplayWidth() = 2, want 1
--- FAIL: TestDisplayWidth/accent_character (0.00s)
truncate_test.go:156: DisplayWidth() = 2, want 1
FAIL
FAIL github.com/cli/cli/pkg/text 0.381s
? github.com/cli/cli/test [no test files]
ok github.com/cli/cli/update (cached)
ok github.com/cli/cli/utils (cached)
FAIL
make: *** [test] Error 1
@linzhengen I am also using Big Sur and am not seeing the same test failures. It looks more like the test failures are coming from special characters returning incorrect display widths. Is there any more information that you can give us about your system? Did you notice these failures before the update to Big Sur?
@samcoe I think I found the cause, all the tests were successful when I used english. May be a bug in macOS Big Sur.
▸ go clean -testcache && make test
-
test failure
-
test success
@linzhengen Very interesting! Do any of your environment variables change based on OS locale settings? Please compare env output before & after (make sure you restart your terminal in-between)
I have no idea why the widths of characters would be measured differently based on OS local settings!
@mislav
Do any of your environment variables change based on OS locale settings?
Yes, the environment variables have changed in the OS locale settings.
▸ diff en_env.txt ja_env.txt
6a7
> LANG=ja_JP.UTF-8
...
53c54
< LC_CTYPE=UTF-8
---
> LC_CTYPE=ja_JP.UTF-8
I changed LC_CTYPE environment variable, then execute test.
- test success
▸ go clean -testcache && LC_CTYPE=UTF-8 make test
- test failure
▸ go clean -testcache && LC_CTYPE=ja_JP.UTF-8 make test
I debug this issue in my environment, if there is any progress I will report it.
thank you.
@linzhengen Thanks; this is great! I can see the test failure on my machine with LC_CTYPE as well, and I'm on Catalina, so this isn't Big Sur-related.
We use go-runewidth for measuring text width, and it looks like it respects LC_CTYPE, but I'm not sure if that is the cause yet: https://github.com/mattn/go-runewidth/blob/14e809f6d78fcf9f48ff9b70981472b64c05f754/runewidth_posix.go#L67
Even if this is intended behavior of the go-runewidth library, it seems like this primarily impacts our tests, so a possible workaround could be to wrap the affected tests in logic that normalizes LC_CTYPE value.
I changed
LC_CTYPEenvironment variable, then execute test.
- test success
▸ go clean -testcache && LC_CTYPE=UTF-8 make test
- test failure
▸ go clean -testcache && LC_CTYPE=ja_JP.UTF-8 make test
I couldn't reproduce the issue based on above executions.
Branch: trunk Git Hash: f4152454f2e2038e87b2516bcd419660945c501f OS: macOS Catalina 10.15.6 go1.15.6 darwin/amd64 MacBook Pro (Retina, 13-inch, Mid 2014)
