rules_swift icon indicating copy to clipboard operation
rules_swift copied to clipboard

Character encoding issue with international filenames.

Open johnno1962 opened this issue 3 years ago • 3 comments

Hi,

I've been chasing an issue where: if I include a swift file such as (for example) a 进行正式友好访问 2.swift in a source directory and bring it in using either a glob() in a swift_library entry or enter the full filename it gets garbled and bazel is unable to build giving an error such as the following:

<unknown>:0: error: error opening input file 'ios-app/UrlGet/a 进行æ£å¼å‹å¥½è®¿é—® 2.swift' (No such file or directory)

The garbled file name is as if the filename is being interpreted as latin1 even though it is utf-8 encoded and then written as UTF-8 so it gets encoded twice (bazel examples project tutorials/ios-app, macOS Monterey 12.6 on M1).

Johns-Mac-mini tutorial echo a 进行正式友好访问 2.swift | od -b
0000000   141 040 350 277 233 350 241 214 346 255 243 345 274 217 345 217
0000020   213 345 245 275 350 256 277 351 227 256 040 062 056 163 167 151
0000040   146 164 012
0000043

Johns-Mac-mini tutorial % echo 'a è¿^Ûè¡^Ìæ<00ad>£å¼^Ïå^Ï^Ë好访é^×® 2.swift' | od -b
0000000   141 040 303 250 302 277 136 303 233 303 250 302 241 136 303 214
0000020   303 246 074 060 060 141 144 076 302 243 303 245 302 274 136 303
0000040   217 303 245 136 303 217 136 303 213 303 245 302 245 302 275 303
0000060   250 302 256 302 277 303 251 136 303 227 302 256 040 062 056 163
0000100   167 151 146 164 012
0000105

This seems to be specific to the rules_swift repo as an Objective-C file with an international name in a objc_library entry builds fine. Any ideas where to look? Fixing this would make life easier for international users.

johnno1962 avatar Oct 26 '22 21:10 johnno1962

Unfortunately this appears to be an upstream bazel bug, and it doesn't repro with objc_library because it must not be using params files for all invocations as we are to swift https://github.com/bazelbuild/bazel/issues/16594

keith avatar Oct 27 '22 16:10 keith

Params files are required when using bazel workers, which we have to use for incremental compilation. Theoretically we could disable that if you were forcing WMO, but that's probably too annoying / confusing

keith avatar Oct 27 '22 16:10 keith

ok, thanks @keith

johnno1962 avatar Oct 27 '22 17:10 johnno1962