Windows Release Build Localization

[AleksaBajat]

Building a Windows binary in release using:

cargo build --release --features "dd" --no-default-features

results in coreutils not working with error:

> .\coreutils.exe dd --help
Could not init the localization system: Locales directory not found: Release locales directory not found starting from [Absolute Path]\dist\coreutils\target\release

This is the case when building with entire suite of tools as well.

Building the coreutils in debug works as intended.

[sylvestre]

did you try with the current main ? we landed a recent change make should work also

[sudhackar]

Looks like the same in an Alpine Linux docker container

$ docker run --name test -it --rm rust:alpine sh
# apk add musl-dev
# cd
# git clone https://github.com/uutils/coreutils.git
# cd coreutils
# cargo install --path .
# coreutils cp --help
Could not init the localization system: Locales directory not found: Release locales directory not found starting from /usr/local/cargo/bin
# cargo build --release
# ./target/release/coreutils cp --help
Could not init the localization system: Locales directory not found: Release locales directory not found starting from /tmp/host/uu/target/release
# cargo run --release cp --help
    Finished `release` profile [optimized] target(s) in 0.13s
     Running `target/release/coreutils cp --help`
Could not init the localization system: Locales directory not found: Release locales directory not found starting from /tmp/host/uu/target/release

However

# cargo run cp --help

works

looks like release is broken

[sylvestre]

@kevinburke as you touched this code recently, maybe you want to give it a try :)

[AleksaBajat]

@sylvestre I grabbed my Linux machine to check it out there as well. It seems like during the initialization in debug it picks up a path:

locales_dir: "/home/abajat/Documents/projects/coreutils/src/uucore/../uu/dd/locales"

However, in release, it looks like the .ftl file should've been copied to the release folder, making that file a tightly coupled dependency. Here are the paths that are being looked up when it fails.

coreutils: "/home/abajat/Documents/projects/coreutils/target/release/locales/dd/[locale].ftl"
fhs path: "/home/abajat/Documents/projects/coreutils/target/share/locales/dd/[locale].ftl"
fallback: "/home/abajat/Documents/projects/coreutils/target/release/dd/[locale].ftl"

Debug mode bakes in the CARGO_MANIFEST_DIR environment variable in the executable, making the binary appear to be working even when it is placed outside of the target directory.

I don't mind fixing the issue (and would love to contribute of course); I just don't know the codebase well enough to decide what the best way to fix this is. There are a couple of options: We could copy that file to the locale directory during build time or make a way to ingest the default ftl if it is not specified. This was already outlined in the comments:

// TODO: plug the deactivation of the translation
// and load the English strings directly at compilation time in the
// binary to avoid the load of the flt
// Could be something like:
// #[cfg(not(feature = "only_english"))] 

[kevinburke]

Can either of you clarify which Git commit or tag is running when you ran into these issues? Specifically I submitted 3f94a3e4d7fbd70656179f3716279b36dfe0eb1c on June 20/21, if you are running into these issues before or after that change was merged, it will help isolate the problem

[AleksaBajat]

@kevinburke, I was using the code from the upstream, not from a specific tag or release — It's an issue when you just clone the codebase and build it in release.

[kevinburke]

Okay. Does any locales directory at all exist under [Absolute Path]\dist ?

[AleksaBajat]

Each tool has it's own locales. For example in uu/dd/locales there is en-US.ftl.

[kevinburke]

OK, so to be clear, that exists at [Absolute Path]\dist\uu\dd\locales\en-US.ftl or somewhere else.

I don't have a Windows machine, so I'm not clear how this is broken for Windows clients or how to fix it.

[AleksaBajat]

The folder structure looks completely the same on Windows as it looks on Linux. Both operating systems behave in the same broken way in release. As I have said don't mind fixing it as well as long as we have a consensus on the solution.

[kevinburke]

I don't have any opinions on what a solution looks like

[ealap]

This happens on cargo install --git for me as well:

Could not init the localization system: Locales directory not found: Release locales directory not found starting from /home/ealap/.local/share/cargo/bin

Manually creating a locales directory inside the mentioned path does not resolve the issue.

[sylvestre]

Use make as a workaround

[sylvestre]

build.rs needs to be updated to install the locale if someone wants to give it a try

[sudhackar]

I'm not sure build.rs should be the best place to achieve that - it should not be treated as an install script.

[sylvestre]

@sudhackar feel free to implement a better solution :)

[mucookul]

I made a simple python script to copy all the locales into a folder called "locales"

import os, sys, shutil

def main():
    base_dir = os.getcwd()
    source_path = os.path.join(base_dir, "src", "uu")
    target_path = os.path.join(base_dir, "locales")

    if not os.path.exists(source_path):
        print(f"ERROR  Source path not found: {source_path}")
        sys.exit(1)

    os.makedirs(target_path, exist_ok=True)

    names = [entry for entry in os.listdir(source_path) if os.path.isdir(os.path.join(source_path, entry))]

    for name in names:
        source_locale_path = os.path.join(source_path, name, "locales")
        target_locale_dir = os.path.join(target_path, name)

        if not os.path.exists(source_locale_path):
            print(f"WARNING  Source locale file/directory for \"{name}\" not found: \"{source_locale_path}\"")
            continue

        if os.path.isfile(source_locale_path):
            with open(source_locale_path, "r", encoding="utf-8") as f:
                content = f.read().strip()
                source_locale_path = os.path.join(source_path, name, content)

                if not os.path.exists(source_locale_path):
                    print(f"WARNING  Source locale directory for \"{name}\" not found: \"{source_locale_path}\"")
                    continue

        shutil.copytree(source_locale_path, target_locale_dir, dirs_exist_ok=True)

if __name__ == "__main__":
    main()