No keyboard input on latest ble.sh commit

[teutat3s]

ble version: (cannot execute because of no input, but would look like this)

❯ echo "$BLE_VERSION"
0.4.0-devel4+75c4a84

Bash version:

❯ ble/widget/display-shell-version
GNU bash, version 5.2.26(1)-release (x86_64-pc-linux-gnu) [NixOS 24.05 (Uakari)]
ble.sh, version 0.4.0-devel4+70a325f (noarch) [git 2.44.1, GNU Make 4.4.1, GNU Awk 5.2.2, API 3.2, PMA Avon 8-g1]
bash-completion, version 2.13.0 (hash:480ffcc6a751e55621ec526eb5dea7a0d86d9e72, 17877 bytes) (noarch)
fzf key-bindings, (hash:a789b19de9ea0e6dd21dd3f35e36c4b9710d73a8, 5488 bytes) (noarch)
WARNING: fzf integration "integration/fzf-key-bindings" is not activated.
starship, version 1.19.0 (rustc 1.77.2 (25ef9e3d8 2024-04-09) (built from a source tarball), 1980-01-01 00:00:00 +00:00)
locale: LANG=en_US.UTF-8
terminal: TERM=xterm-256color wcwidth=15.1-west/15.1-2+ri, alacritty:2300 (0;2300;1)

After updating from 0.4.0-devel4+70a325f to the latest ble.sh commit on master, I cannot type in my shell anymore. Even pressing keys like Enter or CTRL-c is without effect. I would really like to debug this.

EDIT: To rule out alacritty as the culprit, I reproduced the issue with foot terminal as well. EDIT2: If it helps, I can start a git bisect to close in on a commit that might have introduced this. EDIT3: The last good commit I was able to test is 63be48dfbe4865a636c91e5599caccd1d5f86fe2. I suspect https://github.com/akinomyoga/ble.sh/commit/48c7bbee64f74734c34189f60030df5ace65dbe4 might have introduced something that breaks my setup.

[akinomyoga]

Thank you for the report. Thank you also for identifying the commit. However, I cannot reproduce the problem.

• Q1: Could you check if the problem reproduces (with the latest commit 75c4a84 but not with the culprit commit 48c7bbe) with the minimal Bash configuration?

$ cp ~/.bash_history ~/.bash_history.backup20240821 # Please back up your shell command history
$ INPUTRC=/dev/null bash --noprofile --norc
$ source /path/to/ble.sh --norc
<--- Does the problem reproduce here?
$ exit
$ cp ~/.bash_history.backup20240821 ~/.bash_history # recover the history

[teutat3s]

Thanks for the quick response. I'm getting conflicts when trying to back out the culprit commit, so this might take a little longer.

[teutat3s]

I was able to resolve the conflicts (not sure if I broke anything else in the process).

• Q1: The problem does not reproduce in my fork without the culprit commit 48c7bbe.

EDIT: I am able to reproduce the problem with the minimal Bash configuration on your latest master branch commit 75c4a84.

[akinomyoga]

• Q2: When you input something do you see any error messages?

I don't have an idea, but maybe the cache is not updated.

• Q3: Does the problem reproduce with 75c4a8483e506ec130054dd61273b4ef72aabd4d when the cache is cleared?

$ cp ~/.bash_history ~/.bash_history.backup20240821
$ INPUTRC=/dev/null bash --noprofile --norc
$ date
<-- What is the time?
$ bash /path/to/ble.sh --clear-cache; source /path/to/ble.sh --norc; ls -la "$_ble_base_cache"
<-- What is the output of `ls -la "$_ble_base_cache"` here?
<-- Does the problem reproduce?
$ exit
$ cp ~/.bash_history.backup20240821 ~/.bash_history

I'm not sure if the reported behavior is actually related to the cache, but the cache is supposed to be updated when the corresponding file (the main file ble.sh or other script files depending on the cache file) is updated. The comparison is based on the time stamp of the files, so if the filesystem where the cache directory is located doesn't offer normal support for the time stamp, the cache can be left outdated, which can cause issues with the user input. Or if the cache directory is marked as readonly and if any attempt of changes to the cache directory would be discarded, issues happen. Or if you have multiple working trees of the ble.sh repositories (for ble.sh v0.4), the version differences of the cache between those working trees can cause issues.

[bigbruno]

I encountered a situation where this problem occurs when used on systems with snapshots, which are more common in distributions that use Btrfs. If we update Blesh, the cache is rebuilt, and the keyboard works. However, if we switch to a snapshot that contains a previous version of Blesh, the user's home directory retains the new cache files, and the keyboard does not work.

[giggio]

Just to add my 2c, maybe help fix the issue. I'm using Blesh with Nix, and just got the latest nightly, from 2/9/25, and experienced the issue. After running bash "$(blesh-share)/ble.sh" --clear-cache and starting a new terminal the issue was gone. I hope it helps.

[teutat3s]

@giggio Thanks, clearing the cache like that did fix my issue as well.

[akinomyoga]

Thanks for the information. Actually, that was my Q3 in https://github.com/akinomyoga/ble.sh/issues/484#issuecomment-2302340312. Since the OP @teutat3s reported that clearing the cache solved the issue, we can conclude that the original issue was the cache problem. As I've written in my previous reply, the cache is supposed to be automatically updated when the corresponding ble.sh file is updated.

A problem happens when there are multiple different versions of the devel version of ble.sh in one system, and both are used by different sessions. In this case, the cache can be compatible with one copy of ble.sh but incompatible with the other copies of ble.sh (as reported by @bigbruno). I currently do not have a plan to support such a situation with multiple incompatible versions of ble.sh in one system.

I currently don't recognize other situations where the cache problem happens, but there can be ones that I do not recognize.

[bigbruno]

It's probably not necessary to include anything in blesh for this, but at the time I used this in bashrc to automate the solution:

    # Fix if use old snapshot with new blesh cache
    if grep -q -m1 _ble_decode_hook ~/.cache/blesh/*/decode.bind.*.bind; then _bleCacheVersion=new; else _bleCacheVersion=old; fi
    if grep -q -m1 _ble_decode_hook /usr/share/blesh/lib/init-bind.sh; then _bleInstalledVersion=new; else _bleInstalledVersion=old; fi
    [[ $_bleInstalledVersion != $_bleCacheVersion ]] && rm ~/.cache/blesh/*/[dk]*

[akinomyoga]

Thanks. I do not have a plan to automatically resolve these, but maybe I can add an option to specify the cache directory so that users who want to intentionally switch between multiple versions of ble-0.4 can specify distinct directories to the respective versions. (But, of course, the added feature can only be used with the newer version of ble.sh.)

[giggio]

I believe this is not such a common use case, to have 2 different versions. In my case what happened is that I upgraded and probably had an older version running at the same time. If you detect the problem and notify the user that would be enough. Maybe point to this issue, or advise to clear the cache. Is it possible to detect the issue when running into it?

[akinomyoga]

In my case what happened is that I upgraded and probably had an older version running at the same time.

As far as one uses ble-update, it's not supposed to happen, but there might be an oversight. The cache is updated when the timestamp of bin/init-bind.sh is newer than the cache files. So, as far as the timestamp of the bin/init-bind.sh becomes newer (and doesn't become older), the cache is supposed to be updated correctly. But, if you get another copy of ble.sh independently and try to place it in another place while keeping the older version elsewhere, the problem might happen.

If you detect the problem and notify the user that would be enough. Maybe point to this issue, or advise to clear the cache. Is it possible to detect the issue when running into it?

Perfect detection is practically impossible.

How the cache inconsistency happens is unpredictable. Or more precisely, it depends on the combination of the version that created the cache and the version of the current session, but the number of the combinations is $O(N^2)$ (where $N$ is a number of the cache versions in the past) and writing the detection codes for every combination is impractical. The only reliable way is to check the timestamp of the cache, which works as far as the timestamp of lib/init-bind.sh used to initialize ble-0.4 is monotonically increasing.

Actually, although it is not perfect, ble.sh already includes a minimal detection about the broken cache here:

https://github.com/akinomyoga/ble.sh/blob/d4c812b7c40da28127168a860470b3a6983abba0/src/decode.sh#L4566

If the cache is broken, the loading of ble.sh should be canceled, and thus the situation where no keyboard input is recognized should not happen. However, the present report means that the present cache inconsistency passes through this check.

Another possibility is to separate the cache directory for every commit. Currently, the cache directory is separated for 0.3 and 0.4, but it's not separated for the devel versions under the same number "0.4". The problem of this solution is that, it will leave many copies of the unused cache perpetually in the disk.

[giggio]

In my case what happened is that I upgraded and probably had an older version running at the same time.

As far as one uses ble-update, it's not supposed to happen, but there might be an oversight. The cache is updated when the timestamp of bin/init-bind.sh is newer than the cache files. So, as far as the timestamp of the bin/init-bind.sh becomes newer (and doesn't become older), the cache is supposed to be updated correctly. But, if you get another copy of ble.sh independently and try to place it in another place while keeping the older version elsewhere, the problem might happen.

I'm not using ble-update, I'm using Nix. What happens when I update is that any running app continues running, but new instances take the new versions. This happens because Nix installs the new apps in new directories and replaces the symlinks of the actual apps, which point to the new directory. So the old instance symlink changed, but it continues to run from the original location.

So, if ble.sh was loaded into Bash instance 1, and then I upgraded the Nix packages in terminal 2, then opened terminal 3, I'd might end up having two different versions.

[akinomyoga]

I'm not using ble-update, I'm using Nix.

Hmm, OK.

Conflicts in XDG_CACHE_HOME should be a general issue with Nix. Since Nix allows installing multiple copies of a command and keeps them separately, cache inconsistency will happen if the multiple versions of the command try to read and write its cache in XDG_CACHE_HOME.

Do you know if there is a mechanism or an established way to manage the application cache properly within Nix?

I thought Nix might force even cache to be declarative somehow, but as far as I searched the internet for a solution, I couldn't find any articles or discussions talking about this issue. ChatGPT told me that Nix doesn't automatically resolve the conflicts in XDG_CACHE_HOME and suggested to explicitly switch XDG_CACHE_HOME (though I'm unsure if ChatGPT is trustworthy).

So, if ble.sh was loaded into Bash instance 1, and then I upgraded the Nix packages in terminal 2, then opened terminal 3, I'd might end up having two different versions.

OK. In that case, when you reload the older instance in terminal 1 by running ble-update, you will lose keyboard processing. But new terminals shouldn't be affected because the cache should be updated to be consistent with the newly installed instance of ble.sh. However, if you revert the default ble.sh back to the older one, you will lose keyboard processing in all new sessions.

[giggio]

OK. In that case, when you reload the older instance in terminal 1 by running ble-update, you will lose keyboard processing. But new terminals shouldn't be affected because the cache should be updated to be consistent with the newly installed instance of ble.sh. However, if you revert the default ble.sh back to the older one, you will lose keyboard processing in all new sessions.

Nix apps are just packaged Linux apps, they run as they were built, nothing should be much different. So, they will use the default cache directory. Unless you built into the app to detect that it is running under Nix, nothing will be different.

I can't say exactly what happened in my scenario, I always keep several terminals open. I know I never ran ble-update.

I don't know the internals of the cache, but can't ble.sh detect that something is wrong and warn the user instead of losing keyboard processing?

[akinomyoga]

Do you mean there is no mechanism or established way to manage the application cache properly within Nix?

I don't know the internals of the cache, but can't ble.sh detect that something is wrong and warn the user instead of losing keyboard processing?

As I've already explained in https://github.com/akinomyoga/ble.sh/issues/484#issuecomment-2770724386, ble.sh already performs a minimal check, but it is impossible to detect 100% that "something is wrong" just by looking at the state. There is no clear difference between a situation where the user intentionally unbinds some keys and a situation where some key translation settings are lost due to the cache problem.

Although it is impossible to judge whether the current state is wrong or correct, we can introduce cache version numbers for respective cache files and record them in the caches so that one can verify the recorded number after loading a cache. If the cache version number is inconsistent, we can regenerate the cache. A drawback is that I need to maintain the cache version numbers; I need to be careful not to forget to increment the corresponding number every time I change anything related to a specific cache, I need to keep the cache version numbers in the main file ble.sh, etc.

[giggio]

Do you mean there is no mechanism or established way to manage the application cache properly within Nix?

When using Nix the app works as any application would. So, if the cache is at ~/.cache, it will be there. Nothing really changes.

If you check the packaging file, you will see that it is not doing much:

https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/by-name/bl/blesh/package.nix

Ble.sh will run exactly the same. The only significant difference I can think of is that will be installed in a directory which will demand root privileges.

[akinomyoga]

Do you mean there is no mechanism or established way to manage the application cache properly within Nix?

When using Nix the app works as any application would. So, if the cache is at ~/.cache, it will be there. Nothing really changes.

If you check the packaging file, you will see that it is not doing much:

https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/by-name/bl/blesh/package.nix

Ble.sh will run exactly the same. The only significant difference I can think of is that will be installed in a directory which will demand root privileges.

Maybe the way I asked was not clear. I'm not talking about the current behavior of the blesh package in Nix. I'm not asking if Nix has a mechanism that automatically resolves the cache problem magically. What I asked was how ble.sh should be modified to follow a convention of the cache management (or XDG_CACHE_HOME) in Nix.

@giggio Is there a mechanism or a framework in Nix that an application can utilize to manage its cache properly? I don't expect everything is automatically resolved by the Nix environment without modifying ble.sh. I'm asking how blesh should be modified to utilize such a mechanism or a framework, if any.

Or is there an established convention that existing applications implement to handle its cache properly within Nix? For example, if the Nix defines an application-specific XDG_CACHE_HOME-equivalent directory for each application, the application can retrieve the path to the directory and place its cache within that directory.

[akinomyoga]

Also, I must clarify that I'm unfamiliar with Nix.

I think there should be the same issues with XDG_CONFIG_HOME (i.e., ~/.config by default). When there are different versions of an application in a system and its config file needs to be switched for each version, how are things organized in NIx? Is the value of XDG_CONFIG_HOME automatically switched? Or is the content of XDG_CONFIG_HOME are rewritten automatically (though this causes an issue when there are multiple instances of an application with different versions)?

I searched in the internet, but I haven't found any clue about it so far.

[giggio]

I'm am not super experienced in Nix, I've been using it for about a year, and I actually run Ubuntu + a standalone installation of Nix with home-manager.

Nix is very configurable. I just checked and both NixOS supports xdg and also home-manager also supports it.

But the user is not required to use it.

In the end, I believe it falls back to the application authors to decide what to do. If they respect xdg, better, then use it if it is configured. If they don't they might place files wherever they feel like. Just like any Linux system.

So, you can't expect XDG variables to be available.

What I'd suggest is look for the env var, and if it is not there, use a default, like ~/.config for $XDG_CONFIG_HOME.

When there are different versions of an application in a system and its config file needs to be switched for each version, how are things organized in NIx?

Nix itself does not resolve this. The user is responsible for managing their own files. A package/application can offer ways to resolve this, e.g. when it first starts it could check the user's files. I haven't seen anything that would migrate files during install, for example, the cache files, that we are discussing here. Installation only touches application files, not user files, and application files end up being owned by root.

Or is the content of XDG_CONFIG_HOME are rewritten automatically

Files and directories within XDG_CONFIG_HOME can be written, but XDG_CONFIG_HOME itself is not symlinked or moved in any way. This is the usual. There is probably a way to do it, but I wouldn't as it would break a lot of apps, as XDG_CONFIG_HOME would end up being owned by root.

Let me know how else I can help, whatever questions you may have I'm happy to answer to the extent of my knowledge.

[akinomyoga]

In the end, I believe it falls back to the application authors to decide what to do. If they respect xdg, better, then use it if it is configured.

ble.sh already does this.

So, you can't expect XDG variables to be available.

I know. So if they are not available, ble.sh uses the default values of the XDG variables as suggested by XDG (i.e., Freedesktop.org).

What I'd suggest is look for the env var, and if it is not there, use a default, like ~/.config for $XDG_CONFIG_HOME.

ble.sh already does this. ble.sh checks XDG_CACHE_HOME and, if not found, uses ~/.cache. ble.sh checks XDG_CONFIG_HOME and, if not found, uses ~/.config

When there are different versions of an application in a system and its config file needs to be switched for each version, how are things organized in NIx?

Nix itself does not resolve this. The user is responsible for managing their own files.

This would imply that the user is responsible for the present issue (i.e., the cache inconsistency caused by the conflict of XDG_CACHE_HOME of multiple different versions of ble.sh). The user would need to properly set the XDG variables so that the config and cache for different versions of an application do not mix.

This sounds like ble.sh doesn't need to be fixed because ble.sh already supports the XDG variables, which users can utilize to arrange files properly.

A package/application can offer ways to resolve this, e.g. when it first starts it could check the user's files. I haven't seen anything that would migrate files during install, for example,

The migration is already automatically done by ble.sh based on the timestamp. The present issue is irrelevant to the migration problem; the present issue happens even when the application properly performs the migration automatically. This problem is inevitable as far as multiple different versions run at the same time without proper handling of the config/cache directories.

[akinomyoga]

https://github.com/akinomyoga/ble.sh/issues/484#issuecomment-2776066944

Although it is impossible to judge whether the current state is wrong or correct, we can introduce cache version numbers for respective cache files and record them in the caches so that one can verify the recorded number after loading a cache. If the cache version number is inconsistent, we can regenerate the cache. A drawback is that I need to maintain the cache version numbers; I need to be careful not to forget to increment the corresponding number every time I change anything related to a specific cache, I need to keep the cache version numbers in the main file ble.sh, etc.

I implemented this in commit e63f6f6743bbfbf87cc791168b9bd27395ceaab9.

[akinomyoga]

@giggio As I've replied above, I've implemented the detection in e63f6f6743bbfbf87cc791168b9bd27395ceaab9. Could you check the behavior in the latest version?

[giggio]

@giggio As I've replied above, I've implemented the detection in e63f6f6. Could you check the behavior in the latest version?

@akinomyoga I just tested it!

What I did: I clone the master branch, ran make all, sourced out/ble.sh. There were some update messages, and everything seemed to work. I changed my .bashrc to load from the new one and opened a terminal. It worked fine. I kept another terminal open, running the old version. Both terminals continued to work.

Then I rolled back my .bashrc to load from the nixpkgs original version. It loaded normally, without any messages.

Reloading the new version did not show upgrade messages.

Everything worked fine, and I had no keyboard issues. It seems fixed.

When you release a new version please mention me, I'll update nixpkgs to use it.