Albert Zeyer

RF `conv`/`pool` use consistent batch-independent padding for striding, but `stft` does not.

A followup to #1694. When you have `dims=[static_dim, dyn_dim]`, it will currently disable the masking on the `out_dim` (`out_dim = Dim(prod(d.get_dim_value_tensor() for d in dims), name="merged")`, as the masking would...

Even with `handle_dynamic_dims=True`, it handles masking correctly, but it's not enough. I think we need to fallback to `rf.concat` which should handle it correctly now.

RF native failed to compile / load, inconsistent behavior to pure Python, dividing a tensor of type int

3

``` PyExtModCompiler call: g++ -shared -O2 -std=c++11 -fno-strict-overflow -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -O2 -fexceptions -g...

RF upcast to f32 for certain ops (layernorm, attention, etc) (bf16/f16)

1

RF doesn't do any special logic to upcast to f32 when the input is f16 or bf16 or so (e.g. via Torch AMP) for modules like `LayerNorm`. But maybe it...

Our CI GitHub Actions `pycharm-inspect` job runs `tests/pycharm-inspect.py` which uses pycharm-community-2020.2. I usually develop with the latest PyCharm, i.e. currently 2025.1.1. The differences in code style warnings has so far...

RETURNN should have direct support for FSDP. I think we should just follow the [FSDP tutorial](https://docs.pytorch.org/tutorials/intermediate/FSDP_tutorial.html). We also should run some benchmarks on how speed compares to our existing distributed...

Use https://github.com/c-testsuite/c-testsuite/ to have some more tests.

No space left on device during creating the tar file

2

Some error like this: ``` Post job cleanup. /usr/bin/tar --posix -cf cache.tzst --exclude cache.tzst -P -C /home/runner/work/owner/repo --files-from manifest.txt --use-compress-program zstdmt zstd: error 70 : Write error : cannot write...

``` ... Cache Size: ~9302 MB (9753348013 B) /usr/bin/tar -xf /home/runner/work/_temp/df03d4e4-948e-4053-be7b-a08c6d648879/cache.tzst -P -C /home/runner/work/returnn/returnn --use-compress-program unzstd /usr/bin/tar: ../../../.local/lib/python3.8/site-packages/nvidia/cublas/lib/libcublasLt.so.12: Wrote only 1536 of 10240 bytes ... /usr/bin/tar: ../../../.local/lib/python3.8/site-packages/nvidia/cublas/include/__pycache__: Cannot mkdir: No...