cb88

Something is still not right with it though... llama_kv_cache_init: ROCm0 KV buffer size = 4000.00 MiB ggml_cuda_host_malloc: failed to allocate 156000.00 MiB of pinned memory: out of memory ggml_backend_cpu_buffer_type_alloc_buffer: failed...

I did not have a envsetup_user.sh so it was completely default.

Crashed here yesterday when I tried to build I updated again today, and tried again same thing, ... I suspect the new version of cmake on arch? [ 9%] Generating...

Seems to blow up much later now after updating... as well as reinstalling some of my system packages. ``` LIST_BINFO_FILE_FULLNAME[78]: /home/cb88/rocm_sdk_builder/binfo/core/035_AMDMIGraphX.binfo APP_INFO_FULL_NAME: /home/cb88/rocm_sdk_builder/binfo/core/035_AMDMIGraphX.binfo --------------------------- [78] BINFO_APP_NAME: AMDMIGraphX BINFO FILE: /home/cb88/rocm_sdk_builder/binfo/core/035_AMDMIGraphX.binfo...

Tried building vllm at this point. Note I did install setuptools_scm package via pacman -S python-setuptools-scm ``` [cb88@M31-AR0 rocm_sdk_builder]$ ./babs.sh -b binfo/extra/ai_tools.blist ROCM_TARGET_TRIPLED: x86_64-rocm-linux-gnu ROCM_PYTHON_VERSION: v3.11.11 INSTALL_DIR_PREFIX_SDK_ROOT: /opt/rocm_sdk_612 INSTALL_DIR_PREFIX_SDK_AI_MODELS: /opt/rocm_sdk_models...

I was able to build pytorch acutally, but not torch vision or audio. I also built the older tensorflow verison Went back and tried to build vllm again (after running...

Having another go at building this on Arch. llvm build failed to detect pfmlib.h I installed libpfm via pacman and it seems to confinue building.

Fails here now. ``` CCLD libucs.la /opt/rocm_sdk_612/bin/ld: /opt/rocm_sdk_612/lib/libbfd.a(elf64.o): warning: relocation against `bfd_elf64_swap_reloca_out' in read-only section `.text' /opt/rocm_sdk_612/bin/ld: /opt/rocm_sdk_612/lib/libbfd.a(bfd.o): relocation R_X86_64_PC32 against symbol `_bfd_error_buf' can not be used when making a...

There is also the issue of failed loop unrolling for gfx900 which may be slowing it down. gfx906 for whatever reason doesn't throw this error. ``` mmq.cuh:2502:24: warning: loop not...

Some additional benchmarks that were requested ``` rocblas-bench -f gemm_ex --transposeA N --transposeB N -m 4096 -n 512 -k 4096 --alpha 1 --a_type f16_r --lda 4096 --b_type f16_r --ldb 4096...