Sergey Vakhreev

> could someone give us a new link to download the pretrained model? https://drive.google.com/open?id=1LfvMpFySyig5go_Jv3w1ds2aw7iodK_3

Hi! We faced this problem, so there are several things you can do: - make lora finetune params as "small" as they can be, use finetune settings section for that...

@hazratisulton have you managed to reproduce it?

``` I discovered that changing 'aux' to '_aux' allows locating the Python module ``` It's interesting, we never had such import namings, check this out https://github.com/smallcloudai/refact/blob/main/self_hosting_machinery/finetune/scripts/finetune_filter.py#L16 Are you sure you...

@ChinnYu Sorry for the delay, completely forgot about this Yes, I've got the problem, it's due to some legacy Windows folder naming limitations. We'll rename those folders in the next...

We can do a monkey patch to the finetune config using gpu runtime information. Or we can simply add another config for `Refact 1.6` with lower context size for those...

@ds5t5 Hi there! We are going to slightly change modelling and weights respectively at the [HF](https://huggingface.co/smallcloudai/Refact-1_6B-fim). The changes will include: - combining `attn.k` and `attn.v` into `attn.kv` - combining `mlp.linear_1`...