Hoang Ha

Realated task: Featured model https://github.com/janhq/jan/issues/860

Related: https://github.com/janhq/jan/issues/897

^ When I want to navigate the the end of my input, I have to use the keyboard which is quite uncomfortable

Just to add detail for the input dataset. 1. Columns We should accept only 3 columns/fields named |system|instruction|response| |---|---|---| ||| 2. Split train/test Normally, HF datasets only have `train` split...

I thought about this before. But don't know how it should work. Maybe we can pre-download Tinyllama for users, and then we can use it as a title maker?

After researching, I think we need a way to parse the *character card* for roleplaying if users need. Reference: janitorai

Thank you. Wonderful findings. Can I ask about your hardware to run this yaml? and do you use --cuda?

For reference: ``` import openai from openai import OpenAI client = OpenAI() try: #Make your OpenAI API request here response = client.completions.create( prompt="Hello world", model="gpt-3.5-turbo-instruct" ) except openai.APIError as e:...

Calculation: https://www.databricks.com/blog/llm-inference-performance-engineering-best-practices Function (bytes): batch_size * seqlen * (hidden_size/num_attention_heads) * num_hidden_layers * 2 (K and V) * 2 (bytes per Float16) * num_key_value_heads ^ I adjusted the function with the...

1. We will need to gather information about the model in `config.json` e.g. Mistral 7B ``` { "architectures": [ "MistralForCausalLM" ], "attention_dropout": 0.0, "bos_token_id": 1, "eos_token_id": 2, "hidden_act": "silu", "hidden_size":...