mrcabbage972 comments

Results 28 comments of


                                            mrcabbage972

Setup separate environments on Redmond.ai box

So the way to create users is: 1. login as the ubuntu user 2. sudo adduser [user name] 3. sudo passwrd [user name] - will prompt for the password 4....

Train baseline models for evaluation

I didn't understand the part about the 1000 training examples. Our datasets are much bigger than that!

Train baseline models for evaluation

We have an issue for Eval Harness in the backlog.

Train baseline models for evaluation

@jordiclive @ontocord We had used layers 9-13 when we trained the experts. See: https://github.com/ontocord/MDEL/blob/main/src/mdel/train.sh#L4

Train baseline models for evaluation

@jordiclive Any updates on this issue?

Training instruction followers as composable layers and expert layers

@ontocord Can you please provide some context on what is the goal here?

Integrate with LLM evaluation frameworks

PubMedQA seems most relevant among the LM harness tasks. The harness also has an adapter for BigBench, which has a gigantic amount of tasks including some about programming: auto_debugging, python_programming_challenge,...

Integrate with LLM evaluation frameworks

@kenhktsui Should we create a separate issue to find relevant tasks to be used for the downstream evaluation?