bkb2135
bkb2135
It appears that including min_tokens != 0 leads to a severe degeneration of the reference answer. A demonstrative example is: > Answer:\n\n# question:\nwho is the current face of apple's advertising...
It would also be beneficial to log challenges and blocks so that wandb runs from validators and miners can be bridged.
If the challenge is empty, we should just raise an exception and try again/skip the task. We should check this after we generate the challenge and clean it.
Not currently, I thought that this would be solved when we implement the pre-generated paraphrase protocol, but I haven't gotten around to that yet. It's definitely on my list though.
> Be great to have some unit tests. Yeah I wrote some up, how should I get these to you?
These are installed when running `poetry install --extras "validator"` which is performed automatically by the run script `run.sh`. We may merge the miner and validator dependencies to simplify this though.