peter-sk

Results 28 comments of peter-sk

[FEATURE REQUEST] Support seq-to-seq architectures

I am currently testing with google/t5-v1_1-base, and at least SweReC seems to work. What problems would I expect to encounter?

[MODEL EVALUATION REQUEST] google/gemma-7b

I am doing this model on all languages except for NER tasks.

[MODEL EVALUATION REQUEST] Large QWEN-1.5 models

This doesn't work (current main branch): ```ScandEval$ CUDA_VISIBLE_DEVICES=0 scandeval -c .cache0 --no-ignore-duplicates -m Qwen/Qwen1.5-14B --use-flash-attention --raise-errors 2024-02-12 08:02:33 ⋅ Benchmarking Qwen/Qwen1.5-14B on the truncated version of SweReC 2024-02-12 08:02:35 ⋅...

new task for tabular data synthesis?

Thanks, @Wauplin! I guess the documentation for how to add tasks needs to be updated, too. It currently says to open an issue with the huggingface_hub repository. @osanseviero @adrinjalali Do...

new task for tabular data synthesis?

Any thoughts on this one?

new task for tabular data synthesis?

Thanks for the feedback. We will start by uploading some models and datasets :-)

new task for tabular data synthesis?

Yes, I mean models like CTGAN. Our group has just completed two systematic comprehensive reviews of the field, and there is a zoo of models and model-based generative tools out...

Tracking simple metrics in remote server is really slow

Slightly related to PR #3203. Copying a few megabytes worth of 1M steps tracked sequence is really slow. The PR updates aim to be able to update the remote tree...