no evaluation script for open-set problem

[alexaway]

Hello, thank you for providing a good idea of research on audio question answering. When I was testing, I found that there was no evaluation script for open-set problem in the code library, could you please provide one for reference?

[YuanGongND]

hi there,

Thanks for the question. The batch inference script does the inference for open-set questions:

https://github.com/YuanGongND/ltu/blob/0fa0923f9c9d04346486a28477ba69b7d957130c/src/ltu/inference_batch.py#L150-L153

The actual open-ended test set can be downloaded from https://www.dropbox.com/scl/fo/juh1dk9ltvhghuj0l1sag/h?rlkey=0n2cd5kebzh8slwanjzrfn7q6&dl=0, check open_as_gpt4.json and open_as_gpt3.json.

For your own question, you can use https://github.com/YuanGongND/ltu/blob/main/src/ltu/inference_gradio.py.

-Yuan