Any instructions for fine-tuning on custom datasets?

[2000ZRL]

Thanks for your excellent work. I am curious if there are any instructions for fine-tuning video-llava on my own dataset?

[yinanhe]

You can do so by adding your meta file in the instruction_data file. The meta file is a json and follows the following schema:

[
  {
    "video": "string",
    "QA": {
      "i": "string",
      "q": "string",
      "a": "string"
    }
  },
  {
    "image": "string",
    "QA": {
      "i": "string",
      "q": "string",
      "a": "string"
    }
  },
  ...
]