Request for GPT-4-0314 Dump JSON for Zero-Shot Evaluation

[ParthS007]

trafficstars

Hello @InhwanBae,

Thank you for your work and research. I’ve been trying to reproduce the results from the paper, especially since gpt-4-0314 is no longer accessible via the API. I’d like to understand the presented results in a similar way to what’s available for gpt-3.5-turbo-0301 in the v1.0 tag.

Could you please include the dump JSON in the v1.0 release or share it here? If it’s not available, I’d appreciate any insights into why it was omitted from the v1.0 tag release.

Thanks again for your help!

[InhwanBae]

Hi @ParthS007,

Thank you for your interest in my work! I have included the hotel scene for the gpt-4-0314 model in v1.0 tag. If I remember correctly, the other scenes were excluded at that time because some random frames were accidentally overwritten with outputs from different model types due to wrong code execution, and I cannot ensure their integrity. I tried to locate backup files and found a dump file for the eth scene, and the 0th frames from the univ, zara1, and zara2 scenes in the initial zero-shot test code folder. I have uploaded them here. I hope this is helpful to you.

./zero-shot/output_dump/gpt-4-0314/eth_chatgpt_api_dump.json ./zero-shot/output_dump/gpt-4-0314/hotel_chatgpt_api_dump.json ./zero-shot/output_dump/gpt-4-0314/univ/univ_chatgpt_api_dump_0000.json ./zero-shot/output_dump/gpt-4-0314/zara1/zara1_chatgpt_api_dump_0000.json ./zero-shot/output_dump/gpt-4-0314/zara2/zara2_chatgpt_api_dump_0000.json

Additionally, it seems that the gpt-4-0314 model is still available via the API: OpenAI Playground - gpt-4-0314.

[ParthS007]

Thank you, @InhwanBae, for the quick response! I really appreciate your help and will follow up with more questions as I continue my research.

[ParthS007]

Hello @InhwanBae

I am currently exploring zero-shot prediction and went through the evaluation section of the zero-shot approach. I noticed that you have a visualization of prediction results for zero-shot (as shown in the attached image).

If possible, could you kindly share the script used for this visualization? It would be incredibly helpful for my analysis.

Thanks again for your help!

[InhwanBae]

Hi @ParthS007, Please check the response at this issue. Let me know if you have any questions!