Prompting totto

[shanyas10]

Addressing the comments from https://github.com/bigscience-workshop/promptsource/pull/739

[jzf2101]

Please close #739 if you want this PR to replace it.

[shanyas10]

Hi @jzf2101 I have removed the prompts which don't contribute to the original task and made some minor changes to prompts to make them sound less technical. Regarding rendering of table, I've tried to create prompts in format other than linearized_input as well. Do you suggest that we should use linearized_input only to represent the table? However this again doesn't contain full table information (all rows and columns) but was used as input sequence to the baseline models as mentioned by @sebastianGehrmann (author of the paper) in #739

[stephenbach]

I still think there's an issue with the prompts based on these examples:

The prompt doesn't seem to provide the right info to generate that response.

I also don't think the table_to_text prompt is a useful way to do things. It took me a minute, but I think you're trying to indicate the highlighted cell by row and column number? I would not expect any current language model to be able to do that kind of counting. Perhaps there's a better way to do it?

[shanyas10]

Thanks for reviewing the prompts @stephenbach . I have fixed the bug in the templates where the values were not rendering correctly.

I also don't think the table_to_text prompt is a useful way to do things. It took me a minute, but I think you're trying to indicate the highlighted cell by row and column number? I would not expect any current language model to be able to do that kind of counting. Perhaps there's a better way to do it?

Right! This was a temporary prompt that I had created to discuss the rendering of the table that @jzf2101 mentioned. I have removed that in the recent commit

[stephenbach]

Sorry for the delay here. I think best_describe_highlighted_sentence, final_text_describing_table, and summarize_the_highlighted_cell are good.

• I don't really get the given_highlighted_cells_give_target_sentence task. What is the goal? The target usually seems nearly identical to the "reference."
• convert_linearized_input_to_sent also seems a little off because the prompt suggests the task is to summarize the entire table, but it's really summarizing the highlighted cells.