consistency checks

[nschneid]

@mjpost pointed me to this repo as the place for the next-gen ACL publications software. Some feature requests based on experience with the Anthology:

• Warn about missing abstracts
• Specify SIGs for workshops
• For colocated events, ensure consistency of the address field by default. Currently this is a source of confusion (e.g. BlackboxNLP 2021 has "Punta Cana, Dominican Republic" in the metadata and "Online" in the PDF footer, as opposed to the main conference (EMNLP) which has "Online and Punta Cana, Dominican Republic").

[ryancotterell]

Hey Nathan!

Thanks for reaching out! The abstract warning code should probably be in the pubcheck, I think. If I understand correctly, you would like to output a warning if the author has forgotten an abstract?

The second two bullet points, again if I have understood correctly, seem harder to implement. They are consistency across workshops, is that right? I think we could do that with some additional code. Are there any other things that come to might for cross-volume consistency?

[nschneid]

Thanks for reaching out! The abstract warning code should probably be in the pubcheck, I think. If I understand correctly, you would like to output a warning if the author has forgotten an abstract?

A warning if the proceedings lacks paper abstracts for whatever reason. The Anthology has entire volumes that are missing abstracts—I don't know if the final submission form didn't collect them or they got lost at some other step in the process.

The second two bullet points, again if I have understood correctly, seem harder to implement. They are consistency across workshops, is that right? I think we could do that with some additional code. Are there any other things that come to might for cross-volume consistency?

Location is the most obvious one. I suppose if there is a formal way to indicate a main venue for colocated events (e.g. workshop at NAACL-HLT 2021) there could be a check that the name is spelled consistently.

[ryancotterell]

I wanted to revisit this now as we are actually building the proceedings for workshops. The general abstract will be that all formatting issues related to individual publications will be done with the pubcheck script. However, you also raised some issues with workshop volumes. At the moment, our build fails if you don't give a location or much of the meta-data. We are relaxing that to errors. Would that be enough?

[nschneid]

I don't know the exact workflow but it seems like the location should be provided at the event level, not the individual volume level. Otherwise you could end up with the location spelled differently in different workshops of the same conference.

[ryancotterell]

The workflow is more or less that each workshop builds its own proceedings. I see your point now. We should have an event-level check for consistency? That is certainly doable and automatable.

[crux82]

As the first version of the whole procedure is almost complete, I would like to discuss these points to check if I completely understood:

• We could add a dedicated warning if one or more papers in the papers.yml lack the abstract. @zhzhang can you add this dedicated warning?
• The information about the event (like the location) is included in the conference_details.yml. Since they can be sent separately to ACL Anthology... do you think is a check that should be applied in aclpub2 or in the import procedure? We talk with @mjpost about this...

Did I miss something?