We are pleased to announce the first call for papers for the First Workshop on Dynamic Adversarial Data Collection (DADC), taking place on the 14th July, 2022 and co-located with NAACL 2022 in Seattle, Washington.
Dynamic Adversarial Data Collection (DADC) has been gaining traction in the NLP research community as a promising approach to improving data collection practices, model evaluation and task performance (Yang et al., 2017; Dua et al., 2019; Dinan et al., 2019; Nie et al., 2020; Bartolo et al., 2020; Kiela et al. 2021). DADC allows the dynamic collection of human-written data with models in the loop. Human annotators can be tasked with finding adversarial examples that fool current state-of-the-art models (SOTA) or they can cooperate with assistive models-in-the-loop to find interesting examples.
Recently, various efforts have shown that the DADC process can yield richer training datasets for tasks such as Question Answering (Bartolo et al., 2020; Kaushik et al., 2021), Natural Language Inference (Nie et al., 2020; Sheng et al., 2021), Sentiment Analysis (Potts et al., 2021), and Hate Speech Detection (Vidgen et al., 2021; Kirk et al., 2021). Research on DADC-based approaches has also found that it provides a more realistic evaluation setting (Ma et al., 2021), that the benefits of DADC can be scaled using synthetic data generation (Bartolo et al., 2021), that it yields more diverse training data with fewer artifacts (Wallace et al., 2021), and that generative assistants can improve both collection efficiency and effectiveness (Bartolo et al., 2021).
Building on this interest in the community, we would like to invite researchers to share their latest work in designing and understanding methods around dynamic adversarial data collection methods. We welcome work on (but not limited to) the following topics:
We will have an archival track as well as a non-archival track. Archival track submissions either go through a standard double-blind review process, or can be submitted with ARR reviews. The non-archival track seeks recently published work -- it does not need to be anonymized and will not go through the review process. The submission should clearly indicate the original venue and will be accepted if the committee thinks the work will benefit from exposure to the workshop audience. Non-archival papers will not be included in the workshop proceedings. For archival and non-archival papers, we accept short papers (4 pages of content + references) and long papers (8 pages of content + references).
To submit your paper, please see the OpenReview DADC workshop page.
We will also be running a Shared Task competition focused on better annotation, better training data and better models. For more information, see our Call for Participation.
|February 14, 2022||First Call for Workshop Papers|
|April 15, 2022||Submission deadline (papers requiring peer review)|
|May 6, 2022||Submission deadline (papers with ARR reviews)|
|May 6, 2022||Submission deadline (non-archival papers)|
|May 16, 2022||Notification of Acceptance|
|May 27, 2022||Camera-ready Papers Due|
|July 14, 2022||Workshop Date at NAACL 2022|
All deadlines are 11:59pm AOE (anywhere on earth).
Please contact the conference organizers at
email@example.com with any questions.