Dynamic Adversarial Data Collection (DADC) Workshop at NAACL 2022

The First Workshop on Dynamic Adversarial Data Collection (DADC) at NAACL 2022 in Seattle, Washington.

Invited Speakers & Program
Shared Task
Call for Papers
Submit Paper
Sign up for notifications

Call for Papers

We are pleased to announce the first call for papers for the First Workshop on Dynamic Adversarial Data Collection (DADC), taking place on the 14th July, 2022 and co-located with NAACL 2022 in Seattle, Washington.

Dynamic Adversarial Data Collection (DADC) has been gaining traction in the NLP research community as a promising approach to improving data collection practices, model evaluation and task performance (Yang et al., 2017; Dua et al., 2019; Dinan et al., 2019; Nie et al., 2020; Bartolo et al., 2020; Kiela et al. 2021). DADC allows the dynamic collection of human-written data with models in the loop. Human annotators can be tasked with finding adversarial examples that fool current state-of-the-art models (SOTA) or they can cooperate with assistive models-in-the-loop to find interesting examples.

Recently, various efforts have shown that the DADC process can yield richer training datasets for tasks such as Question Answering (Bartolo et al., 2020; Kaushik et al., 2021), Natural Language Inference (Nie et al., 2020; Sheng et al., 2021), Sentiment Analysis (Potts et al., 2021), and Hate Speech Detection (Vidgen et al., 2021; Kirk et al., 2021). Research on DADC-based approaches has also found that it provides a more realistic evaluation setting (Ma et al., 2021), that the benefits of DADC can be scaled using synthetic data generation (Bartolo et al., 2021), that it yields more diverse training data with fewer artifacts (Wallace et al., 2021), and that generative assistants can improve both collection efficiency and effectiveness (Bartolo et al., 2021).


Building on this interest in the community, we would like to invite researchers to share their latest work in designing and understanding methods around dynamic adversarial data collection methods. We welcome work on (but not limited to) the following topics:

Format and Submissions

We will have an archival track as well as a non-archival track. Archival track submissions either go through a standard double-blind review process, or can be submitted with ARR reviews. The non-archival track seeks recently published work -- it does not need to be anonymized and will not go through the review process. The submission should clearly indicate the original venue and will be accepted if the committee thinks the work will benefit from exposure to the workshop audience. Non-archival papers will not be included in the workshop proceedings. For archival and non-archival papers, we accept short papers (4 pages of content + references) and long papers (8 pages of content + references).

The OpenReview DADC workshop page is now open and accepting paper submissions. Papers should follow the official *ACL Paper Styles and paper formatting guidelines (also see Overleaf template).

To submit your paper, please see the OpenReview DADC workshop page.

Submitting through OpenReview

DADC Shared Task

We will also be running a Shared Task competition focused on better annotation, better training data and better models. For more information, see our Call for Participation.

Important Dates

February 14, 2022First Call for Workshop Papers
April 15, 2022Submission deadline (papers requiring peer review)
May 6, 2022Submission deadline (papers with ARR reviews)
May 6, 2022Submission deadline (non-archival papers)
May 16, 2022Notification of Acceptance
May 27, 2022Camera-ready Papers Due
July 14, 2022Workshop Date at NAACL 2022

All deadlines are 11:59pm AOE (anywhere on earth).


Please contact the conference organizers at dadc-workshop@googlegroups.com with any questions.