From October 30 until November 3rd, 2017, Tessa van Charldorp (UU), Tom Koole (RUG) and Wyke Stommel (RU) brought together researchers from different disciplines (sign language, conversation analysis, health communication, first language acquisition, language technology, computer vision) and developers to discuss the future development of a sustainable Advanced Video Annotation and Analysis Tool called ADVANT. The KNAW (Royal Dutch Academy of Sciences) has recently put this tool on the Dutch national scientific agenda for large research facilities as one of the facilities that should be in place in the Netherlands in 2025. The workshop was held at the Lorentz Center in Leiden which facilitates workshops that stimulate interaction amongst highly diverse researchers in terms of nationality, seniority, culture and discipline.
The workshop from Monday till Friday
On Monday the researchers presented their needs for video annotation and analysis tools from their various perspectives: from a sign language researcher (Onno Crasborn) who spends 250 minutes transcribing and annotating one minute of sign language interaction who wants to automate this process to a conversation analyst (Christian Heath) who needs ways of more accurately analyzing and visually representing the handover of surgical instruments. The participants found common language, common ground and shared their wish lists for the perfect tool.
Tuesday was the day of the developers. They shared their knowledge of how tools such as ELAN (Han Sloetjes) and speech recognition programs (Arjan van Hessen) were developed over time. The developers reflected on the various “wish lists” that were expressed from the various disciplines and methodologies represented at the workshop, including a secure pipeline for working with privacy sensitive video data. We concluded that a lot of features and functionalities exist in various programs but are not compatible with each other. A need for a modular system was established, both web and desktop based, that allows for flexibility.
On Wednesday, Julia Noordegraaf presented the already existing CLARIAH infrastructure and the Media Suite possibilities in which some of the ADVANT applications may find its future – depending on funding. We learned insightful lessons from similar German initiatives at the Institut für Deutsche Sprache from Thomas Schmidt and thought about the unique features that will make the new ADVANT tool flexible, sustainable and safe.
On Thursday we visualized the road towards having the tool developed: what societal impact will this tool have, what research questions does it help to answer, what collaborations are required and what is our timeline? The input from the previous three days came together in poster presentations, visualizations and presentations. On Friday, a small group of workshop participants started to turn all the ideas, lessons and insights from the entire week into products and started writing the beginnings of a proposal for funding.
Interested? Contact us and let’s collaborate.
Wyke Stommel, Tessa van Charldorp & Tom Koole
 Gertjan Burghouts (TNO, Netherlands), Marisa Casillas (MPI Nijmegen, Netherlands), Onno Crasborn (RU, Netherlands), Manolis Fragkiadakis (UL, Netherlands), Christian Heath (Kings College London, United Kingdom), Micha Hulsbosch (RU, Netherlands), Tom Koole (RUG, Netherlands), Marijn Koolen (Huygens ING, Netherlands), Paul Mcilvenny (Aalborg University, Denmark), Gregory Mills (RUG, Netherlands), Liliana Melgar (UvA, Netherlands), Julia Noordegraaf (UvA, Netherlands) Janneke Noordman (NIVEL, Netherlands), Jan Odijk (UU, Netherlands), Ronald Poppe (UU, Netherlands) Caroline Rowland (MPI, Netherlands), Thomas Schmidt (IDS, Mannheim, Germany), Han Sloetjes (MPI, Netherlands), Ellen Smets (AMC, The Netherlands), Michael Smith (UC Los Angeles, United States), Wyke Stommel (RU, Netherlands), Burak Tekin (University of Basel, Switzerland), Khiet Truong (University Twente, Netherlands), Tessa van Charldorp (UU, Netherlands), Arjan van Hessen (University Twente, Netherlands), Jeroen van Rest (TNO, Netherlands), Sytske Wiegersma (University Twente, Netherlands)