EVALITA 2016 call for tasks and important dates

The Call for Tasks for EVALITA 2016 is out together with the important dates of the initiative.

The Submission deadline for task proposals is Sunday, 21 February 2016.

Tasks proposals should be no longer than 4 pages and should include:

task title and acronym
names and affiliation of the organizers (minimum 2 organizers)
brief task description, including motivations and state of the art showing the international relevance of the task
description and examples of the data needed for the task, including information about their availability and development stage
expected number of participants and attendees
names and contact information of the organizers

In submitting your proposal, please bear in mind that we encourage:

the re-annotation of datasets from previous years with new annotation types or texts from publicly available corpora
application-oriented tasks, that is tasks that have a clearly defined end-user application showcasing
both new tasks and re-runs. For new tasks, you will have to specify in the proposal why this would attract a reasonable number of participants, and why is needed. For a re-run, you should defend in your proposal the need for having such task again (new metrics, new types of data, new domains, growing interest, etc.)

The organizers of the accepted tasks should take care of planning, according to the scheduled deadlines (see below):

the development and distribution of data sets needed for the contest, i.e. data for training and development, and data for testing; the scorer to be used to evaluate the submitted systems should be included in the release of development data
the development of task guidelines, where all the instructions for the participation are made clear together with a detailed description of data and evaluation metrics applied for the evaluation of the participant results; the collection of participants results
the evaluation of participants results according to standard metrics
the solicitation of submissions
the reviewing process of the papers describing the participants approach and results (according to the template to be made available by the EVALITA-2016 chairs)
the production of a paper describing the task (according to the template to be made available by the EVALITA-2016 chairs)

Email your proposal in PDF format to evalita2016[AT}gmail.com with “Evalita 2016 TASK Proposal” as the subject line by the submission deadline (21st February 2016).

Please feel free to contact the EVALITA-2016 chairs at evalita2016[AT]gmail.com in case of any questions or suggestions.