[Elsnet-list] SemEval-2010 Second Call for Task Proposals

Katrin Erk katrin.erk at mail.utexas.edu
Fri Sep 12 16:24:44 CEST 2008

[apologies if you receive multiple copies of this announcement]

SemEval-2010: 5th International Workshop on Semantic Evaluations


The Senseval Committee invites proposals for tasks to be run as part
of SemEval-2010.  Following the spirit of the past SemEval-2007, the
nature of the tasks in previous Senseval editions has evolved to
include semantic analysis tasks outside of word sense disambiguation.

Thus we welcome proposals for any tasks that can test an automatic
system for semantic analysis of text, be it application dependent or
independent. We especially encourage tasks for different languages,
cross-lingual tasks, and tasks that are relevant to particular NLP
applications such as machine translation, information retrieval and
information extraction.

It will help us if tasks are designed in such that they can be run and
scored automatically from a central website (as in Senseval-3 and
We'll be happy to help with task design, data formatting, and so on.

Information on the design of previous Senseval-1/2/3 tasks and
SemEval-2007 can be found at:


The time period for SemEval-2010 has not yet been finalised, but it
will be held over a 1-2 month period in the first part of 2010.


Proposals for tasks will ideally contain:

 - A description of the task (about 1 page)
 - How the training/testing data will be built and/or procured
 - The evaluation methodology to be used including clear evaluation
 - The availability of the resources to the participants (copyright,
  costs, etc)
 - The resources required to prepare the task (time, money, and human)

If you are not yet at a point to provide outlines of all of these,
that's fine, but please give some thought to each, and present an
overview of your first ideas.  We will gladly give feedback.

Please submit proposals as soon as possible, preferably by electronic
mail in plain ASCII text to the SemEval-2010 chairs:

Katrin Erk, University of Texas at Austin (katrin.erk at mail.utexas.edu)
Carlo Strapparava, FBK-irst, Italy (strappa at fbk.eu)


Sep 21, 2008  Intention to submit (send a short-paragraph description)
Oct 19, 2008  Submission deadline for task proposals
Nov 16, 2008  Notification of acceptance

In SemEval-2010 we are going to give plenty of time both in tasks
preparation phase, especially regarding the tuning of data sets, and
to task participants to prepare for the task and to tune their
systems. Along this rationale, we strongly recommend that each task
organizers come up with the presentation of trial data set, well in
advance (roughly, summer 2009). Please consider this point in
designing the task schedule.

More information about the Elsnet-list mailing list