[Elsnet-list] GENERATION CHALLENGES 2009: Announcement and First
Call for Task Proposals
a.gatt at abdn.ac.uk
Wed Apr 2 20:34:51 CEST 2008
Announcement and First Call for Task Proposals
GENERATION CHALLENGES 2009
To be held in conjunction with ENLG 2009.
Following the NSF/SIGGEN Workshop on Shared Tasks and Comparative
Evaluation in NLG in April 2007
(http://www.ling.ohio-state.edu/~mwhite/nlgeval07/), and the INLG'06
Special Session on Sharing Data and Comparative Evaluation
there has recently been a lot of activity involving shared tasks in
Natural Language Generation (NLG). The Pilot Attribute Selection for
Generating Referring Expressions (ASGRE) Challenge took place between
May and September 2007; the Referring Expression Generation (REG)
2008 is currently underway; and other challenges are being prepared.
To provide a common forum for these activities, we are organising
Generation Challenges 2009, an umbrella event designed to bring together
a variety of shared-task evaluation efforts that involve the generation
of natural language. The following is a provisional list of tasks that
have been proposed so far:
1. The GIVE Challenge (Koller, Byron, Moore, Oberlander & Striegnitz):
Generation of natural-language instructions to aid human task-solving
in a virtual environment.
2. GREC Task (Belz, Gatt & Viethen): Generation of references to named
entities in discourse; exact task definition to be announced.
3. TUNA Progress Test (Gatt & Belz): An opportunity to improve on
REG'08 Task 3, i.e. mapping from TUNA domain representations to
At this point, we invite any team of researchers interested in
organising a shared task as part of the Generation Challenges 2009 event
to submit a task proposal.
This is not intended to be a selective process. Anyone interested in
running a shared task as part of Generation Challenges 2009 is in
principle welcome to do so. We would simply like to get an idea of how
many tasks there are likely to be for organisational purposes, and also
to be able to offer advice and support where needed. In the unlikely
event that more tasks are proposed than can be accommodated, we would
make a selection together with the ENLG'09 organisers.
Each team of task organisers will be reponsible for organising and
evaluating their task. The Generation Challenges 2009 coordinators will
merely coordinate issues relating to the ENLG'09 special session,
presentations and reports.
Please send task proposals to gre-stec at itri.brighton.ac.uk by 31 May
2008. Please bear in mind the timeline below, and include information
regarding organisers, task description, size and state of completion of
data to be used, evaluation plans and provisional timeline.
Don't hesitate to get in touch with any questions. Best wishes,
Anja Belz and Albert Gatt
Generation Challenges 2009 Timeline:
Deadline for submission of task proposals 31 May 2008
Recommended first call for participation Jun 2008
Recommended availability of training data Jul-Aug 2008
Recommended evaluation period Feb 2009
Expected deadline for camera-ready reports end Feb 2009
ENLG'09 (provisional) 30-31 March 2009
The University of Aberdeen is a charity registered in Scotland, No SC013683.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Elsnet-list