[Elsnet-list] GENERATION CHALLENGES 2009: Announcement and First Call for Task Proposals

Gatt, A. a.gatt at abdn.ac.uk
Wed Apr 2 20:34:51 CEST 2008

Announcement and First Call for Task Proposals





To be held in conjunction with ENLG 2009.


Following the NSF/SIGGEN Workshop on Shared Tasks and Comparative
Evaluation in NLG in April 2007
(http://www.ling.ohio-state.edu/~mwhite/nlgeval07/), and the INLG'06
Special Session on Sharing Data and Comparative Evaluation

there has recently been a lot of activity involving shared tasks in
Natural Language Generation (NLG).  The Pilot Attribute Selection for
Generating Referring Expressions (ASGRE) Challenge took place between
May and September 2007; the Referring Expression Generation (REG)

2008 is currently underway; and other challenges are being prepared.


To provide a common forum for these activities, we are organising
Generation Challenges 2009, an umbrella event designed to bring together
a variety of shared-task evaluation efforts that involve the generation
of natural language.  The following is a provisional list of tasks that
have been proposed so far:


1. The GIVE Challenge (Koller, Byron, Moore, Oberlander & Striegnitz):

   Generation of natural-language instructions to aid human task-solving

   in a virtual environment.


2. GREC Task (Belz, Gatt & Viethen): Generation of references to named

   entities in discourse; exact task definition to be announced.


3. TUNA Progress Test (Gatt & Belz): An opportunity to improve on

   REG'08 Task 3, i.e. mapping from TUNA domain representations to

   referring expressions.


At this point, we invite any team of researchers interested in
organising a shared task as part of the Generation Challenges 2009 event
to submit a task proposal.


This is not intended to be a selective process.  Anyone interested in
running a shared task as part of Generation Challenges 2009 is in
principle welcome to do so.  We would simply like to get an idea of how
many tasks there are likely to be for organisational purposes, and also
to be able to offer advice and support where needed.  In the unlikely
event that more tasks are proposed than can be accommodated, we would
make a selection together with the ENLG'09 organisers.


Each team of task organisers will be reponsible for organising and
evaluating their task.  The Generation Challenges 2009 coordinators will
merely coordinate issues relating to the ENLG'09 special session,
presentations and reports.


Please send task proposals to gre-stec at itri.brighton.ac.uk by 31 May
2008.  Please bear in mind the timeline below, and include information
regarding organisers, task description, size and state of completion of
data to be used, evaluation plans and provisional timeline.


Don't hesitate to get in touch with any questions.  Best wishes,


Anja Belz and Albert Gatt



Generation Challenges 2009 Timeline:


Deadline for submission of task proposals 31 May 2008

Recommended first call for participation  Jun 2008

Recommended availability of training data Jul-Aug 2008

Recommended evaluation period             Feb 2009

Expected deadline for camera-ready reports      end Feb 2009

ENLG'09     (provisional)                       30-31 March 2009


The University of Aberdeen is a charity registered in Scotland, No SC013683.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://stratus.let.uu.nl/pipermail/elsnet-list/attachments/20080402/f148ebf0/attachment-0001.htm

More information about the Elsnet-list mailing list