[Elsnet-list] GeoCLEF - Web query parsing task using 800, 000 actual Web queries

Mark Sanderson m.sanderson at sheffield.ac.uk
Mon Feb 19 13:52:11 CET 2007


This year, GeoCLEF will tackle the problem of query parsing. This 
sub-task will be run by Microsoft Research Asia who will be supplying 
a substantial set of Web queries to geo-parse. It is hoped that 
800,000 queries will be provided for any participant of GeoCLEF.

A geographic query is usually composed of three components, "what", 
"geo-relation" and "where". How to parse queries and extract these 
components from them is a key problem for geographic information 
retrieval (GIR). Therefore, there will be a geographic query parsing 
task for GeoCLEF 2007.

The contest is open to any party planning to attend CLEF 2007.

We will evaluate the submitted result based on several criteria, 
including precision, recall, and F1-score. We will use multiple human 
editors to tag a subset of queries selected from the total test set. 
The collection of human editors is assumed to have the most complete 
knowledge about Internet as compared with any individual end user. 
You will not know which queries will be used for evaluation and are 
asked to categorize all queries given.

The evaluation will run on the selected test queries and rank your 
results by how closely they match to the results from human editors. 
Here are the set of measures we will use to evaluate results 
submitted by the contestants: For more information, please see the Web site


(Main geoCLEF web site http://ir.shef.ac.uk/geoclef/)

If you're interested in registering for the query parsing track, 
please contact Mark Sanderson (m.sanderson at shef.ac.uk)

Mark Sanderson
Reader in Information Retrieval

Room 225
Department of Information Studies
University of Sheffield, Regent Court
Portobello St, Sheffield, S1 4DP, UK

Tel: +44 (0) 114 22 22648
Fax: +44 (0) 114 27 80300
mailto:m.sanderson at shef.ac.uk

Good judgement comes from experience, experience comes from bad judgement
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://stratus.let.uu.nl/pipermail/elsnet-list/attachments/20070219/ecc55a08/attachment.html

More information about the Elsnet-list mailing list