BioASQ challenge on large-scale biomedical semantic indexing and question answering
(BioASQ workshop to be collocated with CLEF 2013 in Valencia, Spain on September 27, 2013)
The BioASQ challenge aims to push for a solution to the information access problem of biomedical experts. It will evaluate the ability of systems to perform various tasks in the biomedical QA process:
1. large-scale classification of biomedical documents onto ontology concepts (semantic indexing),
2. classification of biomedical questions onto relevant concepts,
3. retrieval of relevant document snippets, concepts and knowledge base triples, and
4. delivery of the retrieved information in a concise and user-understandable form.
In particular, the challenge will comprise two main tasks:
Task 1a. Large-scale online biomedical semantic indexing
Large-scale semantic indexing will be evaluated on the whole of PubMed. In particular, participants will be asked to classify incoming documents before the human curators do:
* BioASQ will distribute new unclassified PubMed documents.
* Participants will have a limited response time to attach MeSH terms.
Task 1a will run for three consecutive periods (batches) of 6 weeks each. The first batch will start on April 15, 2013. Separate winners will be announced for each batch. Participation in the task can be partial, i.e. in some of the batches.
Task 1b. Biomedical semantic QA
The systems will be evaluated against gold answers created by a team of biomedical experts. The task will run in two phases:
In Phase A:
* BioASQ will transmit simultaneously questions from the benchmark.
* Participants will have limited time to respond with concepts, snippets, triples.
In Phase B:
* BioASQ will distribute questions + concepts, snippets, triples.
* Participants will respond with facts, summaries, etc.
The two phases will run consecutively, i.e. Phase B will start after the end of phase A. Phase A will start on June 3, 2013. Separate winners will be announced for each phase and each task target, i.e. concepts, snippets, triples, etc. Participation in the task can be partial, i.e. in one of the two phases and only for some of the targets.
Prizes will be awarded to the winners. Details about the prizes will be announced on the Web site of the challenge.
March 18, 2013: Training data available for task 1a. Dry-run data available for task 1b.
April 15, 2013: Start of task 1a.
June 3, 2013: Start of task 1b.
July 15, 2013: Submission of papers for BioASQ workshop
August 30, 2013: End of challenge
September 27, 2013: BioASQ workshop, collocated with CLEF 2013
The BioASQ challenge and workshop are organised by the BioASQ project, supported by the European Commission within the 7th Framework Programme (Grant Agreement No. 318652).