 |
 |
 |
 |
 |
 |
 |
 |
Task 14b: Biomedical Semantic QA
| PHASE A | |||||
| BATCH 1 | BATCH 2 | BATCH 3 | BATCH 4 | ||
| DOCUMENTS | 1st | BSRC | BSRC | BSRC | UL |
| (SELECTION STRATEGY: MAP) | 2nd | UL | UL | UL | UR |
| SNIPPETS | 1st | Fudan | MQU | UR | USTGZ-MDI |
| (SELECTION STRATEGY: F-measure) | 2nd | BSRC | BSRC | NN | UR |
| PHASE A+ | |||||
| BATCH 1 | BATCH 2 | BATCH 3 | BATCH 4 | ||
| EXACT | 1st | MQU | MQU | Fudan | UR |
| (SELECTION STRATEGY: AVG RANKING IN THREE TYPES: Y/N-MACRO F1, FACT-MRR, LIST MEAN F-MEASURE) | 2nd | UL | UL | UR | Fudan |
| IDEAL | 1st | MQU | Fudan | MQU | UR |
| (SELECTION STRATEGY: MEAN MANUAL SCORE) | 2nd | Fudan | UR | UR | MQU |
| PHASE B | |||||
| BATCH 1 | BATCH 2 | BATCH 3 | BATCH 4 | ||
| EXACT | 1st | NSUT | MQU | CSA-IISR | CSA-IISR |
| (SELECTION STRATEGY: AVG RANKING IN THREE TYPES: Y/N-MACRO F1, FACT-MRR, LIST MEAN F-MEASURE) | 2nd | CSA-IISR | UL | MQU | UL |
| IDEAL | 1st | MQU | UL | Evidence Prime | NCU-IISR |
| (SELECTION STRATEGY: MEAN MANUAL SCORE) | 2nd | Fudan | NCU-IISR | UL | UL |
Teams (Task 14b):
-
MQU team: Dima Galat and Marian-Andrei Rizoiu
(MQU: Macquarie University, Australia) -
UR team: Samy Ateia and Udo Kruschwitz
(UR: University of Regensburg, Germany) -
Fudan team: Huiying Yan, Yuyi Zhang, Yujie Dong, and Shanfeng Zhu
(Fudan: Fudan University, China) -
BSRC team: Dimitra Panou, Alexandros C. Dimopoulos, Manolis Koubarakis, and Martin Reczko
(BSRC: Biomedical Sciences Research Center “Alexander Fleming”, Greece) -
UL team: Yun Wang
(UL: University of Ljubljana, Slovenia) -
CSA-IISR team: Richard Tzong-Han Tsai, Yen-Sung Chiu, Cheng-Yu Hou, and Chen-Hau Hsu
(CSA-IISR: National Central University & Chingshin Academy, Taiwan) -
NCU-IISR team: Yi-Chieh Hung, Jen-Chieh Han, Hsi-Chuan Hung, and Richard Tzong-Han Tsai
(NCU-IISR: National Central University, Taiwan) -
Evidence Prime team: Damian Stachura, Joanna Konieczna, and Artur Nowak
(Evidence Prime, Poland) -
NSUT team: Poojan Vachharajani
(NSUT: Netaji Subhas University of Technology, India) -
USTGZ-MDI team: Ri Su and Zhao Chen
(USTGZ-MDI: Hong Kong University of Science and Technology (Guangzhou) - intelligent Medical Data Laboratory, China) -
NN team: Felix A Kruger
(NN: Novo Nordisk, Denmark)
Results at:
https://participants-area.bioasq.org/results/14b/phaseA/ https://participants-area.bioasq.org/results/14b/phaseAplus/ https://participants-area.bioasq.org/results/14b/phaseB/
Task Synergy 14: Biomedical Semantic QA for Developing Topics
| Round 1 | Round 2 | Round 3 | Round 4 | ||
| DOCUMENTS | 1st | DePaul | DePaul | Fudan | Fudan |
| (SELECTION STRATEGY: MAP) | 2nd | Fudan | Fudan | DePaul | DePaul |
| SNIPPETS | 1st | Fudan | Fudan | BSRC | Fudan |
| (SELECTION STRATEGY: F-measure) | 2nd | DePaul | DePaul | Fudan | DePaul |
| EXACT | 1st | - | DePaul | BSRC | BSRC |
| (SELECTION STRATEGY: AVG RANKING IN THREE TYPES: Y/N-MACRO F1, FACT-MRR, LIST MEAN F-MEASURE) | 2nd | - | BSRC | UR | Fudan |
| IDEAL | 1st | - | DePaul | CMU | BSRC |
| (SELECTION STRATEGY: MEAN MANUAL SCORE) | 2nd | - | Fudan | Fudan | Fudan |
Teams (Task Synergy 14):
-
Fudan team: Huiying Yan, Yuyi Zhang, Yujie Dong, and Shanfeng Zhu
(Fudan: Fudan University, China) -
DePaul team: Sukprasan Rattanapani, Alexandru Iulian Orhean, Jacob Furst, and Daniela Raicu
(DePaul: DePaul University, US) -
BSRC team: Dimitra Panou, Alexandros C. Dimopoulos, Manolis Koubarakis, and Martin Reczko
(BSRC: Biomedical Sciences Research Center “Alexander Fleming”, Greece) -
CMU team: John Gibson and Eric Nyberg
(CMU: Carnegie Mellon University, US & Liberia) -
UR team: Samy Ateia and Udo Kruschwitz
(UR: University of Regensburg, Germany)
Task MultiClinSum-2: Multilingual Clinical Summarization
The teams that topped the competition were:
| Micro F-Measure | ||
| First place | Second place | |
| MultiClinSum-en | ixa-sum | NLP4Health |
| MultiClinSum-es | ixa-sum | NLP4Health |
| MultiClinSum-fr | ixa-sum | NLP4Health |
| MultiClinSum-pt | NLP4Health | ixa-sum |
| MultiClinSum-it | ixa-sum | - |
| MultiClinSum-ca | ixa-sum | - |
| MultiClinSum-ro | ixa-sum | RaRaMi |
| MultiClinSum-de | ixa-sum | - |
| MultiClinSum-nl | ixa-sum | MediScribes |
| MultiClinSum-ru | PsychAI-Discovery-Lab | - |
| MultiClinSum-sv | PsychAI-Discovery-Lab | - |
Teams (Task MultiClinSum-2):
-
ixa-sum team: Ane Varela, Arantza Casillas, Julene Viña, Nuria Lebeña, and Maite Oronoz
(ixa-sum: University of the Basque Country, Spain) -
PsychAI-Discovery-Lab team: Muhammad Aqeel
(PsychAI-Discovery-Lab: Shifa Tameer-e-Millat University, Pakistan) -
NLP4Health team: Moutushi Roy and Dipankar Das
(NLP4Health: Jadavpur University, India) -
RaRaMi team: Radu Marin, Mihaita Lacusteanu, and Rares-Alexandru Iancu
(RaRaMi: University of Bucharest, Romania) -
MediScribes team: Maaike de Boer, Quirine Smit, and Roos Bakker
(MediScribes: TNO, Vrije Universiteit Amsterdam, & Leiden University, Netherlands)
Task BioNNE-R: Nested Relation Extraction in Russian and English
The teams that topped the competition were:
| Bilingual NRE | ||
| First place | Second place | |
| F1-score | LODAC-NII | savvafq |
| English NRE | ||
| First place | Second place | |
| F1-score | ELiRF-UPV | LODAC-NII |
| Russian NRE | ||
| First place | Second place | |
| F1-score | LODAC-NII | HSE NLP |
Teams (Task BioNNE-R):
-
LODAC-NII team: Lakshan Karunathilake, Shixiong Zhao, and Hideaki Takeda
(LODAC-NII: National Institute of Informatics, Japan) -
ELiRF-UPV team: Pere Marco-Garcia, Vicent Ahuir, Marc Hurtado-Beneyto, and Antonio Molina-Marco
(ELiRF-UPV: Universitat Politècnica de València, Spain) -
savvafq team: Savva Blinov
(savvafq: Higher School of Economics, Russia) -
HSE NLP team: Airat Valiev
(HSE NLP: Higher School of Economics, Russia)
Task ElCardioCC: Clinical Coding in Cardiology
The teams that topped the competition were:
| Multi-label Classification for Automated Clinical Coding | ||
| First place | Second place | |
| F1-score | UoM | Baymax |
Teams (Task ElCardioCC):
-
UoM team: Nikolaos Strafiotis, Panteleimon Stanimeros, Vasiliki Katsara, Georgios Chalkias, Glykeria Tsavlidou, Stelios Magalios, Effrosyni Nalmpanti, and Panteleimon Stamatakis
(UoM: University of Macedonia, Greece) -
Baymax team: Georgios Chalkias, Panagiotis Amanatiadis, Glykeria Tsavlidou, and Sotirios Malamas
(Baymax: Aristotle University of Thessaloniki and University of Macedonia, Greece)
Task GutBrainIE: Gut-Brain Interplay Information Extraction
The teams that topped the competition were:
| NER (Subtask 6.1.1) | ||
| First place | Second place | |
| Micro F1 | YesYoung | Graphwise |
| NERD (Subtask 6.1.2) | ||
| First place | Second place | |
| Micro F1 | YesYoung | Graphwise |
| Mention-level RE (Subtask 6.2.1) | ||
| First place | Second place | |
| Micro F1 | YesYoung | SMTE |
| Concept-level RE (Subtask 6.2.2) | ||
| First place | Second place | |
| Micro F1 | YesYoung | Graphwise |
Honorable mention (out of competition): the TWIX team (Marco Martinelli and Laura Menotti) achieved top results in all subtasks.
Teams (Task GutBrainIE):
-
YesYoung team: Yeyang Liu, Zhaohang Niu, and Chunyu Li
(YesYoung: University of Zurich, Switzerland & Universiti Malaya, Malaysia) -
Graphwise team: Ivelina Nikolova, Mario Kuzmanov, Aleksis Datseris, Maria Barouh, Rute Costa, Chiara Barbero, Matilde Canelas, and Margarida Ramos
(Graphwise: Graphwise, Bulgaria, NOVA University of Lisbon, Portugal, & Bulgarian Academy of Sciences, Bulgaria) -
SMTE team: Sofia Maule and Giorgio Maria Di Nunzio
(SMTE: University of Padua, Italy)
