Résumé
Objectives: The objective of this study is the exploration of Artificial Intelligence
and Natural Language Processing techniques to support the automatic
assignment of the four Response Evaluation Criteria in Solid Tumors (RECIST)
scales based on radiology reports. We also aim at evaluating how languages and
institutional specificities of Swiss teaching hospitals are likely to affect the quality
of the classification in French and German languages.
Methods: In our approach, 7 machine learning methods were evaluated to
establish a strong baseline. Then, robust models were built, fine-tuned
according to the language (French and German), and compared with the expert
annotation.
Results: The best strategies yield average F1-scores of 90% and 86% respectively
for the 2-classes (Progressive/Non-progressive) and the 4-classes (Progressive
Disease, Stable Disease, Partial Response, Complete Response) RECIST
classification tasks.
Conclusions: These results are competitive with the manual labeling as measured
by Matthew’s correlation coefficient and Cohen’s Kappa (79% and 76%). On this
basis, we confirm the capacity of specific models to generalize on new unseen
data and we assess the impact of using Pre-trained Language Models (PLMs) on
the accuracy of the classifiers.