An observer comparison study to evaluate a machine learning model to quantify the infected pneumonia on lung CT images

Alireza Abdihamzehkolaei, Seyedehnafiseh Mirniaharikandehei, Angel Choquehuanca, Marco Aedo, Wilmer Pacheco, Laura Estacio, Victor Cahui, Luis Huallpa, Kevin Quiñonez, Valeria Calderón, Ana Maria Gutierrez, Ana Vargas, Dery Gamero, Eveling Castro-Gutierrez, Yuchen Qiu, Bin Zheng, Javier A. Jo

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


Quantification of infected lung volume using computed tomography (CT) images can play a critical role in predicting the severity of pulmonary infectious disease. Manual segmentation of infected areas from several CT image slices, however, is not efficient and viable in clinical practice. To assist clinicians in overcoming this challenge, we developed a new method to automatically segment and quantify the percentage of the infected lung volume. First, we used a public dataset of 20 COVID-19 patients, which consists of manually annotated lung and infection masks, to train a new joint deep learning (DL) model for lung and infection segmentation. As for lung segmentation, a Mask-RCNN model was applied to the lung volume with a novel postprocessing technique. Following that, an ensemble model with a customized residual attention UNet model and feature pyramid network (FPN) models was employed for infection segmentation. Next, we assembled another set of 80 CT scans of Covid-19 patients. Two chest radiologists manually evaluated each CT scan and reported the infected lung volume percentage using a customized graphical user interface (GUI). The developed DL-model was also employed to process these CT images. Then, we compared the agreement between the radiologist (manual) and model-based (automated) percentages of diseased regions. Additionally, the GUI was used to let radiologists rate acceptance of the DL-model generated segmentation results. Analyzing the results demonstrate that the agreement between manual and automated segmentation is >95% in 28 testing cases. Furthermore, >53% of testing cases received the top assessment rating scores from two radiologists (between four-five- score). Thus, this study illustrates the feasibility of developing a DL-model based automated tool to effectively provide quantitative evaluation of infected lung regions to assist in improving the efficiency of radiologists in infection diagnosis.

Original languageEnglish
Title of host publicationMedical Imaging 2023
Subtitle of host publicationImage Perception, Observer Performance, and Technology Assessment
EditorsClaudia R. Mello-Thoms, Yan Chen
ISBN (Electronic)9781510660397
StatePublished - 2023
EventMedical Imaging 2023: Image Perception, Observer Performance, and Technology Assessment - San Diego, United States
Duration: 21 Feb 202323 Feb 2023

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
ISSN (Print)1605-7422


ConferenceMedical Imaging 2023: Image Perception, Observer Performance, and Technology Assessment
Country/TerritoryUnited States
CitySan Diego

Bibliographical note

Publisher Copyright:
© COPYRIGHT SPIE. Downloading of the abstract is permitted for personal use only.


  • Assessment of machine learning models
  • COVID-19 detections
  • COVID-19 severity assessment
  • Lung disease quantification
  • Observer comparison study
  • Observer preference study
  • Segmentation of pneumonia regions


Dive into the research topics of 'An observer comparison study to evaluate a machine learning model to quantify the infected pneumonia on lung CT images'. Together they form a unique fingerprint.

Cite this