한빛사논문
- 조회2,089
Seung Seog Han1☯, Ik Jun Moon2☯, Seong Hwan Kim3, Jung-Im Na4, Myoung Shin Kim5, Gyeong Hun Park6, Ilwoo Park7, Keewon Kim8, Woohyung Lim9, Ju Hee Lee10*, Sung Eun Chang11*
1Department of Dermatology, I Dermatology Clinic, Seoul, Korea, 2Department of Dermatology, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea, 3Department of Plastic and Reconstructive Surgery, Kangnam Sacred Hospital, Hallym University College of Medicine, Seoul, Korea, 4Department of Dermatology, Seoul National University Bundang Hospital, Seongnam, Korea, 5Department of Dermatology, Sanggye Paik Hospital, Inje University College of Medicine, Seoul, Korea, 6Department of Dermatology, Dongtan Sacred Heart Hospital, Hallym University College of Medicine, Seoul, Korea, 7Department of Radiology, Chonnam National University Medical School and Hospital, Gwangju, Korea, 8Department of Rehabilitation Medicine, Seoul National University College of Medicine, Seoul, Korea, 9LG Sciencepark, Seoul, Korea, 10Department of Dermatology, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea, 11Department of Dermatology, Asan Medical Center, Ulsan University College of Medicine, Seoul, Korea
☯ These authors contributed equally to this work.
*Corresponding author
Abstract
Background
The diagnostic performance of convolutional neural networks (CNNs) for diagnosing several types of skin neoplasms has been demonstrated as comparable with that of dermatologists using clinical photography. However, the generalizability should be demonstrated using a large-scale external dataset that includes most types of skin neoplasms. In this study, the performance of a neural network algorithm was compared with that of dermatologists in both real-world practice and experimental settings.
Methods and findings
To demonstrate generalizability, the skin cancer detection algorithm (https://rcnn.modelderm.com) developed in our previous study was used without modification. We conducted a retrospective study with all single lesion biopsied cases (43 disorders; 40,331 clinical images from 10,426 cases: 1,222 malignant cases and 9,204 benign cases); mean age (standard deviation [SD], 52.1 [18.3]; 4,701 men [45.1%]) were obtained from the Department of Dermatology, Severance Hospital in Seoul, Korea between January 1, 2008 and March 31, 2019. Using the external validation dataset, the predictions of the algorithm were compared with the clinical diagnoses of 65 attending physicians who had recorded the clinical diagnoses with thorough examinations in real-world practice.
In addition, the results obtained by the algorithm for the data of randomly selected batches of 30 patients were compared with those obtained by 44 dermatologists in experimental settings; the dermatologists were only provided with multiple images of each lesion, without clinical information.
With regard to the determination of malignancy, the area under the curve (AUC) achieved by the algorithm was 0.863 (95% confidence interval [CI] 0.852–0.875), when unprocessed clinical photographs were used. The sensitivity and specificity of the algorithm at the predefined high-specificity threshold were 62.7% (95% CI 59.9–65.1) and 90.0% (95% CI 89.4–90.6), respectively. Furthermore, the sensitivity and specificity of the first clinical impression of 65 attending physicians were 70.2% and 95.6%, respectively, which were superior to those of the algorithm (McNemar test; p < 0.0001). The positive and negative predictive values of the algorithm were 45.4% (CI 43.7–47.3) and 94.8% (CI 94.4–95.2), respectively, whereas those of the first clinical impression were 68.1% and 96.0%, respectively.
In the reader test conducted using images corresponding to batches of 30 patients, the sensitivity and specificity of the algorithm at the predefined threshold were 66.9% (95% CI 57.7–76.0) and 87.4% (95% CI 82.5–92.2), respectively. Furthermore, the sensitivity and specificity derived from the first impression of 44 of the participants were 65.8% (95% CI 55.7–75.9) and 85.7% (95% CI 82.4–88.9), respectively, which are values comparable with those of the algorithm (Wilcoxon signed-rank test; p = 0.607 and 0.097).
Limitations of this study include the exclusive use of high-quality clinical photographs taken in hospitals and the lack of ethnic diversity in the study population.
Conclusions
Our algorithm could diagnose skin tumors with nearly the same accuracy as a dermatologist when the diagnosis was performed solely with photographs. However, as a result of limited data relevancy, the performance was inferior to that of actual medical examination. To achieve more accurate predictive diagnoses, clinical information should be integrated with imaging information.
논문정보
- Research article
- 2020년 11월 (BRIC 등록일 2020-12-09)
- 국내 연구진
- 의약학 > 응용의학
댓글 작성 로그인
팝업 닫기관련 링크
관련분야 연구자보기
관련분야 논문보기
