한빛사
>
한빛사논문
한빛사논문
- 조회169
Alzheimers. Dement., Apr 2025, 21 (4) | https://doi.org/10.1002/alz.70109
Addressing overfitting bias due to sample overlap in polygenic risk scoring
Seokho Jeong 1,2, Manu Shivakumar 2, Sang-Hyuk Jung 2,3, Hong-Hee Won 4, Kwangsik Nho 5,6, Heng Huang 7, Christos Davatzikos 8, Andrew J Saykin 9,10, Paul M Thompson 11, Li Shen 2, Young Jin Kim 12, Bong-Jo Kim 12, Seunggeun Lee 1, Dokyoon Kim 2,13
1Graduate School of Data Science, Seoul National University, Seoul, Republic of Korea.
2Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA.
3Department of Medical Informatics, Kangwon National University, College of Medicine, Chuncheon, Republic of Korea.
4Samsung Advanced Institute for Health Sciences and Technology (SAIHST), Samsung Medical Center, Sungkyunkwan University, Seoul, Republic of Korea.
5Indiana Alzheimer's Disease Research Center, Indiana University School of Medicine, Indianapolis, Indiana, USA.
6Center for Computational Biology and Bioinformatics, Indiana University School of Medicine, Indianapolis, Indiana, USA.
7Department of Electrical and Computer Engineering, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.
8Center for Biomedical Image Computing and Analytics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA.
9Department of Radiology and Imaging Sciences, and Indiana Alzheimer Disease Center, Indiana University School of Medicine, Indianapolis, Indiana, USA.
10Department of Medical and Molecular Genetics, Indiana University School of Medicine, Indianapolis, Indiana, USA.
11Imaging Genetics Center, Laboratory of Neuro Imaging, Department of Neurology & Psychiatry, UCLA School of Medicine, Los Angeles, California, USA.
12Division of Genome Science, Department of Precision Medicine, National Institute of Health, Cheongju, Republic of Korea.
13Institute for Biomedical Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA.
Seokho Jeong and Manu Shivakumar contributed equally and should be regarded as joint first authors.
Correspondence : Seunggeun Lee, Dokyoon Kim
1Graduate School of Data Science, Seoul National University, Seoul, Republic of Korea.
2Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA.
3Department of Medical Informatics, Kangwon National University, College of Medicine, Chuncheon, Republic of Korea.
4Samsung Advanced Institute for Health Sciences and Technology (SAIHST), Samsung Medical Center, Sungkyunkwan University, Seoul, Republic of Korea.
5Indiana Alzheimer's Disease Research Center, Indiana University School of Medicine, Indianapolis, Indiana, USA.
6Center for Computational Biology and Bioinformatics, Indiana University School of Medicine, Indianapolis, Indiana, USA.
7Department of Electrical and Computer Engineering, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.
8Center for Biomedical Image Computing and Analytics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA.
9Department of Radiology and Imaging Sciences, and Indiana Alzheimer Disease Center, Indiana University School of Medicine, Indianapolis, Indiana, USA.
10Department of Medical and Molecular Genetics, Indiana University School of Medicine, Indianapolis, Indiana, USA.
11Imaging Genetics Center, Laboratory of Neuro Imaging, Department of Neurology & Psychiatry, UCLA School of Medicine, Los Angeles, California, USA.
12Division of Genome Science, Department of Precision Medicine, National Institute of Health, Cheongju, Republic of Korea.
13Institute for Biomedical Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA.
Seokho Jeong and Manu Shivakumar contributed equally and should be regarded as joint first authors.
Correspondence : Seunggeun Lee, Dokyoon Kim
Abstract
Introduction: Numerous studies on Alzheimer's disease polygenic risk scores (PRSs) overlook sample overlap between International Genomics of Alzheimer's Project (IGAP) and target datasets like Alzheimer's Disease Neuroimaging Initiative (ADNI).
Methods: To address this, we developed overlap-adjusted PRS (OA PRS) and tested it on simulated data to assess biases from different scenarios by varying training, testing, and overlap proportions. OA PRS was used to adjust for sample bias in simulations; then, we applied OA PRS to IGAP and ADNI datasets and validated through visual diagnosis.
Results: OA PRS effectively adjusted for sample overlap in all simulation scenarios, as well as for IGAP and ADNI. The original IGAP PRS showed an inflated area under the receiver operating characteristic (AUROC: 0.915) on overlapping samples. OA PRS reduced the AUROC to 0.726, closely aligning with the AUROC of non-overlapping samples (0.712). Further, visual diagnostics confirmed the effectiveness of our adjustments.
Discussion: With OA PRS, we were able to adjust the IGAP summary-based PRS for the overlapped ADNI samples, allowing the dataset to be fully used without the risk of overfitting.
Highlights: Sample overlap between large Alzheimer's disease (AD) cohorts poses overfitting bias when using AD polygenic risk scores (PRSs). This study highlighted the effectiveness of overlap-adjusted PRS (OA -PRS) in mitigating overfitting and improving the accuracy of PRS estimations. New PRSs based on adjusted effect sizes showed increased power in association with clinical features.
Methods: To address this, we developed overlap-adjusted PRS (OA PRS) and tested it on simulated data to assess biases from different scenarios by varying training, testing, and overlap proportions. OA PRS was used to adjust for sample bias in simulations; then, we applied OA PRS to IGAP and ADNI datasets and validated through visual diagnosis.
Results: OA PRS effectively adjusted for sample overlap in all simulation scenarios, as well as for IGAP and ADNI. The original IGAP PRS showed an inflated area under the receiver operating characteristic (AUROC: 0.915) on overlapping samples. OA PRS reduced the AUROC to 0.726, closely aligning with the AUROC of non-overlapping samples (0.712). Further, visual diagnostics confirmed the effectiveness of our adjustments.
Discussion: With OA PRS, we were able to adjust the IGAP summary-based PRS for the overlapped ADNI samples, allowing the dataset to be fully used without the risk of overfitting.
Highlights: Sample overlap between large Alzheimer's disease (AD) cohorts poses overfitting bias when using AD polygenic risk scores (PRSs). This study highlighted the effectiveness of overlap-adjusted PRS (OA -PRS) in mitigating overfitting and improving the accuracy of PRS estimations. New PRSs based on adjusted effect sizes showed increased power in association with clinical features.
논문정보
- Research article
- 2025년 04월 (BRIC 등록일 2025-04-08)
- 국내(교신)+국외 연구진
- 생명과학 > 유전학
관련 보도자료
다유전자 위험점수에서 표본 중복에 의한 편향 효과에 관한 연구
머신러닝 모델을 활용한 질병 위험도 예측에서 신뢰성 향상을 위해서는 편향 효과를 제거해야 한다. 대규모 바이오뱅크와 치매 관련 대형 코호트(International Genomics of Alzheimer's Project, IGAP, Alzheimer's Disease Neuroimaging Initiative, ADNI)의 발전으로 유전체 기반 치매 위험도 예측 모델인 다유전자 위험점수(PRS) 개발이 활발하게 진행되고 있다. 그러나 서로 다른 코호트 간 표본 중복 현상이 증가하면서, 이로 인한 편향 효과가 위험점수의 신뢰성을 저해할 가능성이 제기되고 있다. 따라서 이러한 문제를 면밀히 분석하고 해결하기 위한 구체적인 연구가 필요하다.
- Bio통신원
- 2025. 04. 17
댓글 작성 로그인
팝업 닫기관련 링크
관련분야 연구자보기
소속기관 논문보기
관련분야 논문보기
