Yongju Lee ^{# 1}, Jeong Hwan Park ^{# 2 3}, Sohee Oh ^{# 4}, Kyoungseob Shin ^{# 1}, Jiyu Sun ⁴, Minsun Jung ^{2 5}, Cheol Lee ^{2 6}, Hyojin Kim ^{2 7}, Jin-Haeng Chung ^{2 7}, Kyung Chul Moon ^{8 9}, Sunghoon Kwon ^{10 11 12 13 14 15}

¹Department of Electrical and Computer Engineering, Seoul National University, Seoul, Republic of Korea.
²Department of Pathology, Seoul National University College of Medicine, Seoul, Republic of Korea.
³Department of Pathology, SMG-SNU Boramae Medical Center, Seoul, Republic of Korea.
⁴Medical Research Collaborating Center, SMG-SNU Boramae Medical Center, Seoul, Republic of Korea.
⁵Department of Pathology, Severance Hospital, Yonsei University College of Medicine, Seoul, Republic of Korea.
⁶Department of Pathology, Seoul National University Hospital, Seoul, Republic of Korea.
⁷Department of Pathology and Translational Medicine, Seoul National University Bundang Hospital, Seongnam, Republic of Korea.
⁸Department of Pathology, Seoul National University College of Medicine, Seoul, Republic of Korea. 
⁹Department of Pathology, Seoul National University Hospital, Seoul, Republic of Korea. 
¹⁰Department of Electrical and Computer Engineering, Seoul National University, Seoul, Republic of Korea. 
¹¹Interdisciplinary Program in Bioengineering, Seoul National University, Seoul, Republic of Korea. 
¹²Bio-MAX Institute, Seoul National University, Seoul, Republic of Korea. 
¹³BK21+ Creative Research Engineer Development for IT, Seoul National University, Seoul, Republic of Korea. 
¹⁴Biomedical Research Institute, Seoul National University, Seoul, Republic of Korea.
¹⁵Institutes of Entrepreneurial BioConvergence, Seoul National University, Seoul, Republic of Korea. 
^#Contributed equally.

These authors contributed equally: Yongju Lee, Jeong Hwan Park, Sohee Oh, Kyoungseob Shin.
Corresponding authors: Correspondence to Kyung Chul Moon or Sunghoon Kwon.

Methods of computational pathology applied to the analysis of whole-slide images (WSIs) do not typically consider histopathological features from the tumour microenvironment. Here, we show that a graph deep neural network that considers such contextual features in gigapixel-sized WSIs in a semi-supervised manner can provide interpretable prognostic biomarkers. We designed a neural-network model that leverages attention techniques to learn features of the heterogeneous tumour microenvironment from memory-efficient representations of aggregates of highly correlated image patches. We trained the model with WSIs of kidney, breast, lung and uterine cancers and validated it by predicting the prognosis of 3,950 patients with these four different types of cancer. We also show that the model provides interpretable contextual features of clear cell renal cell carcinoma that allowed for the risk-based retrospective stratification of 1,333 patients. Deep graph neural networks that derive contextual histopathological features from WSIs may aid diagnostic and prognostic tasks.