On January 4, 2024, a research paper titled “A deep learning system for predicting time to progression of diabetic retinopathy” was published in the prestigious journal Nature Medicine. The following teams jointly contributed to this article: Academician Weiping Jia and Professor Huating Li’s team from the Department of Endocrinology of Shanghai Jiao Tong University Affiliated Sixth People's Hospital and Metabolism and Shanghai Key Laboratory of Diabetes, Professor Tien Ying Wong's team from Tsinghua University School of Medicine, and Professor Bin Sheng's team from Department of Computer Science and Engineering, and MOE Key laboratory of Artificial Intelligence of Shanghai Jiao Tong University.

The team has previously developed “Deep DR” in 2021, an artificial intelligence (AI) assisted diagnostic system for diabetic retinopathy (DR), while “DeepDR Plus” stands out as a deep learning (DL) warning system for DR progression based on temporal retinal imaging. Capable of accurately predicting DR progression using retinal images over five years, the technology could contribute China’s wisdom to the global fight against diabetic complications, and provide guidance for future strategies in DR screening and prevention.

Figure1 The paper is published in Journal Nature Medicine on 2024 Jan.

DR is the most common microvascular complication of diabetes and the leading cause of preventable blindness worldwide. Notably, DR mainly develops and progresses asymptomatically in the early stages until loss of vision occurs in the later stages of disease. However, the risk of DR progression is highly variable among different individuals, due to many modifiable and non-modifiable risk factors. Currently, it is not possible to identify which patients with diabetes would develop DR or progress faster or slower.

Artificial intelligence technologies, such as DL with convolutional neural networks, have been utilized for the automated detection of DR. However, predicting DR risk remain a challenging public health issue worldwide. Diabetic patients are often only screened or followed up at relatively fixed time intervals as the exact occurrence or progression time of complications is unknown. As a result, traditional DL models cannot accurately model the temporal trajectory of disease progression, and hence cannot predict the personalized timing of disease onset and progression.

Faced with the technological challenges and great needs in clinical practice, this study developed, validated and externally tested a DL system, DeepDR Plus, to predict individualized patient trajectories for DR progression within 5 years, based on a diverse, multiethnic, multi-country dataset that comprised retinal images and clinical metadata from over 200,000 participants with diabetes.

DeepDR Plus includes a feature extractor and a predictor. The feature extractor uses ResNet-50 as the backbone to extract features from the fundus images and utilizes a soft-attention layer to select the most informative features. The predictor is proposed to generate a fundus score (reflecting the risk of DR progression) by estimating the survival time in a fixed-size mixture of Weibull distributions. With these innovative designs, DeepDR Plus could accurately predict personalized risk and time to DR progression.

Furthermore, the system was tested and validated in a real-world study within prospective cohorts of individuals with diabetes in China and India. The successful integration into clinical workflow suggests that DeepDR Plus could promote patient-specific risk assessment and thus personalized DR management, based on a single-baseline retinal image. This provides a basis for personalized screening and management decisions in future diabetic complication prevention and management.

Figure 2. The design of the DeepDR Plus system

Contributions:

Early screening and intervention are crucial for DR prevention and management. Domestic and international organizations both recommend routine annual fundus photography for diabetes patients with no or mild DR, which could timely detect retinal lesions and provide interventions. However, because of limitations in economic and medical resources, it is challenging to implement and popularize routine fundus exams for diabetic patients, especially in middle-income and low-income countries (LMICs).

Based solely on baseline fundus images, the DeepDR Plus system could accurately predict the individualized risk and timing of DR progression over the next 5 years, outperforming traditional clinical parameter models. Additionally, DeepDR Plus can accurately stratify high-risk and low-risk populations, providing AI-driven personalized recommended screening intervals (longer intervals for low-risk patients without missing vision-threatening DR) and management strategies (more intensive interventions for high-risk patients).

Incorporating AI-based personalized screening intervals into diabetic retinopathy screening systems, especially in developing countries, can greatly improve the efficiency, equity, and accessibility of retinal photography screening. This study provides new evidence for guidelines regarding screening, prevention and treatment for DR, which might impact future clinical workflows and medical costs for DR worldwide. It not only contributes China and Asia’s technology and wisdom to the global intelligent prevention and management for diabetes, but also pioneers a new pathway to improve the quality and efficiency of diabetes management for countries and regions along the Belt and Road Initiative as well as other LMICs.

Acknowledgements

These authors contributed equally: Ling Dai, Bin Sheng, Tingli Chen, Qiang Wu, Ruhan Liu, Huating Li. These authors jointly supervised this work: Bin Sheng, Carol Y. Cheung, Gavin Tan Siew Wei, Yih-Chung Tham, Ching-Yu Cheng, Huating Li, Tien Yin Wong, Weiping Jia.

This study was supported by the National Key Research and Development Program of China (2022YFA1004804) to W.J. and H.L.; the Shanghai Municipal Key Clinical Specialty, Shanghai Research Center for Endocrine and Metabolic Diseases (2022ZZ01002) and the Chinese Academy of Engineering (2022-XY-08) to W.J.; the National Key R & D Program of China (2022YFC2502800) and National Natural Science Fund of China (8238810007) to T.Y.W.; the Excellent Young Scientists Fund of NSFC (82022012), General Fund of NSFC (81870598), Innovative research team of high-level local universities in Shanghai (SHSMU-ZDCX20212700) to H.L.; the General Program of NSFC (62272298), the National Key Research and Development Program of China (2022YFC2407000), the Interdisciplinary Program of Shanghai Jiao Tong University (YG2023LC11 and YG2022ZD007), National Natural Science Foundation of China (62272298 and 62077037), the College-level Project Fund of Shanghai Jiao Tong University Affiliated Sixth People’s Hospital (ynlc201909) and the Medical-industrial Cross-fund of Shanghai Jiao Tong University (YG2022QN089) to B.S.; the National Natural Science Foundation of China (82100879) to L.W.; the Clinical Special Program of Shanghai Municipal Health Commission (20224044) and three-year action plan to strengthen the construction of public health system in Shanghai (GWVI-11.1-28) to T.C.

International Comments

Considering the importance of this achievement, Nature Medicine published a Research Briefing titled "A deep learning system for predicting time to progression of diabetic retinopathy" on the same day of the paper's publication, which summarized international academia's and Nature Medicine editors' comments and recommendations for the findings.

The renowned scholar Professor Massimo Porta from the University of Turin, Italy, noted that "The authors also address the main problem of AI — identifying which features the tool uses in its predictions."

The editors and editorial team of Nature Medicine commented that this achievement "becomes central to enabling a precision medicine approach to diabetic retinopathy prevention and early diagnosis."

Co-corresponding author: Professor Bin SHENG, Department of Computer Science and Engineering, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University

Bin Sheng is a full professor and PhD supervisor in the Department of Computer Science and Engineering, School of Electronic Information and Electrical Engineering at Shanghai Jiao Tong University. He is the deputy director of the Institute of Computer Applications and the leading PI of intelligent healthcare in the Ministry of Education Key Laboratory of Artificial Intelligence. He is also the deputy director of Shanghai Belt and Road International Joint Laboratory for Intelligent Prevention and Treatment of Metabolic Disorders. He serves as an Associate Editor for IEEE Tans. CSVT, The Visual Computer and other SCI journals, and an Associate Editor for the Chinese VR journal Virtual Reality and Intelligent Hardware (VRIH). He is also a co-chair of the program/organization committees for CGI 2023, CGI 2024 and CASA 2024. He has published 98 journal papers including Nature Medicine, Nature Communications, Cell Reports Medicine, IEEE TPAMI, and IEEE TVCG. He has published over 60 papers in top conferences including IEEE VR, ICCV, and ACM Multimedia. He received the 2023 Outstanding Contribution Award from the Computer Graphics Society, and two SAIL AWARD honors at the World Artificial Intelligence Conference.

Original Link：https://www.nature.com/articles/s41591-023-02702-z