Comparison of RETFound and a Supervised Convolutional Neural Network for Detection of Referable Glaucoma from Fundus Photographs.

PURPOSE

To compare the performance of a vision transformer-based foundation model (RETFound) and a supervised convolutional neural network (VGG-19) for detecting referable glaucoma from fundus photographs.

DESIGN

An evaluation of diagnostic technology.

PARTICIPANTS

Six thousand one hundred sixteen participants from the Los Angeles County Department of Health Services Teleretinal Screening Program.

METHODS

Fundus photographs were labeled for referable glaucoma (cup-to-disc ratio ≥0.6) by certified optometrists. Four deep learning models were trained on cropped and uncropped images (training N = 8996; validation N = 3002) using 2 architectures: RETFound, a vision transformer with self-supervised pretraining on fundus photographs, and VGG-19. Models were evaluated on a held-out test set (N = 1000) labeled by glaucoma specialists and an external test set (N = 300) from University of Southern California clinics. Performance was assessed while varying training set size and stratifying by demographic factors. xRAI was used for saliency mapping.

MAIN OUTCOME MEASURES

Area under the receiver operating characteristic curve (AUC-ROC) and threshold-specific metrics.

RESULTS

The cropped image VGG-19 model achieved the highest AUC-ROC (0.924 [0.907-0.940]), which was comparable (= 0.07) to the cropped image RETFound model (0.911 [0.892-0.930]), which achieved the highest Youden-optimal performance (sensitivity 82.6% and specificity 88.2%) and F1 score (0.801). Cropped image models outperformed their uncropped counterparts (RETFound 0.889 [0.868-0.909], VGG-19 0.898 [0.879-0.917]) within each architecture (< 0.001 for AUC-ROC comparisons). The uncropped image RETFound model performed best on external data (0.886 [0.849-0.924] vs. the next-highest 0.797 [0.746-0.848],< 0.001 for AUC-ROC comparisons). RETFound models had a performance advantage when trained on smaller datasets (N < 2000 images), and the cropped image RETFound model performed consistently across ethnic groups (= 0.20), whereas the others did not (< 0.04). Performance did not vary by age or gender. Saliency maps for both architectures consistently included the optic nerve.

CONCLUSIONS

Although both RETFound and VGG-19 models performed well for classification of referable glaucoma, foundation models may be preferable when training data are limited and when domain shift is expected. Training models using images cropped to the region of the optic nerve improves performance regardless of architecture but may reduce model generalizability.

FINANCIAL DISCLOSURES

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.