Patch-level Double Machine Learning for Causal Interpretation of Convolutional Neural Networks in Diabetic Retinopathy Classification Frameworks

By Aniruth Ananthanarayanan, USA. 

Abstract

Diabetic retinopathy (DR) is a progressive eye complication stemming from diabetes and remains a leading cause of vision loss worldwide. Despite the high accuracy of convolutional neural networks (CNNs) in DR detection from retina images, their interpretability is often limited to saliency-based visualizations, which do not explain causal relationships between the images and DR severity. By reframing and training models on images chunked into smaller “patches,” this project proposes the integration of a novel patch-level double machine learning (DML) scheme into CNN-based DR classification pipelines to isolate and interpret the causal contributions of various image sections. By combining accurate deep learning models with econometric causal inference, this research aims to provide clinicians with a set of specific retinal patch biomarkers that causally influence DR progression and a framework for interpretable and actionable analysis for improved diagnostic decision-making.

​

1. Introduction

Diabetic retinopathy is a vision-threatening complication of diabetes affecting over 100 million individuals across the world. A 2021 study estimated that the number of adults globally with DR and vision-threatening DR (VTDR) totaled 103.12 million individuals, with this total projected to reach 160.50 million in 2045 (Teo et al.). Early-stage diagnosis and treatment is crucial to prevent further complications that could potentially result in extenuating comorbid conditions. Moreover, to translate AI-driven diagnostics into real-world practice, models must be both highly accurate and interpretable, ensuring clinicians can trust and effectively act upon their predictions. As the global diabetic population grows, the need for accurate early-stage diagnosis and intervention increases because, if left undetected, DR can progress into nearly untreatable advanced stages such as vision loss and other serious comorbidities. Current saliency methods (e.g., Grad-CAM) highlight only correlational importance and may reflect artifacts or dataset biases; we need validation techniques that go beyond correlation and identify truly causal biomarkers in retinal images to build confidence and ensure actionable insights.

2. Existing Literature

Deep learning models, particularly convolutional neural networks (CNNs), have shown remarkable performance in the classification of diabetic retinopathy from fundus photographs (Bhimavarapu & Battineni, 2022). Still, these models primarily remain “black-boxes” and function only as standalone tools as the opaqueness of these highly nonlinear models raise significant concerns in clinical settings (Tjoa & Guan, 2021). In order to address these issues, gradient-based saliency methods such as Grad-CAM and integrated gradients highlight which parts of an image were most important for a model’s prediction, effectively capturing the correlational patterns (Arun et al., 2021). However, these patterns are just that: correlational; these methods cannot distinguish causal signals from spurious ones introduced by noise, image quality artifacts, or biases in the training set (Saw et al. 2025). It is this lack of causal attribution that limits the clinical utility of these models and/or extension of the model’s learned patterns: for AI adopted in medical diagnostics, clinicians must be able to trust that the model’s focus

aligns with medically valid biomarkers rather than overfitted associations within the dataset (Joshi et al., 2025).

​

3. Methodology

To enable causal interpretation of CNNs in DR classification, we devise a novel patch-level DML framework. Our approach begins with preprocessing the EyePACS Diabetic RetinopathyDetection dataset, where high-resolution retinal fundus images are first center-cropped and resized to 2048x2048 for consistency (Dugas et al., 2015). Each image is then divided into smaller non-overlapping patches, creating a new feature representation that treats each patch as a distinct covariate. This granular view may appear similar to saliency-based heat maps, but also enables the use of causal inference by treating image regions as separate observational units, allowing us to analyze the individual causal contribution of each patch. For each patch we can then associate the CNN’s predicted DR classification with its corresponding feature vector, allowing us to model the contribution of each patch to the overall classification. To ensure that confounding between patches does not bias our estimates, we can then integrate a two-stage DML procedure based on the partially linear model (Chernozhukov et al., 2016). (Here we assume that individual patches are independent of adjacent patches, which may not necessarily be true due to spatial correlations in the retina. However, the DML framework handles this residual spatial dependence by orthogonalizing each patch’s feature representation against the remaining patches, effectively partialling out confounding before estimating causal effects.) In the first stage, we can use L2-regularized regression to predict each patch’s visual representation from the remaining patches, efficiently estimating nuisance parameters that capture spurious associations. In a second stage, we can then regress the model’s predicted DR outcome on the residual patch features obtained from the first stage to approximate the causal effect of each step. This method will be embedded within a cross-fitting loop with K-fold sample splitting to

mitigate overfitting and small-sample bias. Uncertainty will be quantified through bootstrapping

and confidence intervals for causal patch effects can be generated. To validate robustness, multiple predictive backbones (e.g., ResNet50, InceptionV3, ViT) and nuisance estimators (e.g., ridge regression, gradient boosting, multi-layer perceptrons) are tested across DML stages (He et al., 2016; Szegedy et al., 2016; Dosovitskiy et al., 2020; Yu & Zhang, 2021). Additionally, we compute the Spearman rank correlation between CNN feature importances (from SHAP or integrated gradients) and DML-derived patch effects to quantify alignment between predictive and causal saliency (Lundberg & Lee, 2017). Finally, significant patches—those with statistically non-zero causal effects—can be mapped back onto the original image to produce a heatmap of clinically relevant regions. This contrasts with existing saliency-based methods by offering causal, not correlational, visual explanations, and bridges deep learning’s predictive power with causal inference’s interpretability to enable clinically actionable insights.

4. Conclusion

This study introduces a novel paradigm that augments the predictive capabilities of convolutional neural networks with the interpretive rigor of econometric causal inference through patch-level double machine learning. By decomposing retinal images into patches and applying a rigorous DML framework we move beyond the limitations of correlation-driven saliency maps and towards a framework capable of estimating causal contributions of retinal regions to DR diagnosis. The resulting effect maps preserve the CNN’s diagnostic accuracy and uncover interpretable, statistically grounded biomarkers aligned with clinical relevance. This approach paves the way for next-generation diagnostic systems that are not only powerful but also transparent and trustworthy. By also mapping predictive models to causal drivers, we evade the pitfall of misaligned importance rankings (akin to undertriage in trauma care) as models’ internal importance features are less influenced by confounding variables. Future work could expand this framework to longitudinal datasets, multi-modal imaging, or active learning schemes that prioritize the acquisition of data from uncertain or causally ambiguous regions, pushing the boundary of what explainable AI can offer in medicine.

​

References 

​

1. Arun, N., Gaw, N., Singh, P., Chang, K., Aggarwal, M., Chen, B., Hoebel, K., Gupta, S., Patel, J., Gidwani, M., Adebayo, J., Li, M. D., & Kalpathy-Cramer, J. (2021). Assessing the Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imaging. Radiology. Artificial intelligence, 3(6), e200267. doi.org/10.1148/ryai.2021200267

​

2. Bhimavarapu, U., & Battineni, G. (2022). Deep Learning for the Detection and Classification of Diabetic Retinopathy with an Improved Activation Function. Healthcare (Basel, Switzerland), 11(1), 97. doi.org/10.3390/healthcare11010097

​

3. Chernozhukov, V ., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., & Robins, J. (2016). Double/debiased machine learning for treatment and causal parameters. arXiv preprint arXiv:1608.00060.

​

4. Dugas, E., Jared, Jorge, & Cukierski, W. (2015). Diabetic Retinopathy Detection. Retrieved from https://kaggle.com/competitions/diabetic-retinopathy-detection

​

5. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., ... & Houlsby, N. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929.

​

6. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).

​

7. Joshi, S., Urteaga, I., van Amsterdam, W. A. C., Hripcsak, G., Elias, P., Recht, B., … Ranganath, R. (2025). AI as an intervention: improving clinical outcomes relies on a causal approach to AI development and validation. Journal of the American Medical Informatics Association : JAMIA, 32(3), 589–594. doi.org/10.1093/jamia/ocae301

​

8. Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. Advances in neural information processing systems, 30.

​

9. Saw, S. N., Yan, Y . Y ., & Ng, K. H. (2025). Current status and future directions of explainable artificial intelligence in medical imaging. European journal of radiology, 183, 111884. doi.org/10.1016/j.ejrad.2024.111884

​

10. Szegedy, C., Vanhoucke, V ., Ioffe, S., Shlens, J., & Wojna, Z. (2016). Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2818-2826).

​

11. Teo, Z. L., Tham, Y . C., Yu, M., Chee, M. L., Rim, T. H., Cheung, N., Bikbov, M. M., Wang, Y . X., Tang, Y ., Lu, Y ., Wong, I. Y ., Ting, D. S. W., Tan, G. S. W., Jonas, J. B., Sabanayagam, C., Wong, T. Y ., & Cheng, C. Y . (2021). Global Prevalence of Diabetic Retinopathy and Projection of Burden through 2045: Systematic Review and Meta-analysis. Ophthalmology, 128(11), 1580–1591. doi.org/10.1016/j.ophtha.2021.04.027

​

12. Tjoa, E., & Guan, C. (2021). A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI. IEEE transactions on neural networks and learning systems, 32(11), 4793–4813. doi.org/10.1109/TNNLS.2020.3027314

​

13. Yu, Y ., & Zhang, Y . (2021). Multi-layer perceptron trainability explained via variability. arXiv preprint arXiv:2105.08911.

​