The resolution of immunofluorescent pathological images affects diagnosis for not only artificial intelligence but also human

Implication for health policy/practice/research/medical education: We examined how the image resolution affects the diagnosis not only artificial intelligence but also nephrologists in this study. The differences between human and artificial intelligence is specificity on diiferent resolution image diagnosis. The resolution of images might be important for not artificial intelligence but human on the point of specificity. Please cite this paper as: Takahashi K, Kitamura S, Fukushima K, Sang Y, Tsuji K, Wada J. The resolution of immunofluorescent pathological images affects diagnosis for not only artificial intelligence but also human. J Nephropathol. 2021;10(3):e26. DOI: 10.34172/jnp.2021.26. ARTICLE INFO

staining images, light microscope images and electron microscope images. On IgA nephropathy, the diagnosis was performed only from pathological images because it is necessary to observe the deposition of IgA immune complex in glomeruli. Thus, the pathological diagnostic images are very important for definitive diagnosis.
On the other hand, various technologies have emerged in the medical world, and the medical environment is changing. Artificial intelligence has already entered the fields of medical research and clinical practice, and various applications have been made (1). Artificial intelligence is expected to deal with big data, and contribute to real world. Image diagnosis is one of the fields that artificial intelligence is often applied (2). Image analysis of artificial intelligence is divided into three categories; image classification (3), image detection (4), and image segmentation (5). AI image diagnosis from renal pathological images has been attempted. Hermsen et al reported that deep learning could assess renal histopathological images in kidney tissue on the point of image segmentation (6). Ginley et al reported that artificial intelligence could diagnose segmentation and classification of diabetic glomerulosclerosis (7). There are many reports that artificial intelligence could contribute to diagnose from medical images. For human, the resolution of medical images are important because the high resolution images give much information for human to diagnose clearly. However, there has been few reports that discuss how the image resolution affect the differences between human and artificial intelligence.

Objectives
We studied whether the different resolution of immunofluorescent images of IgA nephropathy affect to diagnosis for human and artificial intelligence.

Patients information
This study is a retrospective observational study. As for informed consent, the contents of the research are posted on our department homepage and in the hospital, and public informed consent is given. Kidney samples consisted of needle biopsy samples from patients that hospitalized to undergo the examination of renal biopsy in Okayama university hospital from January 2008 to May 2018. The number of renal biopsy patients is 885 from 2012-2018 in Okayama university hospital. During 2012-2018, the number of IgA nephropathy renal biopsy patients is 162 in Okayama university hospital. For comparison, 723 non-IgA nephropathy patients' images were collected during 2012-2018. We excluded 463 patients because of renal transplantation episode biopsy patients, complicated multiple nephropathy and uncertain diagnosis. After that the exclusion, we enrolled 260 non-IgA nephropathy patients for comparison. The representative validation images from the patients with IgA nephropathy were shown in Figure S8 (Supplementary file 1), and representative validation images from the patients with non-IgA nephropathy patients were shown in supplemental figures S9. Almost 40% of renal biopsy patients are IgA nephropathy patients (Table S1, Supplementary file 1). The diagnosis of the renal disease was decided by the nephrologists with the discussion of conference in the light of medical history, physical information, laboratory data and pathological findings including of immunofluorescence, light microscopy and electron microscopy images.

Renal immunofluorescent images
The kidney samples were obtained by renal biopsy from the patients administrated in Okayama University hospital. We prepared frozen tissues from renal biopsy samples, and cut at 4 μm in a cryostat. We stained the frozen sections by fluorescein isothiocyanate (FITC)-conjugated antibodies in moist chamber for one hour. FITC-conjugated goat anti-human IgA was purchased from MP biomedicals and FITC-conjugated goat anti-human IgG from medical and biological laboratories Co., LTD. The images were obtained by fluorescence microscopes (Olympus, Japan).

Data preprocessing
The images data were obtained as JPEG or Tiff file. We changed the resolution of the files from 2776 × 2074 pixel to 4 × 4, 16 × 16, 32 × 32, 64 × 64, 128 × 128, 256 × 256 and 512 × 512 pixel. After the conversion, we converted the JPEG file to PNG file to analyze.

Deep learning
The programming language (python), the environment (Visual studio code, Microsoft) and software, neural network console (Sony Inc.) were used. This neural network console application software contains two main features. One is easy editing by dragging and dropping. This could design a neural network with layers of each parameter. Another feature is that it has an automatic structure search function. This is the ability to automatically find higher performance and lighter neural network structures. Therefore, complicated functions and tuning work are conducted automatically with neural network console software. We described Neural Network Console previously (8). Input is IgA immunofluorescent images converted previously described. And the renal pathological image is classified into training images and validation images at the ratio of 8: 2. The validation images showed (Figurs S1 and S2). We performed supervised training for deep learning with convolutional neural network (CNN).

Ethical issues
The research followed the tenets of the Declaration of Helsinki. This study was approved by the committee of Okayama university (approved#ken1908-008). Accordingly, written informed consent was taken from all participants before any intervention. This study is a retrospective observational study, not clinical trial. As for informed consent, the contents of the research are posted on our department homepage and in the hospital. Public informed consent is given. As this retrospective observation study, the committee approved public infromed consent.

Statistical analysis
Statistical analysis was performed by JMP (SAS Institute Inc. version 11.0.0 for Windows software). Statistical significance was defined by one-way analysis of variance (ANOVA) with student t test. Data are shown as the mean ± SE. Significance was defined as P < 0.05. In addition, to determine the cut-off value of the diagnosis ratio, a receiver operating characteristic (ROC) curve was constructed using statistical analysis software JMP.

Overview of schema of deep learning
We collected IgA nephropathy images from 162 patients and non-IgA nephropathy images from 260 non-IgA nephropathy patients (Figure 1). The diagnosis of the patients showed in Table S1 (Supplementary file 1). An overview of the computational schema is showed ( Figure  2). We inputted IgA immunofluorescent images on seven kinds of resolution types, 4 × 4, 16 × 16, 32 × 32, 64 × 64, 128 × 128, 256 × 256 and 512 × 512 pixels. The construction of model is that middle layer is composed by three layers, the CNN layers is composed by convolutional, pooling and function layers. Each image analyzed, and connected to output. The renal pathological IgA images are classified into training images and validation images at the ratio of 8: 2. We used the software, neural network console provided from Sony Inc. This software is to automatically add or deletes some layers and adjust parameters to get optimum result. Using this software, we performed supervised training for deep learning. This software structured optimum model automatically.

AI could diagnose IgA nephropathy from IgA renal immunofluorescent pathological images
We created some models for each resolution images, from 4 × 4 pixels images to 512 × 512 pixels images. The representative images are shown (Figure 3 a-g). The ROC curve for each resolution image are shown ( Figure  3 h-n). In the regard of 4 × 4 pixels images (Figure 3a), although it is a fairly blurry images, artificial intelligence could diagnose that the AUC of created superior models   Figure S3. Next, in the regard of 16 × 16 pixels images (Figure 3a), the AUC (area under the curve) of created superior models was 0.93255 (Figure 3i; ROC curve). Representative model showed in Figure S4. Next, in the regard of 32 × 32 pixels images (Figure 3c), the AUC of created superior models was 0.93242 (Figure 3j; ROC curve). Representative model showed in Figure S5. Next, in the regard of 64X64 pixels images (Figure 3d), the AUC of created superior models was 0.94868 (Figure 3k; ROC curve). Representative model showed in Figure S6. Next, in the regard of 128 × 128 pixels images (Figure 3e), the AUC of created superior models was 0.94281 (Figure 3l; ROC curve). Representative model showed in Figure S7. Next, in the regard of 256 × 256 pixels images (Figure 3f ), the AUC of created superior models was 0.97581 ( Figure  3m; ROC curve). Representative model showed in Figure  S8. Next, in the regard of 512 × 512 pixels images ( Figure  3g), the AUC of created superior models was 0.98904 ( Figure 3n; ROC curve). Representative model showed in Figure S9. These data indicated that artificial intelligence is automatically extracted as features from limited image information, and the resolution of images affects the diagnosis. Deep learning could diagnose IgA nephropathy higher with high resolution images. However, there are so many times to analyze the images with high resolution images.

The diagnostic comparison between nephrologist and artificial intelligence
To compare the differences between human and artificial intelligence, we tested diagnosis by nephrologist using the same validation data set. With 32 pixels images, nephrologists could diagnose the IgA nephropathy, however, the diagnosis ratio is a little inferior compared to artificial intelligence (Figure 4a). On the other hand, with 256 pixels images, nephrologists could diagnose the IgA nephropathy, however the diagnosis ratio is a little inferior compared to artificial intelligence, too ( Figure  4b). The nephrologist usually comprehensively diagnose IgA nephropathy with the information of clinical course, laboratory data, pathological images and so on. As same as artificial intelligence, nephrologist diagnosed higher sinsitivity depending on resolution (Figure 4c). However, we could observe the tendency for specificity to increase as the resolution increased (Figure 4d). The differences in expert experience may lead to differences in specificity even if the low resolution images are hard for human eye to idintify specific staining site. These results suggested that high resolution images are easier to identify and distinguish for human in image diagnosis, especially affected the specificity for human.

Discussion
Artificial intelligence is reported to predict the occurrence of acute kidney injury (9), and assume the glomerular filtration rate from CT image (10). On the pathology, artificial intelligence could diagnose the metastases superior to human eye (11). As same as metastases pathology, diagnosis of IgA nephropathy is made by renal pathological, especially immunofluorescent images. This is because IgA nephropathy will be diagnosed when coarse granular deposits are found in the mesangial region in glomeruli by immunostaining of IgA and C3 from the sample of the renal biopsy. We have studied how artificial intelligence can intervene in such pathological diagnosis.
In this study, we examined the difference in diagnosis due to the difference in image resolution. Artificial intelligence can automatically extract features from images of any resolution that human is difficult to distinguish. On the image of CT and MRI, it is reported to be better to diagnose for AI with the high-resolution images (12,13). It is reported that the model trained with higher resolution showed higher accuracy (14). On the other hand, it is indicated that accuracy was not affected by the difference of resolution (15). Other report applied the resolution of 32 × 32 pixels and obtained an AUC of 0.86 on poly detection (16). However, the resolution of more than 224 × 224 pixels is believed to generally use because it is necessary for enough information. Our data suggested that high resolution images show high accuracy and sensitivity not only for human but also artificial intelligence.
In image diagnosis of medicine, it is easier for human eyes to make a diagnosis with a high-resolution image. In this study, the 4 × 4 pixels images could not lead to diagnose on human eyes. However, artificial intelligence could diagnose the features from the low-resolution images. Artificial intelligence converts each image into a digital signal, calculates them with a function, and creates a calculation formula that matches the correct answer. Therefore, the diagnosis in that process is also called a black box problem. Therefore, it was assumed that the difference of diagnosis depending on the resolution could be different from nephrologists, but the result in this study shows higher resolution of images leads to higher diagnostic accuracy. These results suggest that high-resolution images are also desirable for artificial intelligence because the high-resolution images provide more information. In this study, we input images of different resolutions to the model optimum for each resolution. The model optimum for high resolution could make a diagnosis well with low-resolution images, however the model optimum for low resolution could not with high-resolution images. It is similar to the fact that an adult could deal with a low degree of knowledge well, on the other hand, a child couldn't with a high degree of knowledge.
In addition, nephrologists' diagnosis observed large variations in specificity depending on resolution. For experienced nephrologists, even if the low resolution images were difficult to observe, they will expect to complement where difficult-to-observe part according to experience and lead to high diagnosis, but for inexperienced nephrologist, it was difficult to complement and advance diagnosis might be made only when the resolution is high. The difference is reflected in the specificity, further examination is needed to resolve the differences.

Conclusion
The high resolution of images provided more information to human and artificial intelligence, and affected the diagnosis. In this study, the high resolution of images affected for artificial intelligence to diagnosis, and affected for human large variations in specificity depending on resolution. The differences might be from the differences in the analysis and processing system between human and artificial intelligence. Taken together, the resolution of images might be important to diagnose for not only for human but also artificial intelligence.

Limitations of the study
In this study limitation, diagnosis of artificial intelligence was performed from only immunofluorescence images. In addition, the images were adjusted for easy observing. Moreover, human doctors diagnose comprehensive judgments with clinical course, clinical findings and clinical images, such as light microscope images and electron microscope images. To compare the differences between human and artificial intelligence, further examination is needed. and wrote the manuscript. KF, YS and KTu assisted to collect the images and clinical data. JW conceived the overall research scheme, coordinated with the study team. All authors participated in preparing the final draft of the manuscript and critically evaluated the intellectual contents. All authors have read and approved the content of the manuscript and confirmed the accuracy or integrity of any part of the work.

Conflicts of interest
The authors report no conflicts of interest in this work.

Ethical considerations
Ethical issues (including plagiarism, data fabrication, double publication) have been completely observed by the authors.