The quality of automated detection of sinusitis based on radiography results in practical healthcare (prospective diagnostic study)

Purpose. To study the diagnostic accuracy of software based on artificial intelligence technologies for the diagnosis of maxillary and frontal sinusitis (J01, J32) when used in practical healthcare.

Materials and methods. Prospective diagnostic study (10.2023–12.2024) using the STARD 2015 methodology. The Index Test is four anonymized AI services for analyzing the X-ray imaging of the paranasal sinuses, integrated into Unified Radiological Information Service (ERIS) as part of the Moscow Experiment. The Reference Standard is an original methodology of clinical monitoring with a double expert review, an assessment of accuracy according to the criteria of “interpretation” and “localization”, calculation of sensitivity, specificity and AUC. After testing in the ERIS circuit, the AI services worked with real data and underwent monthly clinical monitoring.

Results. From October 2023 to December 2024, AI services analyzed 126,547 radiographs of the paranasal sinuses, of which 53% were processed by AI service-4. The scope of the research varied due to the different duration of the services.

Regular clinical monitoring covered 2,320 radiographs. The correctness of pathology detection, the accuracy of its classification and localization were evaluated. The highest diagnostic accuracy was found in AI services-1 and 2 (AUC 0.98 and 0.95; integral score – 95.1% and 83.3%). They demonstrated high specificity and precise localization of pathologies.

AI services-3 and 4 demonstrated comparatively lower results (AUC 0.89; integral score – 76.6% and 79.4%), with lower specificity for AI service-3 (0.65).

Conclusions. As part of the Moscow Experiment, AI services successfully analyzed 126,547 radiographs of the paranasal sinuses (2023–2024), showing an average level of accuracy (AUC 0.93) and quality (83.6%). The results obtained confirm the applicability of AI in clinical practice.

Differences in accuracy between services require regular monitoring of safety and quality. The developed methodologies make it possible to effectively carry out such control. The achieved level of quality opens prospects for the transformation of the interaction of doctors of different specialties, which will be the topic of future research.

1. Aksenova A.V., Briko N.I., Kleymenov D.A. Chronic pharyngitis, nasopharyngitis, sinusitis, and rhinitis: epidemiological prevalence and incidence rates in Moscow and the Russian Federation from 1996 to 2009. Russian Otorhinolaryngology. 2012; 2 (57): 11–19. (In Russian)

2. Juraeva Sh.F., Rakhimov Z.K. Epidemiology of maxillary odontogenic sinusitis and principles of inpatient treatment. Integrative Dentistry and Maxillofacial Surgery. 2022; 1 (1): 50–54. (In Russian)

3. Kolosov V.P., Manakov L.G., Perelman Yu.M., Samsonov V.P. Chronic respiratory diseases: epidemiological monitoring and prevention. Bulletin of Physiology and Pathology of Respiration. 2020; 76: 8–18. https://doi.org/10.36604/1998-5029-2020-76-8-18 (In Russian)

4. Shkurakova E.A., Gavenene E.N. Study of sinusitis incidence in Novocherkassk. International Research Journal. 2024; 7 (145): 1–5. https://doi.org/10.60797/IRJ.2024.145.72 (In Russian)

5. Ali M.A., Taishan W.S., Almaymoni A.A. et al. Prevalence of Ear, Nose, and Throat Problems in Saudi Arabia: A Cross-Sectional Study. Cureus. 2025; 17 (1): e76927. https://doi.org/10.7759/cureus.76927

6. Tuladhar A.S., Bhattarai A., Bimali S. et al. Sinusitis among Patients Undergoing CT Scan of Paranasal Sinuses in a Tertiary Care Centre: A Descriptive Cross-sectional Study. J. Nepal Med. Assoc. 2022; 60 (254): 857–860. https://doi.org/10.31729/jnma.7595

7. Haider S.P., Sharaf K., Baumeister P., Reichel C.A. Künstliche Intelligenz in der Hals-Nasen-Ohren-Heilkunde [Artificial intelligence in otorhinolaryngology]. HNO. 2022; 70 (2): 87-93. https://doi.org/10.1007/s00106-021-01095-0 (In German)

8. Wu Q., Wang X., Liang G. et al. Advances in Image-Based Artificial Intelligence in Otorhinolaryngology-Head and Neck Surgery: A Systematic Review. Otolaryngol. Head Neck Surg. 2023; 169 (5): 1132–1142. https://doi.org/10.1002/ohn.391

9. Kalinin D.I., Karyakin A.A., Zakharchenko U.R. et al. Development of an intelligent image analysis system for differential diagnosis of ear diseases. Bulletin of the Northern State Medical University. 2023; 2 (50): 35–40. (In Russian)

10. Kaibkhanov U.K., Shirokova V.I. Application of neural networks for disease diagnostics in otorhinolaryngology. In: Mechnikov Readings-2024. Materials of the 97th All-Russian Scientific and Practical Conference of the Student Scientific Society with International Participation. Saint Petersburg, 2024: 651–653. (In Russian)

11. Vasilyev Yu.A., Vladzimirskiy A.V., Arzamasov K.M. et al. Computer Vision in Radiology: The First Stage of the Moscow Experiment: Monograph. Moscow: Publishing Solutions, 2022. 388 p. (In Russian) https://telemedai.ru/biblioteka-dokumentov/kompyuternoe-zrenie-v-luchevoj-diagnostike-pervyj-etap-moskovskogo-eksperimenta?ysclid=mjzoej1vqe83567861

12. Bobrovskaya T.M., Vasilyev Yu.A., Nikitin N.Yu., Arzamasov K.M. Approaches to dataset formation in radiology. Doctor and Information Technologies. 2023; 4: 14–23. https://doi.org/10.25881/18110193_2023_4_14 (In Russian)

13. Vasilyev Yu A., Vladzimirskiy A.V., Omelyanskaya O.V. et al. Methodology for testing and monitoring artificial intelligence-based software for medical diagnostics. Digital Diagnostics. 2023; 4 (3): 252–267. https://doi.org/10.17816/DD321971 (In Russian)

14. Clinical Trials of Artificial Intelligence Systems (Radiology). – Moscow: Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Moscow Health Care Department, 2023. 40 p. (In Russian)

15. Basic Guidelines for the Operation of Artificial Intelligence Services for Radiology: Methodological Recommendations / Comp. by S.P. Morozov, L.R. Abuladze, A.E. Andreichenko et al. Series “Best Practices in Radiology and Instrumental Diagnostics”. Issue 119. Moscow: Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Moscow Health Care Department, 2022. 68 p. (In Russian)

16. Darbari Kaul R., Sacks P.L., Thiel C. et al. Radiomics of the Paranasal Sinuses: A Systematic Review of Computer-Assisted Techniques to Assess Computed Tomography Radiological Data. Am. J. Rhinol. Allergy. 2024: 19458924241304082. https://doi.org/10.1177/19458924241304082

17. Wu Z., Yu X., Chen Y. et al. Deep learning in the diagnosis of maxillary sinus diseases: a systematic review. Dentomaxillofac Radiol. 2024; 53 (6): 354–362. https://doi.org/10.1093/dmfr/twae031

18. Turosz N., Chęcińska K., Chęciński M. et al. Applications of artificial intelligence in the analysis of dental panoramic radiographs: an overview of systematic reviews. Dentomaxillofac. Radiol. 2023; 52 (7): 20230284. https://doi.org/10.1259/dmfr.20230284

19. Webster S., Fraser J. Artificial intelligence and dental panoramic radiographs: where are we now? Evid. Based Dent. 2024; 25 (1): 43–44. https://doi.org/10.1038/s41432-024-00978-9

20. Popadyuk V.I., Balashova M.E., Khabadze Z.S. et al. Diagnostic algorithm for upper airway examination based on CBCT data. Folia Otorhinolaryngologiae et Pathologiae Respiratoriae. 2023; 29 (4): 4–11. (In Russian)

21. Kasatkin A.N., Fomina A.V. Analysis of the prevalence of ENT diseases and their medical and social significance. Modern Problems of Healthcare and Medical Statistics. 2024; 2: 338–351. https://doi.org/10.24412/2312-2935-2024-2-338-351 (In Russian)

22. He S., Chen W., Wang X. et al. Deep learning radiomics-based preoperative prediction of recurrence in chronic rhinosinusitis. iScience. 2023; 26 (4): 106527. https://doi.org/10.1016/j.isci.2023.106527

23. Wang Y., Han Q., Wen B. et al. Development and validation of a prediction model for malignant sinonasal tumors based on MR radiomics and machine learning. Eur. Radiol. 2025; 35: 2074–2083. https://doi.org/10.1007/s00330-024-11033-7

24. Zhu K.Z., He C., Li Z. et al. Development and multicenter validation of a novel radiomics-based model for identifying eosinophilic chronic rhinosinusitis with nasal polyps. Rhinology. 2023; 61 (2): 132–143. https://doi.org/10.4193/Rhin22.361

25. Bulfamante A.M., Ferella F., Miller A.M. et al. Artificial intelligence, machine learning, and deep learning in rhinology: a systematic review. Eur. Arch. Otorhinolaryngol. 2023; 280 (2): 529–542. https://doi.org/10.1007/s00405-022-07701-3

26. Borodulin B.B., Gogoberidze Y.T., Zhilinskaya K.V. et al. Experience of using artificial intelligence for automated analysis of digital radiographs in a city hospital setting. Digital Diagnostics. 2024; 5 (S1): 127–129. https://doi.org/10.17816/DD629896

27. Nikitin E.D. Modeling the individual annotation style of a radiologist to improve the accuracy of neural networks. Digital Diagnostics. 2023; 4 (S1): 99–101. https://doi.org/10.17816/DD430358

28.