DermAI: An Innovative AI-Driven Chatbot for Enhanced Dermatological Diagnosis and Patient Interaction

Pradeep Rajeshkumar, Shubhangi Kharche, Prithvi Poojari, Sachet Utekar, Sahil Saini, Samriddhi Bidwai

Abstract


Skin disorders constitute a noteworthy public health concern globally, with earnest impacts on both physical and mental well-being. However, effective dermatological care faces challenges in resource-limited regions due to poor infrastructure, limited access to medical facilities & expertise, and inadequate advanced diagnostic tools. The existing research work majorly focuses on cancer and uncommon skin diseases with models trying to achieve a higher training accuracy with no regards to misclassification rate. The products currently available in the market provide a limited initial diagnosis and suggest consulting a doctor to get an accurate diagnosis or offer a list of other possible skin disorders. To address these challenges, we propose DermAI, an innovative AI-based Chatbot made entirely of open-source technologies, which integrates the ResNet50 model and LLM via Chainlit, with Retrieval Augmented Generation(RAG), utilising AstraDB vector database and OpenAI embedding model for personalised responses. enabling accurate classification of common skin diseases. The proposed DermAI ensures minimal misclassification and comprehensive coverage of diseases, leveraging Retrieval-Augmented Generation and comparative model analysis. The metrics indicate that the model has a high true positive rate, with a misclassification rate of 2.17%, mean sensitivity, specificity & AUC of 92.6%, 99.8% & 99.9% respectively. This is demonstrated in the situations of melanoma, chickenpox, shingles, impetigo, and nail fungus, where it obtained 100% validation accuracy, a feat not attained by previous studies. Additionally, the model is highly capable of correctly identifying negative cases. The hallucination metric suggests the model may have a minimal tendency to hallucinate as the average hallucination score of 7% which falls far within the manually set threshold value of 50%. By setting the threshold value to 50%, the model generates grounded answers that are pertaining to the knowledge base and also allows it to be flexible with its responses. Overall, DermAI outperforms all solutions proposed in research literature.

Keywords


Deep Learning (DL), Transfer Learning (TL), Natural Language Processing (NLP), Chatbot, Dermatology, Retrieval Augmented Generation (RAG), Large Language Model (LLM), Telemedicine platform

References


A. Dregan, J. Charlton, P. Chowienczyk, and M. C. Gulliford, “Chronic inflammatory disorders and risk of type 2 diabetes mellitus, coronary heart disease, and stroke: a population-based cohort study,” Circulation, vol. 130, no. 10, pp. 837–844, 2014.

H. Feng, J. Berk-Krauss, P. W. Feng, and J. A. Stein, “Comparison of dermatologist density between urban and rural counties in the united states,” JAMA Dermatology, vol. 154, no. 11, pp. 1265–1271, 2018.

“Resnet-50 guide,” https://datagen.tech/guides/computer-vision/resnet-50/, accessed: 2023-08-31.

“Chainlit: Get started overview,” https://docs.chainlit.io/get-started/overview, accessed: 2024-01-16.

“Astra db vector: Concepts and metrics,” https://docs.datastax.com/en/astra/astra-db-vector/get-started/concepts.html#metrics, accessed: 2023-10-26.

K. Vayadande, “Innovative approaches for skin disease identification in machine learning: A comprehensive study,” Oral Oncology Reports, p. 100365, 2024.

D. P. Panagoulias, E. Tsoureli-Nikita, M. Virvou, and G. A. Tsihrintzis, “Dermacen analytica: A novel methodology integrating multi-modal large language models with machine learning in tele-dermatology,” arXiv preprint arXiv:2403.14243, 2024.

N. H. Sany and P. C. Shill, “Image segmentation based approach for skin disease detection and classification using machine learning algorithms,” in 2024 International Conference on Integrated Circuits and Communication Systems (ICICACS), 2024, pp. 1–5.

A. Ansari, A. Singh, M. Singh, and V. Kukreja, “Enhancing skin disease classification: A hybrid cnn-svm model approach,” in 2024 International Conference on Automation and Computation (AUTOCOM), 2024, pp. 29–32.

K. Sudharson, K. S. Essakki, V. R. Thanujaa, and S. Varsha, “Dermatec: Transformative ai-driven platform for comprehensive skin disease monitoring and dermatologist recommendations,” in 2024 IEEE International Students’ Conference on Electrical, Electronics and Computer Science (SCEECS), 2024, pp. 1–5.

M. C. Chiu, Y. Wang, Y. J. Kuo, and P. Y. Chen, “Ddi-coco: A dataset for understanding the effect of color contrast in machine-assisted skin disease detection,” arXiv preprint arXiv:2401.13280, 2024.

D. Xia, “Skin disease diagnosis using deep neural network and large language model,” https://doi.org/10.32657/10356/172895, 2023.

Q. Hu, H. Xia, and T. Zhang, “Chatbot combined with deep convolutional neural network for skin cancer detection,” in Methods, vol. 2, 2023, p. 35.

S. Kohli, U. Verma, V. V. Kirpalani, and R. Srinath, “Dermatobot: An image processing enabled chatbot for diagnosis and tele-remedy of skin diseases,” in 2022 3rd International Conference for Emerging Technology (INCET), 2022, pp. 1–5.

J. Huang, J. Li, Z. Li, Z. Zhu, C. Shen, G. Qi, and G. Yu, “Detection of diseases using machine learning image recognition technology in artificial intelligence,” Computational Intelligence and Neuroscience, vol. 2022, p. 5658641, 2022, retraction in: Comput Intell Neurosci. 2023 Jul 19;2023:9845093.

P. N. Srinivasu, J. G. SivaSai, M. F. Ijaz, A. K. Bhoi, W. Kim, and J. J. Kang, “Classification of skin disease using deep learning neural networks with mobilenet v2 and lstm,” Sensors, vol. 21, no. 8, p. 2852, 2021.

K. Glock, C. Napier, T. Gary, V. Gupta, J. Gigante, W. Schaffner, and Q. Wang, “Measles rash identification using transfer learning and deep convolutional neural networks,” in 2021 IEEE International Conference on Big Data (Big Data), 2021, pp. 3905–3910.

A. A. Elngar, R. Kumar, A. Hayat, and P. Churi, “Intelligent system for skin disease prediction using machine learning,” in 3rd International Conference on Smart and Intelligent Learning for Information Optimization (CONSILIO 2021), vol. 1998, 2021, p. 012037.

T. Goswami, V. K. Dabhi, and H. B. Prajapati, “Skin disease classification from image - a survey,” in 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), 2020, pp. 599–605.

A. K. Verma, S. Pal, and S. Kumar, “Classification of skin disease using ensemble data mining techniques,” Asian Pacific Journal of Cancer Prevention, vol. 20, no. 6, pp. 1887–1894, 2019.

N. C. F. Codella et al., “Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic),” in 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), 2018, pp. 168–172.

A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau, and S. Thrun, “Dermatologist-level classification of skin cancer with deep neural networks,” Nature, vol. 542, no. 7639, pp. 115–118, 2017.

C. Barata, M. Ruela, M. Francisco, T. Mendonc¸a, and J. S. Marques, “Two systems for the detection of melanomas in dermoscopy images using texture and color features,” IEEE Systems Journal, vol. 8, no. 3, pp. 965–979, 2014.

R. Mittal, F. Jeribi, R. J. Martin, V. Malik, S. J. Menachery, and J. Singh, “Dermcdsm: Clinical decision support model for dermatosis using systematic approaches of machine learning and deep learning,” IEEE Access, vol. 12, pp. 47 319–47 337, 2024.

V. S. S. B. T. Sathvika, N. Anmisha, V. Thanmayi, M. Suchetha, E. D. Dhas, S. Sehastrajit, and S. N. Aakur, “Pipelined structure in the classification of skin lesions based on alexnet cnn and svm model with bi-sectional texture features,” IEEE Access, vol. 12, pp. 57 366–57 380, 2024.

A. Kumar, A. Vishwakarma, V. Bajaj, and S. Mishra, “Novel mixed domain hand-crafted features for skin disease recognition using multiheaded cnn,” IEEE Transactions on Instrumentation and Measurement, vol. 73, pp. 1–13, 2024.

C. Tsai, P. P. H. Huang, Z. C. Wu, and J. F. Wang, “Advanced pigmented facial skin analysis using conditional generative adversarial networks,” IEEE Access, vol. 12, pp. 46 646–46 656, 2024.

W. Dai, R. Liu, T. Wu, M. Wang, J. Yin, and J. Liu, “Deeply supervised skin lesions diagnosis with stage and branch attention,” IEEE Journal of Biomedical and Health Informatics, vol. 28, no. 2, pp. 719–729, 2024.

D. Raha, M. Gain, R. Debnath, A. Adhikary, Y. Qiao, M. M. Hassan, A. K. Bairagi, and S. M. S. Islam, “Attention to monkeypox: An interpretable monkeypox detection technique using attention mechanism,” IEEE Access, vol. 12, pp. 51 942–51 965, 2024.

D. Kundu, M. M. Rahman, A. Rahman, D. Das, U. R. Siddiqi, M. G. R. Alam, S. K. Dey, G. Muhammad, and Z. Ali, “Federated deep learning for monkeypox disease detection on gan-augmented dataset,” IEEE Access, vol. 12, pp. 32 819–32 829, 2024.

L. Riaz, H. M. Qadir, G. Ali, M. Ali, M. A. Raza, A. D. Jurcut, and J. Ali, “A comprehensive joint learning system to detect skin cancer,” IEEE Access, vol. 11, pp. 79 434–79 444, 2023.

H. Q. Yu and S. Reiff-Marganiec, “Targeted ensemble machine classification approach for supporting iot enabled skin disease detection,” IEEE Access, vol. 9, pp. 50 244–50 252, 2021.

S. Back, S. Lee, S. Shin, Y. Yu, T. Yuk, S. Jong, S. Ryu, and K. Lee, “Robust skin disease classification by distilling deep neural network ensemble for the mobile diagnosis of herpes zoster,” IEEE Access, vol. 9, pp. 20 156–20 169, 2021.

J. Yang, X. Wu, J. Liang, X. Sun, M. M. Cheng, P. L. Rosin, and L. Wang, “Self-paced balance learning for clinical skin disease recognition,” IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 8, pp. 2832–2846, 2020.

Z. Wu, S. Zhao, Y. Peng, X. He, X. Zhao, K. Huang, X. Wu, W. Fan, F. Li, M. Chen, J. Li, W. Huang, X. Chen, and Y. Li, “Studies on different cnn algorithms for face skin disease classification based on clinical images,” IEEE Access, vol. 7, pp. 66 505–66 511, 2019.

Y. Xia, L. Zhang, L. Meng, Y. Yan, L. Nie, and X. Li, “Exploring web images to enhance skin disease analysis under a computer vision framework,” IEEE Transactions on Cybernetics, vol. 48, no. 11, pp. 3080–3091, 2018.

“Google lens,” https://lens.google/, accessed: 2024-06-24.

“Niramai,” https://www.niramai.com/, accessed: 2024-06-24.

“Qure.ai,” https://www.qure.ai/, accessed: 2024-06-24.

“Sigtuple,” https://sigtuple.com/, accessed: 2024-06-24.

“Mfine,” https://www.mfine.co/, accessed: 2024-06-24.

“Skinvision,” https://www.skinvision.com/, accessed: 2024-06-24.

“Cureskin,” https://cureskin.com/, accessed: 2024-06-24.

“First derm,” https://www.firstderm.com/, accessed: 2024-06-25.

“Model dermatol,” https://modelderm.com/en.html, accessed: 2024-06-25.

“Mamaearth,” https://mamaearth.in/, accessed: 2024-06-25.

“Cipla,” https://www.cipla.com/, accessed: 2024-06-25.

“idoc24,” https://idoc24.com/, accessed: 2024-06-25.

“Himalaya wellness,” https://himalayawellness.in/, accessed: 2024-06-25.

“Aysa,” https://askaysa.com/, accessed: 2024-06-25.

“mcaffeine,” https://www.mcaffeine.com/, accessed: 2024-06-25.

“Dermnet image library,” https://dermnetnz.org/image-library, accessed: 2023-08-03.

“Mapmyindia api addendums,” https://github.com/MapmyIndia/mapmyindia-api-addendums/tree/main/mapmyindia-move-deeplinks/web#1-nearby-facilities, accessed: 2024-04-27.

“Sqlite3 — python 3.13.1 documentation,” https://docs.python.org/3/library/sqlite3.html, accessed: 2024-04-27.

R. Yacouby and D. Axman, “Probabilistic extension of precision, recall, and f1 score for more thorough evaluation of classification models,” in Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems, 2020, pp. 79–91.

R. Banjade, “Deepeval: An integrated framework for the evaluation of student responses in dialogue based intelligent tutoring systems,” https://core.ac.uk/works/127173147, 2014.


Full Text: PDF

Refbacks

  • There are currently no refbacks.


 

Indonesian Journal of Electrical Engineering and Informatics (IJEEI)
ISSN 2089-3272

Creative Commons Licence

This work is licensed under a Creative Commons Attribution 4.0 International License.

web analytics
View IJEEI Stats

Error. Page cannot be displayed. Please contact your service provider for more details. (8)