DermAI: An Innovative AI-Driven Chatbot for Enhanced Dermatological Diagnosis and Patient Interaction

Pradeep Rajeshkumar, Shubhangi Kharche, Prithvi Poojari, Sachet Utekar, Sahil Saini, Samriddhi Bidwai

Abstract


Skin disorders constitute a noteworthy public health concern globally, withearnest impacts on both physical and mental well-being. However, effectivedermatological care faces challenges in resource-limited regions due to poor in-frastructure, limited access to medical facilities & expertise, and inadequate ad-vanced diagnostic tools. The existing research work majorly focuses on cancerand uncommon skin diseases with models trying to achieve a higher training ac-curacy with no regards to misclassification rate. The products currently availablein the market provide a limited initial diagnosis and suggest consulting a doc-tor to get an accurate diagnosis or offer a list of other possible skin disorders.To address these challenges, we propose DermAI, an innovative AI-based Chat-bot made entirely of open-source technologies, which integrates the ResNet-50 model and LLM via Chainlit, with Retrieval Augmented Generation(RAG),utilising AstraDB vector database and OpenAI embedding model for person-alised responses. enabling accurate classification of common skin diseases. Theproposed DermAI ensures minimal misclassification and comprehensive cover-age of diseases, leveraging Retrieval-Augmented Generation and comparativemodel analysis. The metrics indicate that the model has a high true positive rate,with a misclassification rate of 2.17%, mean sensitivity, specificity & AUC of92.6%, 99.8% & 99.9% respectively. This is demonstrated in the situations ofmelanoma, chickenpox, shingles, impetigo, and nail fungus, where it obtained100% validation accuracy, a feat not attained by previous studies. Additionally,the model is highly capable of correctly identifying negative cases. The hallu-cination metric suggests the model may have a minimal tendency to hallucinateas the average hallucination score of 7% which falls far within the manually setthreshold value of 50%. By setting the threshold value to 50%, the model gener-ates grounded answers that are pertaining to the knowledge base and also allowsit to be flexible with its responses. Overall, DermAI outperforms all solutionsproposed in research literature.

Keywords


Deep Learning (DL), Transfer Learning (TL), Natural Language Processing (NLP), Chatbot, Dermatology, Retrieval Augmented Generation (RAG), Large Language Model (LLM), Telemedicine platform

References


A. Dregan, J. Charlton, P. Chowienczyk, and M. C. Gulliford, “Chronic inflammatory disorders and risk of type 2 diabetes mellitus,

coronary heart disease, and stroke: a population-based cohort study,” Circulation, vol. 130, no. 10, pp. 837–844, 2014.

H. Feng, J. Berk-Krauss, P. W. Feng, and J. A. Stein, “Comparison of dermatologist density between urban and rural counties in the

united states,” JAMA Dermatology, vol. 154, no. 11, pp. 1265–1271, 2018.

“Resnet-50 guide,” https://datagen.tech/guides/computer-vision/resnet-50/, accessed: 2023-08-31.

“Chainlit: Get started overview,” https://docs.chainlit.io/get-started/overview, accessed: 2024-01-16.

“Astra db vector: Concepts and metrics,” https://docs.datastax.com/en/astra/astra-db-vector/get-started/concepts.html#metrics, ac-

cessed: 2023-10-26.

K. Vayadande, “Innovative approaches for skin disease identification in machine learning: A comprehensive study,” Oral Oncology

Reports, p. 100365, 2024.

D. P. Panagoulias, E. Tsoureli-Nikita, M. Virvou, and G. A. Tsihrintzis, “Dermacen analytica: A novel methodology integrating

multi-modal large language models with machine learning in tele-dermatology,” arXiv preprint arXiv:2403.14243, 2024.

N. H. Sany and P. C. Shill, “Image segmentation based approach for skin disease detection and classification using machine learning

algorithms,” in 2024 International Conference on Integrated Circuits and Communication Systems (ICICACS), 2024, pp. 1–5.

A. Ansari, A. Singh, M. Singh, and V. Kukreja, “Enhancing skin disease classification: A hybrid cnn-svm model approach,” in 2024

International Conference on Automation and Computation (AUTOCOM), 2024, pp. 29–32.

K. Sudharson, K. S. Essakki, V. R. Thanujaa, and S. Varsha, “Dermatec: Transformative ai-driven platform for comprehensive skin

disease monitoring and dermatologist recommendations,” in 2024 IEEE International Students’ Conference on Electrical, Electronics

and Computer Science (SCEECS), 2024, pp. 1–5.

M. C. Chiu, Y. Wang, Y. J. Kuo, and P. Y. Chen, “Ddi-coco: A dataset for understanding the effect of color contrast in machine-

assisted skin disease detection,” arXiv preprint arXiv:2401.13280, 2024.

D. Xia, “Skin disease diagnosis using deep neural network and large language model,” https://doi.org/10.32657/10356/172895, 2023.

Q. Hu, H. Xia, and T. Zhang, “Chatbot combined with deep convolutional neural network for skin cancer detection,” in Methods,

vol. 2, 2023, p. 35.

S. Kohli, U. Verma, V. V. Kirpalani, and R. Srinath, “Dermatobot: An image processing enabled chatbot for diagnosis and tele-remedy

of skin diseases,” in 2022 3rd International Conference for Emerging Technology (INCET), 2022, pp. 1–5.

J. Huang, J. Li, Z. Li, Z. Zhu, C. Shen, G. Qi, and G. Yu, “Detection of diseases using machine learning image recognition technology

in artificial intelligence,” Computational Intelligence and Neuroscience, vol. 2022, p. 5658641, 2022, retraction in: Comput Intell

Neurosci. 2023 Jul 19;2023:9845093.

P. N. Srinivasu, J. G. SivaSai, M. F. Ijaz, A. K. Bhoi, W. Kim, and J. J. Kang, “Classification of skin disease using deep learning

neural networks with mobilenet v2 and lstm,” Sensors, vol. 21, no. 8, p. 2852, 2021.

K. Glock, C. Napier, T. Gary, V. Gupta, J. Gigante, W. Schaffner, and Q. Wang, “Measles rash identification using transfer learning

and deep convolutional neural networks,” in 2021 IEEE International Conference on Big Data (Big Data), 2021, pp. 3905–3910.

A. A. Elngar, R. Kumar, A. Hayat, and P. Churi, “Intelligent system for skin disease prediction using machine learning,” in 3rd

International Conference on Smart and Intelligent Learning for Information Optimization (CONSILIO 2021), vol. 1998, 2021, p.

T. Goswami, V. K. Dabhi, and H. B. Prajapati, “Skin disease classification from image - a survey,” in 2020 6th International Confer-

ence on Advanced Computing and Communication Systems (ICACCS), 2020, pp. 599–605.

A. K. Verma, S. Pal, and S. Kumar, “Classification of skin disease using ensemble data mining techniques,” Asian Pacific Journal of

Cancer Prevention, vol. 20, no. 6, pp. 1887–1894, 2019.

N. C. F. Codella et al., “Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on

biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic),” in 2018 IEEE 15th International Symposium

on Biomedical Imaging (ISBI 2018), 2018, pp. 168–172.

A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau, and S. Thrun, “Dermatologist-level classification of skin cancer

with deep neural networks,” Nature, vol. 542, no. 7639, pp. 115–118, 2017.

C. Barata, M. Ruela, M. Francisco, T. Mendonc¸a, and J. S. Marques, “Two systems for the detection of melanomas in dermoscopy

images using texture and color features,” IEEE Systems Journal, vol. 8, no. 3, pp. 965–979, 2014.

R. Mittal, F. Jeribi, R. J. Martin, V. Malik, S. J. Menachery, and J. Singh, “Dermcdsm: Clinical decision support model for dermatosis

using systematic approaches of machine learning and deep learning,” IEEE Access, vol. 12, pp. 47 319–47 337, 2024.

V. S. S. B. T. Sathvika, N. Anmisha, V. Thanmayi, M. Suchetha, E. D. Dhas, S. Sehastrajit, and S. N. Aakur, “Pipelined structure in

the classification of skin lesions based on alexnet cnn and svm model with bi-sectional texture features,” IEEE Access, vol. 12, pp.

366–57 380, 2024.

A. Kumar, A. Vishwakarma, V. Bajaj, and S. Mishra, “Novel mixed domain hand-crafted features for skin disease recognition using

multiheaded cnn,” IEEE Transactions on Instrumentation and Measurement, vol. 73, pp. 1–13, 2024.

C. Tsai, P. P. H. Huang, Z. C. Wu, and J. F. Wang, “Advanced pigmented facial skin analysis using conditional generative adversarial

networks,” IEEE Access, vol. 12, pp. 46 646–46 656, 2024.

W. Dai, R. Liu, T. Wu, M. Wang, J. Yin, and J. Liu, “Deeply supervised skin lesions diagnosis with stage and branch attention,” IEEE

Journal of Biomedical and Health Informatics, vol. 28, no. 2, pp. 719–729, 2024.

D. Raha, M. Gain, R. Debnath, A. Adhikary, Y. Qiao, M. M. Hassan, A. K. Bairagi, and S. M. S. Islam, “Attention to monkeypox:

An interpretable monkeypox detection technique using attention mechanism,” IEEE Access, vol. 12, pp. 51 942–51 965, 2024.

D. Kundu, M. M. Rahman, A. Rahman, D. Das, U. R. Siddiqi, M. G. R. Alam, S. K. Dey, G. Muhammad, and Z. Ali, “Federated

deep learning for monkeypox disease detection on gan-augmented dataset,” IEEE Access, vol. 12, pp. 32 819–32 829, 2024.

L. Riaz, H. M. Qadir, G. Ali, M. Ali, M. A. Raza, A. D. Jurcut, and J. Ali, “A comprehensive joint learning system to detect skin

cancer,” IEEE Access, vol. 11, pp. 79 434–79 444, 2023.

H. Q. Yu and S. Reiff-Marganiec, “Targeted ensemble machine classification approach for supporting iot enabled skin disease detec-

tion,” IEEE Access, vol. 9, pp. 50 244–50 252, 2021.

S. Back, S. Lee, S. Shin, Y. Yu, T. Yuk, S. Jong, S. Ryu, and K. Lee, “Robust skin disease classification by distilling deep neural

network ensemble for the mobile diagnosis of herpes zoster,” IEEE Access, vol. 9, pp. 20 156–20 169, 2021.

J. Yang, X. Wu, J. Liang, X. Sun, M. M. Cheng, P. L. Rosin, and L. Wang, “Self-paced balance learning for clinical skin disease

recognition,” IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 8, pp. 2832–2846, 2020.

Z. Wu, S. Zhao, Y. Peng, X. He, X. Zhao, K. Huang, X. Wu, W. Fan, F. Li, M. Chen, J. Li, W. Huang, X. Chen, and Y. Li, “Studies on

different cnn algorithms for face skin disease classification based on clinical images,” IEEE Access, vol. 7, pp. 66 505–66 511, 2019.

Y. Xia, L. Zhang, L. Meng, Y. Yan, L. Nie, and X. Li, “Exploring web images to enhance skin disease analysis under a computer

vision framework,” IEEE Transactions on Cybernetics, vol. 48, no. 11, pp. 3080–3091, 2018.

“Google lens,” https://lens.google/, accessed: 2024-06-24.

“Niramai,” https://www.niramai.com/, accessed: 2024-06-24.

“Qure.ai,” https://www.qure.ai/, accessed: 2024-06-24.

“Sigtuple,” https://sigtuple.com/, accessed: 2024-06-24.

“Mfine,” https://www.mfine.co/, accessed: 2024-06-24.

“Skinvision,” https://www.skinvision.com/, accessed: 2024-06-24.

“Cureskin,” https://cureskin.com/, accessed: 2024-06-24.

“First derm,” https://www.firstderm.com/, accessed: 2024-06-25.

“Model dermatol,” https://modelderm.com/en.html, accessed: 2024-06-25.

“Mamaearth,” https://mamaearth.in/, accessed: 2024-06-25.

“Cipla,” https://www.cipla.com/, accessed: 2024-06-25.

“idoc24,” https://idoc24.com/, accessed: 2024-06-25.

“Himalaya wellness,” https://himalayawellness.in/, accessed: 2024-06-25.

“Aysa,” https://askaysa.com/, accessed: 2024-06-25.

“mcaffeine,” https://www.mcaffeine.com/, accessed: 2024-06-25.

“Dermnet image library,” https://dermnetnz.org/image-library, accessed: 2023-08-03.

“Mapmyindia api addendums,” https://github.com/MapmyIndia/mapmyindia-api-addendums/tree/main/

mapmyindia-move-deeplinks/web#1-nearby-facilities, accessed: 2024-04-27.

“Sqlite3 — python 3.13.1 documentation,” https://docs.python.org/3/library/sqlite3.html, accessed: 2024-04-27.

R. Yacouby and D. Axman, “Probabilistic extension of precision, recall, and f1 score for more thorough evaluation of classification

models,” in Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems, 2020, pp. 79–91.

R. Banjade, “Deepeval: An integrated framework for the evaluation of student responses in dialogue based intelligent tutoring

systems,” https://core.ac.uk/works/127173147, 2014.



Refbacks

  • There are currently no refbacks.


 

Indonesian Journal of Electrical Engineering and Informatics (IJEEI)
ISSN 2089-3272

Creative Commons Licence

This work is licensed under a Creative Commons Attribution 4.0 International License.

web analytics
View IJEEI Stats

Error. Page cannot be displayed. Please contact your service provider for more details. (6)