DermAI: An Innovative AI-Driven Chatbot for Enhanced Dermatological Diagnosis and Patient Interaction

DOI: https://doi.org/10.52549/ijeei.v12i4.5806
Published: 18-12-2024 • Issue: Vol. 12 No. 4: December 2024 • Section: Regular paper

Abstract

Skin disorders constitute a noteworthy public health concern globally, with earnest impacts on both physical and mental well-being. However, effective dermatological care faces challenges in resource-limited regions due to poor infrastructure, limited access to medical facilities & expertise, and inadequate advanced diagnostic tools. The existing research work majorly focuses on cancer and uncommon skin diseases with models trying to achieve a higher training accuracy with no regards to misclassification rate. The products currently available in the market provide a limited initial diagnosis and suggest consulting a doctor to get an accurate diagnosis or offer a list of other possible skin disorders. To address these challenges, we propose DermAI, an innovative AI-based Chatbot made entirely of open-source technologies, which integrates the ResNet50 model and LLM via Chainlit, with Retrieval Augmented Generation(RAG), utilising AstraDB vector database and OpenAI embedding model for personalised responses. enabling accurate classification of common skin diseases. The proposed DermAI ensures minimal misclassification and comprehensive coverage of diseases, leveraging Retrieval-Augmented Generation and comparative model analysis. The metrics indicate that the model has a high true positive rate, with a misclassification rate of 2.17%, mean sensitivity, specificity & AUC of 92.6%, 99.8% & 99.9% respectively. This is demonstrated in the situations of melanoma, chickenpox, shingles, impetigo, and nail fungus, where it obtained 100% validation accuracy, a feat not attained by previous studies. Additionally, the model is highly capable of correctly identifying negative cases. The hallucination metric suggests the model may have a minimal tendency to hallucinate as the average hallucination score of 7% which falls far within the manually set threshold value of 50%. By setting the threshold value to 50%, the model generates grounded answers that are pertaining to the knowledge base and also allows it to be flexible with its responses. Overall, DermAI outperforms all solutions proposed in research literature.

References

A. Dregan, J. Charlton, P. Chowienczyk, and M. C. Gulliford, “Chronic inflammatory disorders and risk of type 2 diabetes mellitus, coronary heart disease, and stroke: a population-based cohort study,†Circulation, vol. 130, no. 10, pp. 837–844, 2014.

H. Feng, J. Berk-Krauss, P. W. Feng, and J. A. Stein, “Comparison of dermatologist density between urban and rural counties in the united states,†JAMA Dermatology, vol. 154, no. 11, pp. 1265–1271, 2018.

“Resnet-50 guide,†https://datagen.tech/guides/computer-vision/resnet-50/, accessed: 2023-08-31.

“Chainlit: Get started overview,†https://docs.chainlit.io/get-started/overview, accessed: 2024-01-16.

“Astra db vector: Concepts and metrics,†https://docs.datastax.com/en/astra/astra-db-vector/get-started/concepts.html#metrics, accessed: 2023-10-26.

K. Vayadande, “Innovative approaches for skin disease identification in machine learning: A comprehensive study,†Oral Oncology Reports, p. 100365, 2024.

D. P. Panagoulias, E. Tsoureli-Nikita, M. Virvou, and G. A. Tsihrintzis, “Dermacen analytica: A novel methodology integrating multi-modal large language models with machine learning in tele-dermatology,†arXiv preprint arXiv:2403.14243, 2024.

N. H. Sany and P. C. Shill, “Image segmentation based approach for skin disease detection and classification using machine learning algorithms,†in 2024 International Conference on Integrated Circuits and Communication Systems (ICICACS), 2024, pp. 1–5.

A. Ansari, A. Singh, M. Singh, and V. Kukreja, “Enhancing skin disease classification: A hybrid cnn-svm model approach,†in 2024 International Conference on Automation and Computation (AUTOCOM), 2024, pp. 29–32.

K. Sudharson, K. S. Essakki, V. R. Thanujaa, and S. Varsha, “Dermatec: Transformative ai-driven platform for comprehensive skin disease monitoring and dermatologist recommendations,†in 2024 IEEE International Students’ Conference on Electrical, Electronics and Computer Science (SCEECS), 2024, pp. 1–5.

M. C. Chiu, Y. Wang, Y. J. Kuo, and P. Y. Chen, “Ddi-coco: A dataset for understanding the effect of color contrast in machine-assisted skin disease detection,†arXiv preprint arXiv:2401.13280, 2024.

D. Xia, “Skin disease diagnosis using deep neural network and large language model,†https://doi.org/10.32657/10356/172895, 2023.

Q. Hu, H. Xia, and T. Zhang, “Chatbot combined with deep convolutional neural network for skin cancer detection,†in Methods, vol. 2, 2023, p. 35.

S. Kohli, U. Verma, V. V. Kirpalani, and R. Srinath, “Dermatobot: An image processing enabled chatbot for diagnosis and tele-remedy of skin diseases,†in 2022 3rd International Conference for Emerging Technology (INCET), 2022, pp. 1–5.

J. Huang, J. Li, Z. Li, Z. Zhu, C. Shen, G. Qi, and G. Yu, “Detection of diseases using machine learning image recognition technology in artificial intelligence,†Computational Intelligence and Neuroscience, vol. 2022, p. 5658641, 2022, retraction in: Comput Intell Neurosci. 2023 Jul 19;2023:9845093.

P. N. Srinivasu, J. G. SivaSai, M. F. Ijaz, A. K. Bhoi, W. Kim, and J. J. Kang, “Classification of skin disease using deep learning neural networks with mobilenet v2 and lstm,†Sensors, vol. 21, no. 8, p. 2852, 2021.

K. Glock, C. Napier, T. Gary, V. Gupta, J. Gigante, W. Schaffner, and Q. Wang, “Measles rash identification using transfer learning and deep convolutional neural networks,†in 2021 IEEE International Conference on Big Data (Big Data), 2021, pp. 3905–3910.

A. A. Elngar, R. Kumar, A. Hayat, and P. Churi, “Intelligent system for skin disease prediction using machine learning,†in 3rd International Conference on Smart and Intelligent Learning for Information Optimization (CONSILIO 2021), vol. 1998, 2021, p. 012037.

T. Goswami, V. K. Dabhi, and H. B. Prajapati, “Skin disease classification from image - a survey,†in 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), 2020, pp. 599–605.

A. K. Verma, S. Pal, and S. Kumar, “Classification of skin disease using ensemble data mining techniques,†Asian Pacific Journal of Cancer Prevention, vol. 20, no. 6, pp. 1887–1894, 2019.

N. C. F. Codella et al., “Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic),†in 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), 2018, pp. 168–172.

A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau, and S. Thrun, “Dermatologist-level classification of skin cancer with deep neural networks,†Nature, vol. 542, no. 7639, pp. 115–118, 2017.

C. Barata, M. Ruela, M. Francisco, T. Mendonc¸a, and J. S. Marques, “Two systems for the detection of melanomas in dermoscopy images using texture and color features,†IEEE Systems Journal, vol. 8, no. 3, pp. 965–979, 2014.

R. Mittal, F. Jeribi, R. J. Martin, V. Malik, S. J. Menachery, and J. Singh, “Dermcdsm: Clinical decision support model for dermatosis using systematic approaches of machine learning and deep learning,†IEEE Access, vol. 12, pp. 47 319–47 337, 2024.

V. S. S. B. T. Sathvika, N. Anmisha, V. Thanmayi, M. Suchetha, E. D. Dhas, S. Sehastrajit, and S. N. Aakur, “Pipelined structure in the classification of skin lesions based on alexnet cnn and svm model with bi-sectional texture features,†IEEE Access, vol. 12, pp. 57 366–57 380, 2024.

A. Kumar, A. Vishwakarma, V. Bajaj, and S. Mishra, “Novel mixed domain hand-crafted features for skin disease recognition using multiheaded cnn,†IEEE Transactions on Instrumentation and Measurement, vol. 73, pp. 1–13, 2024.

C. Tsai, P. P. H. Huang, Z. C. Wu, and J. F. Wang, “Advanced pigmented facial skin analysis using conditional generative adversarial networks,†IEEE Access, vol. 12, pp. 46 646–46 656, 2024.

W. Dai, R. Liu, T. Wu, M. Wang, J. Yin, and J. Liu, “Deeply supervised skin lesions diagnosis with stage and branch attention,†IEEE Journal of Biomedical and Health Informatics, vol. 28, no. 2, pp. 719–729, 2024.

D. Raha, M. Gain, R. Debnath, A. Adhikary, Y. Qiao, M. M. Hassan, A. K. Bairagi, and S. M. S. Islam, “Attention to monkeypox: An interpretable monkeypox detection technique using attention mechanism,†IEEE Access, vol. 12, pp. 51 942–51 965, 2024.

D. Kundu, M. M. Rahman, A. Rahman, D. Das, U. R. Siddiqi, M. G. R. Alam, S. K. Dey, G. Muhammad, and Z. Ali, “Federated deep learning for monkeypox disease detection on gan-augmented dataset,†IEEE Access, vol. 12, pp. 32 819–32 829, 2024.

L. Riaz, H. M. Qadir, G. Ali, M. Ali, M. A. Raza, A. D. Jurcut, and J. Ali, “A comprehensive joint learning system to detect skin cancer,†IEEE Access, vol. 11, pp. 79 434–79 444, 2023.

H. Q. Yu and S. Reiff-Marganiec, “Targeted ensemble machine classification approach for supporting iot enabled skin disease detection,†IEEE Access, vol. 9, pp. 50 244–50 252, 2021.

S. Back, S. Lee, S. Shin, Y. Yu, T. Yuk, S. Jong, S. Ryu, and K. Lee, “Robust skin disease classification by distilling deep neural network ensemble for the mobile diagnosis of herpes zoster,†IEEE Access, vol. 9, pp. 20 156–20 169, 2021.

J. Yang, X. Wu, J. Liang, X. Sun, M. M. Cheng, P. L. Rosin, and L. Wang, “Self-paced balance learning for clinical skin disease recognition,†IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 8, pp. 2832–2846, 2020.

Z. Wu, S. Zhao, Y. Peng, X. He, X. Zhao, K. Huang, X. Wu, W. Fan, F. Li, M. Chen, J. Li, W. Huang, X. Chen, and Y. Li, “Studies on different cnn algorithms for face skin disease classification based on clinical images,†IEEE Access, vol. 7, pp. 66 505–66 511, 2019.

Y. Xia, L. Zhang, L. Meng, Y. Yan, L. Nie, and X. Li, “Exploring web images to enhance skin disease analysis under a computer vision framework,†IEEE Transactions on Cybernetics, vol. 48, no. 11, pp. 3080–3091, 2018.

“Google lens,†https://lens.google/, accessed: 2024-06-24.

“Niramai,†https://www.niramai.com/, accessed: 2024-06-24.

“Qure.ai,†https://www.qure.ai/, accessed: 2024-06-24.

“Sigtuple,†https://sigtuple.com/, accessed: 2024-06-24.

“Mfine,†https://www.mfine.co/, accessed: 2024-06-24.

“Skinvision,†https://www.skinvision.com/, accessed: 2024-06-24.

“Cureskin,†https://cureskin.com/, accessed: 2024-06-24.

“First derm,†https://www.firstderm.com/, accessed: 2024-06-25.

“Model dermatol,†https://modelderm.com/en.html, accessed: 2024-06-25.

“Mamaearth,†https://mamaearth.in/, accessed: 2024-06-25.

“Cipla,†https://www.cipla.com/, accessed: 2024-06-25.

“idoc24,†https://idoc24.com/, accessed: 2024-06-25.

“Himalaya wellness,†https://himalayawellness.in/, accessed: 2024-06-25.

“Aysa,†https://askaysa.com/, accessed: 2024-06-25.

“mcaffeine,†https://www.mcaffeine.com/, accessed: 2024-06-25.

“Dermnet image library,†https://dermnetnz.org/image-library, accessed: 2023-08-03.

“Mapmyindia api addendums,†https://github.com/MapmyIndia/mapmyindia-api-addendums/tree/main/mapmyindia-move-deeplinks/web#1-nearby-facilities, accessed: 2024-04-27.

“Sqlite3 — python 3.13.1 documentation,†https://docs.python.org/3/library/sqlite3.html, accessed: 2024-04-27.

R. Yacouby and D. Axman, “Probabilistic extension of precision, recall, and f1 score for more thorough evaluation of classification models,†in Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems, 2020, pp. 79–91.

R. Banjade, “Deepeval: An integrated framework for the evaluation of student responses in dialogue based intelligent tutoring systems,†https://core.ac.uk/works/127173147, 2014.

Author Biographies

Pradeep Rajeshkumar

Pradeep Rajeshkumar has completed his bachelor’s degree in Electronics and Computer Science Engineering from SIES Graduate School of Technology, Navi Mumbai, affiliated with the University of Mumbai. With a passion for Data Science, Artificial Intelligence, and Deep Learning and a strong foundation in Python programming, he is poised to explore advanced topics such as Advanced Analytics, Machine Learning, and Natural Language Processing. His expertise is further validated through hands-on experience and certifications.

He can be contacted at email: pradeeprecs120@gst.sies.edu.in
Shubhangi Kharche

Shubhangi Kharche holds a PhD in Electronics and Communication and currently serves as an Associate Professor and Head of Electronics and Computer Science Department at SIES Graduate School of Technology, Nerul, Navi Mumbai. With expertise as an AWS certified Cloud Solutions Architect Associate and Cisco Certified Network Associate, she has adeptly trained numerous students and faculty in Linux, Computer Networking, and AWS Cloud skills. Accumulating over two decades of teaching, training, and research experience, Dr. Kharche stands as a beacon of knowledge in her field.

Her contributions to academia and research have been duly recognized, notably with the receipt of the Women Researcher Award at the International Scientist Awards on Engineering, Science, and Medicine in Coimbatore, India, in September 2021. Her research interests encompass a wide array of cutting-edge domains, including Computer Communication Networks, Mobile Cellular Networks, Wireless Sensor Networks, IPv6 Over Low Power Wireless Personal Area Networks, Cloud Computing, Artificial Intelligence, and Data Analytics. With a portfolio boasting more than 35 research papers published in prestigious international conferences and journals, she has demonstrated her prowess as a scholar and innovator. She received the best paper award at the IEEE ANTS conference held at IISc Bangalore in 2016. Furthermore, her dedication to innovation is evident in her intellectual property achievements, having filed one patent and registered several copyrights. Her commitment to professional development is further underscored by her memberships in esteemed organisations such as ISTE, IETE, and Women in 6G. Dr. Shubhangi Kharche's multifaceted expertise and unwavering dedication to advancement make her a true trailblazer in the realms of academia and technology. She can be contacted at email: shubhangik@sies.edu.in, shubahngi.kharche@gmail.com


Prithvi Poojari

Prithvi Poojari has completed his bachelor’s degree in Electronics and Computer Science Engineering from SIES Graduate School of Technology, Navi Mumbai, affiliated with the University of Mumbai. With a never-ending passion for technology and science and its ability to steer societal progress, Prithvi looks at the world with a learner's perspective, constantly seeking ways to expand his horizons. Driven by a desire to make a meaningful impact, Prithvi actively seeks out opportunities to contribute to projects aimed at advancing humanity. He can be contacted at email: prithvipecs120@gst.sies.edu.in


Sachet Utekar

Sachet Utekar has completed his bachelor’s degree in Electronics and Computer Science Engineering from SIES Graduate School of Technology, Navi Mumbai, affiliated with the University of Mumbai. He has a keen interest in Generative Artificial Intelligence (Gen AI), Machine Learning (ML), and Deep Learning (DL), coupled with proficient Python programming skills. He is eager to delve deeper into advanced topics within these fields, exploring cutting-edge concepts and applications. Sachet vigorously ventures out opportunities to engage in work that drive forward societal advancement and improvement.


He can be contacted at email: sachetuecs120@gst.sies.edu.in
Sahil Saini

Sahil Saini has completed his bachelor’s degree in Electronics and Computer Science Engineering from SIES Graduate School of Technology, Navi Mumbai, affiliated with the University of Mumbai. He excels in AI, and blockchain. His dedication to hands-on innovation has earned him recognition in prestigious competitions like TechXter 13.0 and the Smart India Hackathon. Sahil's diverse interests include leadership roles such as Technical Head for IEEE SIES GST and organising a blockchain conference. With his ambition and diverse skill set, Sahil is poised to make significant contributions in the engineering field and beyond.

He can be contacted at email: sahilsecs120@gst.sies.edu.in
Samriddhi Bidwai
a people-oriented Data Scientist, boasts extensive experience in data science and analytics, having worked with industry giants such as Unilever, Hindustan Coca-Cola Beverages, MondelÄ“z International, General Mills, Sutherland Global Services and Cartesian Consulting. She holds a Master’s degree in Statistics from the University of Mumbai and has earned certifications in Tableau and advanced deep learning. Her passion for deep diving into data, business analytics, statistical analysis, and problem-solving has led to array of notable projects, from optimizing salaries for Ice Hockey players to Electrical Equipment Pricing Optimization. With her expertise in the FMCG and CPG domains, Samrridhi has collaborated across sales, marketing, HR, finance, quality, manufacturing, and supply chain sectors. Samrridhi has received multiple recommendations highlighting her teamwork, technical acumen, and leadership qualities. Her work has earned her the Trade Secret Recognition for reducing 90\% of product trial time, the prestigious nomination for Olson Service Excellence Award from General Mills, and other accolades such as the Manager’s Award and Certificate of Valuable Contribution for her impactful analytics work. She can be contacted at email: samrridhibdwai@gmail.com

How to Cite

[1]
P. Rajeshkumar, S. Kharche, P. Poojari, S. Utekar, S. Saini, and S. Bidwai, “DermAI: An Innovative AI-Driven Chatbot for Enhanced Dermatological Diagnosis and Patient Interaction”, IJEEI, vol. 12, no. 4, pp. 1040–1062, Dec. 2024, doi: 10.52549/ijeei.v12i4.5806.