Performant and Small: Can We Have Both? SLMs on Mobile Devices for Healthcare Chatbots

Large language models (LLMs) are increasingly being adopted across diverse healthcare scenarios. However, their deployment on mobile devices is hindered by significant resource demands and privacy concerns associated with cloud-based solutions. Ensuring reliable, private, and accessible healthcare support on mobile devices requires models that are both performant and lightweight. Therefore, small language models (SLMs) present a promising solution for enabling on-device healthcare support. This study explores the trade-offs between model size and performance necessary to effectively execute general medical question-answering tasks on mobile devices. To evaluate this, we present MedicoAI, a cross-platform application designed to support local SLMs inference across mobile, web, and desktop environments. We evaluated four state-of-the-art SLMs with model sizes under 1GB using two prompt templates (a standard baseline and one with medical safety constraints) and three word-limit configurations. Our findings highlight the viability of deploying SLMs for medical question-answering on mobile devices while maintaining user privacy and resource efficiency.