A soil moisture sensor deployed in a Tamil Nadu rice farm cannot send data to a cloud server and wait for irrigation recommendations — connectivity is intermittent and latency requirements are measured in seconds, not minutes. A safety camera on a construction site must detect hard-hat violations in real time without depending on network availability. A medical monitoring device in a rural health post must flag abnormal ECG patterns without requiring a constant internet connection.
Edge AI — the deployment of machine learning inference on resource-constrained hardware at or near the data source — addresses these scenarios. The challenge is that the neural networks that power AI applications are computationally expensive, designed for server-class hardware with gigabytes of memory and watts of power available. Edge devices have milliwatts of power and kilobytes of working memory.
Model compression is the core technical discipline of edge AI. Techniques including quantisation (reducing the numerical precision of model weights from 32-bit floats to 8-bit integers), pruning (removing network connections with low influence on outputs), and knowledge distillation (training a smaller "student" model to mimic the behaviour of a larger "teacher" model) reduce model size by ten to one hundred times with modest accuracy degradation.
Dedicated silicon for edge inference has proliferated. NVIDIA Jetson handles computer vision workloads in industrial applications. Google Coral and its Edge TPU runs TensorFlow Lite models at extraordinary efficiency. Arm Cortex-M microcontrollers with TensorFlow Lite for Microcontrollers run keyword spotting and simple classification in microcontroller environments. Qualcomm's mobile AI hardware powers on-device inference in smartphones.
The deployment and management of edge AI models at scale — updating hundreds or thousands of deployed devices reliably — is an operational challenge that requires an MLOps approach extended to the edge: over-the-air model updates, remote monitoring, and fleet management infrastructure.