A research framework called STELLA can run language-model-based human activity recognition directly on a phone or edge device, no cloud required.
Most LLM approaches to recognizing human movement — walking, running, sitting — feed raw accelerometer and gyroscope data into a model as long text prompts, then lean on cloud infrastructure to handle the compute. STELLA flips the bottleneck. A lightweight hierarchical tokenizer compresses multi-channel sensor readings into a compact set of latent tokens, which get projected into the embedding space of a frozen, pretrained LLM. The model never gets fine-tuned; only the tokenizer adapts. Tested across seven public datasets and eight benchmark settings, it improved over prior methods by up to 11.83% F1. On-device personalization — adapting just the tokenizer on a small sample of user-labeled data — added up to another 21.91% F1 gain as more data accumulated.
The privacy angle here is real, not marketing. Keeping sensor data, the retrieval context, and the LLM all on-device removes the exposure window that comes with streaming biometric movement data to a server. For health monitoring or assistive technology, that matters more than benchmark numbers.
Activity recognition has been a graveyard for "we'll just use a bigger model" thinking — cloud-dependent LLM pipelines trade accuracy for a latency and privacy bill users never agreed to pay. STELLA's bet is that smarter tokenization is worth more than model scale, and the benchmarks at least support the opening argument.