A new self-supervised AI framework called CM-MAE teaches a model to connect what a camera sees with what a wireless radio measures - without hand labels telling it what's correlated.
The framework pairs RGB video frames with 64-beam wireless power readings from the DeepSense 6G dataset, using no ray-traced paths, depth data, or beam-index labels during training. Its key mechanism, a "soft contrastive alignment loss," treats wireless readings with similar directional patterns as related instead of forcing every non-identical pair apart as a mismatch, while a masked joint decoder reconstructs hidden video patches and wireless clusters even when one input is missing. On a test that keeps training and testing scenes fully separate, that softer alignment lifted transfer accuracy from 24.88% to 29.49%, and fine-tuning a fusion layer pushed Top-1 accuracy to 77.38% on unseen scenarios, reaching 78.69% with added normalization adjustments.
That matters because vision-and-wireless systems - the kind that could steer a 5G or 6G antenna beam based on where a phone or car sits - tend to fall apart when camera angle, lighting, or foot traffic shifts from the environment they trained on. CM-MAE's numbers suggest a real, if modest, step toward models that generalize across those shifts instead of needing retraining for every new location.
Worth noting: the headline accuracy figures still require a live wireless reading at inference, not a camera feed alone, so this is a transfer-learning result, not evidence the system can predict beams from video alone.