Researchers propose Air-LLM RF broadcast edge inference architecture

1. 02:16
   Hacker News AI

   Air-LLM: Broadcasting AI Weights for Edge LLM Inference via RF Computing

   The paper proposes AIR-LLM, an edge LLM inference architecture in which a central radio such as a 5G base station broadcasts LLM weights over the air and edge users perform the general matrix-vector multiplication of inference directly in the RF domain using RF mixers, avoiding weight storage and loading. It uses MIMO spatial multiplexing with an energy-efficient precoder-postcoder pair to shorten airtime, and because the central radio stays user-unaware, one broadcast can serve unlimited users in coverage. Evaluated on NVIDIA Sionna ray-traced channels of two urban scenes with a real RF mixer profile, AIR-LLM shows 4.0% WikiText-2 perplexity degradation on LLaMA-3.1-8B while saving 157.7x/40.4x energy versus FP16 and weight-only quantization baselines, and with 20 users its airtime is 104.1x/26.0x shorter.

Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.