Hailo’s Hailo-10 GenAI accelerator delivers 40 TOPS of performance. This chip provides at least 2X more performance at half the power of the Intel Core Ultra NPU. The Hailo-10 architecture allows for local generative AI applications without the need for cloud-based service registration. For example, the chip runs Llama2-7B at 10 tokens per second while maintaining a power envelope of less than five watts. The device also processes Stable Diffusion 2.1 images in under 5 seconds using that same 5-watt limit. These performance levels move generative AI tasks to personal computers and smart vehicles. The Hailo-10 reduces reliance on the processing power of cloud data centers.
| Feature | Hailo-10 Capability |
|---|---|
| Peak Performance | 40 TOPS |
| Llama2-7B | 10 tokens per second |
| Stable Diffusion 2.1 | < 5 seconds per image |
| Power Limit | < 5 watts |
| Intel Comparison | 2X performance at 0.5X power |
Expansion through $120 million in funding
Hailo extended its series C fundraising round with a $120 million investment, bringing total funding to over $340 million. The company serves more than 300 customers globally. The ASUS UGen300 USB AI accelerator, which is powered by the Hailo-10H, allows laptops and other edge computers to run secure, offline classic and generative AI workloads through a ubiquitous and highly reliable plug-in device. Fujitsu uses the Hailo-10H in its HP AI module to streamline shopping through computer-vision checkout systems. In the security sector, Truen and Vicon use Hailo-15 AI vision processors for smart camera functionality. The AI-ISP powered denoising in extreme low light, combined with dynamic privacy masking and state-of-the-art search and indexing capabilities, provides advanced camera functionality for users in the security and surveillance market. The eXpedite x-ray bag screening technology from Evolv, which operates in cooperation with partner Advantech, supports accelerated and autonomous threat detection to provide efficient and accurate scanning in crowded public spaces. Husqvarna uses AI vision to improve the navigation of its autonomous lawn mower. Akara uses Hailo-8 to improve operation room efficiency with its thermal sensing platform. You probably already realize that local processing improves privacy and eliminates network latency.
Software updates for developers
Hailo released HailoRT 4.23 for Raspberry Pi and Ubuntu. This update targets Debian Trixie and does not support the Bookworm operating system. Users with Raspberry Pi can use the unified hailo-all installer for a more stable installation flow. For x86 Ubuntu users, software packages are available through the Dev Zone. The Raspberry Pi AI HAT+ uses the Hailo-8 and Hailo-8L to support vision AI for developers. Running YOLOv8s on a Hailo-8 module can reach 490 FPS when the model is compiled into a single context. The Hailo-8 and Hailo-8L M.2 modules contain an INA device for power measurement via the HailoRT CLI. The Hailo Dataflow Compiler includes a performance mode where the compiler tries to find a solution that fits in a single context to achieve the highest performance, though this method takes significantly longer to complete. For models with multiple contexts, the batch-size parameter increases throughput by reducing context switching overhead. Will the transition of the hailo-rpi5-examples repository to community focus attract more contributors to the ecosystem?
