AI Inference Moves Closer to the Edge
Since the development of artificial neural networks over a decade ago, AI has steadily integrated into nearly every part of modern life and work – from computer vision and robotics to large language models (LLMs) that simplify various tasks. In AI, “inference” refers to the ability of a trained model to make logical decisions based on live data. Using a low-power computer with an integrated inference accelerator close to the data source significantly speeds up response times and can also offer greater autonomy, security and privacy.
A SMARC (Smart Mobility ARChitecture) module is a compact, low-power computer-on-module designed for embedded edge systems. Their small form factor, scalability, and support for advanced AI and virtualization technologies make SMARC modules ideal for industrial, medical, and IoT applications requiring energy efficiency and robust performance [1].
New breakthroughs are now possible for low-power computing in SMARC Computer-on-Modules with Intel’s vastly improved Core i processors. The new “Amston Lake” Atom x7000RE chips can deliver superior capabilities while staying within the same power constraints, offering performance gains of up to 49% for single-threaded and 61% in multi-threaded applications [2]. Even more impressive is the 9.83-fold improvement in image classification performance, making true AI inference capabilities possible on edge and embedded computers such as SMARC modules.
Inside the Amston Lake processors, Intel now offers standardized “Gracemont” microarchitecture for its Efficient-cores (E-cores), allowing the integration of sophisticated x86 ecosystem AI features like Vector Neural Network Instructions (VNNI) in low power systems up to 15 W, which were previously reserved for systems with much higher power budgets.
Smarter, AI-driven industrial systems
The expansion of AI inference logic into low-power systems allows industrial manufacturers to adopt smarter, more efficient systems. AI-powered applications, including object recognition and detection, are becoming essential in improving production processes, such as in pick-and-place operations or quality assurance tasks. In robotics, the ability to process visual data in real-time is essential for autonomous guided vehicles (AGVs), service robots, and collaborative robots (cobots). With the performance boost offered by the new Intel Core i processors in SMARC modules, these systems can now handle more advanced algorithms, leading to faster decision-making and safer operations.
In medical technology, advanced image segmentation and processing are driving progress in fields such as diagnostics and surgery. For instance, mobile ultrasound devices benefit from faster AI-driven processing, which can accelerate diagnosis and enhance patient outcomes. In a similar vein, AI-enhanced security systems, smart city surveillance, and even automated checkout terminals in retail environments could see significant improvements thanks to the enhanced image classification capabilities offered by these new processors.
Streamlined AI development with Intel’s ecosystem
One of the standout features of Intel’s new generation of processors is how easily developers can implement AI technologies across a broad range of applications. Developers already familiar with Intel’s AI inference logic in high-performance systems can now easily transfer that technology to compact, low-power designs with industrial temperature tolerance (-40°C to +85°C). This is because the CPUs, with up to 8 E-cores and the Intel Gen 12 UHD GPU with 32 execution units, are the same as those found in higher-end processors using the Gracemont microarchitecture, which also supports INT8 deep learning inference.
This streamlined development process is particularly valuable for applications where low power consumption and industrial temperature tolerance are critical factors. The standardization of AI-related features, such as VNNI and AVX2 instruction sets, across Intel’s product range means that developers can focus on the unique aspects of their applications, speeding up time to market while maintaining high levels of performance and security. A full suite of tools can also be leveraged, from kernels to toolkits like OpenVino, to quickly bring AI capabilities to their systems without having to invest heavily in custom software development.
Enhanced flexibility and security through virtualization
With Intel doubling the number of E-cores in the Amston Lake processors, this significantly expands the options for virtualization, a technology that offers both flexibility and security in industrial and embedded systems. By consolidating multiple systems onto a single piece of hardware, companies can reduce costs and increase efficiency while maintaining strict separation between different operational functions. This separation is crucial for enhancing cybersecurity, particularly in systems that must comply with regulations like the Cyber Resilience Act.
Virtualization also simplifies system management by allowing developers to create isolated virtual environments for tasks like maintenance or updates. This ensures that critical operations can continue running without interruption, improving both system availability and reliability. For industries relying on the Industrial Internet of Things (IIoT), the ability to run maintenance tasks on a virtual machine separate from the primary system can help reduce downtime and enhance overall system resilience.
Hypervisor-on-Module solution for SMARC modules
Taking virtualization one step further, congatec has introduced a unique feature with its Hypervisor-on-Module technology in its conga-SA8 SMARC module. By integrating a hypervisor directly into the SMARC module’s firmware, congatec simplifies the implementation of virtual machines, reducing the development burden and enabling more secure, cost-effective, and durable solutions. This feature also supports real-time capabilities, which are essential in industrial environments where precise timing and synchronization are required.
With this technology, congatec offers developers a ready-made solution for creating virtualized systems that meet the highest standards of cybersecurity and reliability. Whether the system demands real-time functionality or not, this approach provides a higher level of security compared to traditional containerization methods. Each virtual machine operates independently, creating isolated environments that prevent the spread of threats across the system.
congatec also offers an aReady.COM version of the conga-SA8, allowing customers to purchase pre-configured modules tailored to their needs. For instance, these modules can come with Bosch Rexroth’s ctrlX OS and virtual machines for tasks like real-time control, HMI, AI, IIoT data exchange, firewall, and maintenance or management functions – ready for immediate use. The new SMARC modules are also fully compatible with multiple versions of Microsoft Windows.
The conga-SA8 is among the first SMARC modules to support Wi-Fi 6E, delivering nearly triple the data rate of Wi-Fi 5 and more reliable connections in congested areas. It’s also ready for TSN over Wi-Fi, ensuring deterministic connections with guaranteed throughput, offering a cost-effective alternative to 5G or Ethernet. Designed for industrial use, the module features in-band ECC for enhanced data security and soldered DRAM for greater durability in extreme environments.
Paving the way for future AI innovations
The introduction of Intel Core i processors into SMARC modules marks a significant milestone for embedded and edge computing. Adopting these more powerful SMARC modules will enable new breakthroughs wherever robust, high-performance AI inference logic is needed in small, fanless systems for use in demanding industrial environments. From industrial automation and robotics to healthcare and retail, the potential applications of this technology are vast and growing.
As AI continues to reshape industries, the ability to deploy powerful inference logic in compact, energy-efficient systems will become increasingly important. Developers now have access to an ecosystem that supports rapid implementation, allowing them to focus on innovation and differentiation.
References:
1. SMARC 2.1 technology: https://www.congatec.com/en/technologies/smarc/
2. Compared to Intel Atom processors of the x6000RE product series. Source: https://www.intel.com/content/www/us/en/products/details/embedded-processors/atom/atomx7000re.html
