In the dynamic realm of technological evolution, Edge AI innovation emerges as a pivotal force driving transformative progress. At the intersection of artificial intelligence and edge computing, this cutting-edge discipline revolutionizes the way data is processed, analyzed, and acted upon in real time, directly at the source. By harnessing the power of distributed computing and intelligent algorithms, Edge AI empowers devices to make autonomous, context-aware decisions without constant reliance on centralized cloud servers.
STMicroelectronics’ Senior Manager for AI Competence Center and Smartphone Competence Center in the Asia Pacific Region, Matteo Maravita, unveiled groundbreaking insights into AI solutions during an engaging virtual media event held on Jan 17th 2024, shedding light on ST’s vision, developer challenges, and the company’s proactive strategies to support them. Below are excerpts
Can you elaborate on STMicroelectronics’ vision for AI and its role in shaping a connected world?
Matteo Maravita: Absolutely. At ST, we envision AI as a transformative technology essential for building a future connected world, where billions of devices are more secure, connected, and intelligent. We refer to this vision as the Cloud Connected Intelligent Edge. Our goal is to make devices more autonomous while enhancing connectivity to the cloud, thereby increasing data creation and local processing capabilities across various domains, from homes and factories to cities and mobility solutions.
Tell us about the STMicroelectronics’ journey in the field of AI and the milestones achieved over the past decade.
Matteo Maravita: Certainly. At ST, our journey in AI began approximately 10 years ago with research focused on machine learning techniques tailored for edge devices. In 2018, we released the STM32 Cube.AI library, the first AI library for our STM32 microcontrollers. Since then, we’ve collaborated with customers on various projects, introduced sensors with machine learning accelerators, and expanded AI support to automotive microcontrollers and Linux-based devices. In 2021, we launched the Nano Edge AI ecosystem, followed by advanced MEMS sensors with hardware accelerators in 2022. Last year, we continued enhancing our software ecosystem with tools like the Developer Cloud model zoom and updates for our Stellar Automotive MCUs.
Could you provide examples of how ST is implementing Edge AI in real-world applications?
Matteo Maravita: Let me share some examples. Firstly, in collaboration with HPE Group, we’re optimizing the operation and maintenance of electric vehicle motors using edge AI. We’ve developed a virtual sensor powered by AI algorithms running on our automotive microcontroller, Stellar, which accurately estimates the internal rotor temperature of the motor. This microcontroller not only handles AI tasks but also drives the motor and performs predictive maintenance functions.
Secondly, partnering with HP, we’re leveraging smart sensor technology to enhance power monitoring in laptops, preventing overheating and battery drain. Our MEMS sensors, combined with AI models, recognize user activities and adjust system behavior accordingly, ensuring optimal performance and energy efficiency.
Lastly, in the domain of household appliances, we’re enhancing washing machine efficiency with AI. By integrating AI algorithms into our STM32 microcontroller, we’re accurately estimating the weight of clothes and optimizing motor control accordingly. Additionally, AI algorithms detect potential drum collisions, improving overall robustness and safety.
What specific challenges do developers face in implementing Edge AI?
Matteo Maravita: Developers face several challenges when implementing edge AI. On the hardware side, they need to ensure the overall performance of the application while considering security features and keeping power consumption low. As a semiconductor supplier, STMicroelectronics addresses these hardware challenges by providing a wide range of hardware devices tailored to different application needs. On the software side, developers must create, train, deploy, and maintain machine learning models within the final products, presenting a significant challenge for those addressing edge AI. STMicroelectronics is actively involved in assisting developers with both hardware and software challenges, offering comprehensive solutions to support their edge AI endeavors.
Could you elaborate on the significance of STMicroelectronics’ STM32 microcontroller platform in the realm of Edge AI development?
Matteo Maravita: Our STM32 microcontroller platform has emerged as a leading solution for edge AI development, evident from its dominance in benchmarks like the ML Perf Tiny benchmark, where 73% of all submitted projects are based on STM32. This success can be attributed to three main factors: our leadership in general-purpose microcontrollers for industrial and consumer applications, our active contribution to edge AI benchmarks, and the accessibility of our online platform, the STM32 AI Developer Cloud, which facilitates easy testing of models on a wide range of STM32 boards.
How does STMicroelectronics address the challenges developers face in implementing machine learning models on hardware-constrained devices like the STM32 microcontroller?
Matteo Maravita: Our approach involves optimizing performance, security, and power efficiency through tools and libraries like the Nano Edge AI ecosystem. By leveraging these resources, we assist developers in efficiently implementing and porting machine learning models to hardware devices like the STM32, which may have constraints in computational power and memory. Additionally, we continuously innovate to introduce hardware accelerators, such as NPUs integrated into devices like the STM32N6 and STM32MP2, enabling enhanced AI capabilities while maintaining efficient resource utilisation.
How is edge AI contributing to enhancing safety and preventing accidents in the automotive industry?
Matteo Maravita: Edge AI is playing a pivotal role in improving safety within vehicles by monitoring driver behaviour, detecting potential hazards like fatigue or drowsiness, and issuing timely alerts to prevent accidents, thus ensuring the well-being of drivers and passengers alike. Additionally, edge AI enables advanced functionalities such as in-cabin monitoring and anomaly detection, further enhancing overall safety and driving experience.
Could you showcase a practical application of AI on STMicroelectronics’ hardware-accelerated devices?
Matteo Maravita: Through our collaboration with partners like Lacroix, we’ve developed impressive demonstrations of AI applications running on hardware-accelerated devices like the STM32N6. For instance, we’ve implemented multi-object tracking algorithms capable of detecting and tracking pedestrians, bikes, and cars in real-time video footage. These applications, such as traffic management systems, showcase the remarkable performance of our hardware-accelerated devices, achieving outstanding results with minimal load on the main core.
Could you elaborate on the semiconductor innovations that are crucial for moving AI data processing from the cloud to the edge? What specific technologies and hardware accelerators, such as Neural Processing Units (NPUs) and In-Memory Computing (IMC) solutions, play a key role in this transition?
Matteo Maravita: As you said, exactly NPU are the main innovation that is already present in the market. Already several semiconductor companies are releasing products with NPUs (neural processing unit) integrated in their devices. For ST, we’ll soon release accelerated products – new generations of STM32MP2 and STM32N6. So this is the first innovation that has already been released on the market.
As you mentioned, the next innovation that will arrive quite soon is In-Memory Computing (IMC). And we are already working on that. This technology, IMC, will arrive on the market in a few years as a product. And we believe that it will be a game changer in the market since it will enable the developers to run similar kinds of models, but with a possible target of x10 in terms of performance, and at the same time with a reduction of power consumption down to 10% of actual value, again a factor of 10.
The innovation is not only on the hardware device or platform itself but there are a lot of innovations in the software tools and compilers for NPU and In-Memory Computing. So there is a lot of work that is going on to make sure that the hardware accelerator is fully used and tuned for the specific model selected by the customer.
How is STMicroelectronics assisting customers in addressing software challenges associated with edge AI development?
Matteo Maravita: STMicroelectronics recognizes the diverse skill sets and requirements of engineers involved in edge AI projects. We understand that embedded software engineers focus on implementing edge AI within the overall system, while machine learning engineers concentrate on developing and optimizing machine learning models for specific hardware devices. Additionally, hardware engineers need tools to test AI algorithms across different platforms and find the best compromise in terms of performance, power consumption, size, and price. To address these varied needs, we’re introducing the ST Edge AI Suite, a comprehensive tool designed to cater to the requirements of embedded software engineers, machine learning engineers, and hardware engineers alike.
Can you elaborate on the functionalities of the ST Edge AI Suite and how it aims to streamline edge AI development?
Matteo Maravita: The ST Edge AI Suite is designed to provide a unified solution for engineers working on edge AI projects. It offers features tailored to the specific requirements of embedded software engineers, machine learning engineers, and hardware engineers. For embedded software engineers, the suite provides firmware project examples and customization capabilities to integrate edge AI into the overall system seamlessly. Machine learning engineers benefit from tools for developing and optimizing machine learning models for specific hardware devices, ensuring efficient utilization of resources. Hardware engineers have access to simple tools for testing AI algorithms across various platforms and selecting the optimal configuration based on performance, power consumption, size, and cost considerations. By consolidating these functionalities into a single suite, we aim to streamline the edge AI development process and empower engineers to bring their ideas to fruition more effectively.
How does the introduction of the ST Edge AI Suite align with STMicroelectronics’ commitment to supporting edge AI innovation?
Matteo Maravita: STMicroelectronics is dedicated to fostering edge AI innovation by providing comprehensive solutions that address the diverse needs of developers. The ST Edge AI Suite represents our commitment to empowering engineers with the tools and resources they need to overcome software challenges associated with edge AI development. By offering a unified suite that caters to the requirements of embedded software engineers, machine learning engineers, and hardware engineers, we aim to facilitate collaboration, streamline development processes, and accelerate the deployment of edge AI solutions across various industries.
