Edge AI and Vision: Empowering automation with intelligence

Data privacy and automation in human-centric imaging

Abstract illustration representing edge AI and privacy

Automation is a key business objective in various application domains. Legacy computer programs can automate simple and repetitive tasks by detecting patterns and following rules. However, to deal with complex and dynamic situations, artificial intelligence (AI) is the go-to technology as it aims to mimic human cognitive abilities.

In today’s world, there are two main ways for robots and machines to achieve intelligence: either through cloud connectivity or by implementing AI on the edge. While cloud computing provides abundant and scalable resources, edge solutions with embedded AI offer other benefits, such as low latency, privacy, power efficiency, and offline functionality.

Simplified diagram explaining intelligence acquisition in robots: cloud connectivity vs. edge AI.

This diagram compares the two main ways for robots and machinery to gain intelligence. On the left, cloud connectivity provides access to vast computing power and storage but it requires a stable internet connection and may raise security issues. On the right, an edge device enables faster, private, and energy-efficient processing, as well as offline operation.

The role of Edge AI in human-centric imaging

Non-intrusive vision system for car driver monitoring (DMS)

Image courtesy of thyssenkrupp Presta AG - An automotive DMS co-developed by thyssenkrupp Presta AG and CSEM. It includes multiple cameras and a set of custom-made NIR illuminators, tested in a simulator environment to extract, eye gaze, point of regard, distraction, and drowsiness levels.

AI and imaging for human-machine interfaces, in domains like aviation and automotive, require intelligent systems that are designed with safety, ergonomics, and user-centricity in mind. These systems aim to facilitate natural and effective interactions between humans and machines, especially in high-risk environments, where situational awareness, error prevention, and human lives are at stake.

For instance, autonomous driving in aviation, involves not only exterior sensing and decision-making systems, but also in-cabin monitoring systems that provide a comprehensive understanding of the cabin environment and the crew’s state. A specific example, of such a system is DMS - Driver Monitoring Systems, which continuously monitors the head posture, eye gaze, focus/distraction levels of the driver and provides appropriate alerts.

In such application domains, embedded AI processing is crucial for two primary reasons:

Privacy: sending in-cabin images and/or audio data to the cloud significantly impairs crew’s privacy and exposes the system to external data attacks.
Responsiveness: to avoid network latency and bottlenecks, data-driven processing should be carried out in real time.

In addition, offline operation is a crucial aspect of many time-critical IoT solutions. In many areas with poor network connectivity, including some suburban regions of modern cities, DMS with edge AI capabilities provide reliable and uninterrupted operation.

CSEM’s approach to overcome Edge AI deployment challenges

Most of today’s successful machine learning models are large and need a lot of computing power and resources to run. Building and deploying optimized models for resource-constrained edge devices necessitates a specific set of knowledge and expertise.

Interplay between software and hardware in Edge AI systems

Addressing the complex interplay between software and hardware involves a hardware-aware neural network approach, where hardware-specific factors impact the neural network architectural design process from the beginning. Major key metrics to consider include the number of floating-point operations per second (FLOPs), the number of parameters in the network, quantization levels for inference, memory bandwidth bottleneck of the network, processor utilization efficiency and energy consumption.

Proposed hardware-aware neural network design stages: architecture search, hardware selection, pruning, and quantization.

CSEM’s proposed set of building blocks for a hardware-aware neural network design pipeline. The pipeline involves neural architecture search, hardware design and selection, as well as the processes of model pruning and quantization. The goal of this pipeline is to optimize AI performance.

CSEM’s approach to overcome Edge AI deployment challenges

Our approach involves designing and deploying a pipeline managed concurrently, comprising of three building blocks:

Neural Design and Architecture Search: This step finds the most efficient and effective neural network topologies and configurations. The challenge is reducing the size of the network and its FLOPs rating without compromising its accuracy.
Hardware Design and/or Selection: This involves choosing or designing the appropriate hardware that can support the selected neural network architecture. A known bottleneck, specifically for convolutional neural networks, is data transfer between the memory and the processing unit, which can be addressed with processor-in-memory designs.
Model Reduction through Pruning and Quantization: This step aims to trim redundant or non-essential parts of neural networks to enhance computational efficiency at the expense of slightly reduced accuracy and precision.

Scatter plot analysis: operations vs. model accuracy in neural networks.

This plot illustrates the correlation between accuracy and the number of operations required to run a network. Our approach ensures the solutions we deliver provide close-to-optimal performance while keeping the computational burden limited.

Specialized in Edge AI, our strength lies in designing and training efficient neural networks for computer vision applications. To strike the right balance between accuracy, speed and latency, we employ state-of-the-art techniques in all the three areas mentioned above, resulting in lean yet effective systems. This approach enables efficient deployment of machine learning models on resource-constrained devices.

The Imperative for Edge AI Integration

The resource-intensive nature of today’s deep learning models makes the adoption of edge AI solutions imperative for many industrial applications. Edge AI solutions play a pivotal role in preserving data anonymity and facilitating highly responsive intelligent systems. Our expertise ensures seamless operation of AI at the edge, delivering optimal performance while safeguarding data privacy and enabling real-time decision-making.

Ready to transform your tech with embedded AI and data privacy?

Explore beyond cloud-based models: Discover embedded AI solutions for IoT devices. Curious to learn more? Contact us today to unlock the full potential of AI in your industry!

To probe further

Explore our compiled selection of articles published in renowned journals and conference proceedings for in-depth insights into Edge AI and Vision advancements:

S. Narduzzi, E. Türetken, J. -P. Thiran and L. A. Dunbar, "Adaptation of MobileNetV2 for Face Detection on Ultra-Low Power Platform," 2022 9th Swiss Conference on Data Science (SDS), Lucerne, Switzerland, 2022, pp. 1-6, doi:10.1109/SDS54800.2022.00008.
P. Pad, S. Narduzzi, C. Kündig, E. Türetken, S. A. Bigdeli and L. A. Dunbar, "Efficient Neural Vision Systems Based on Convolutional Image Acquisition," 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 12282-12291, doi: 10.1109/CVPR42600.2020.01230.
Y. Sepehri, P. Pad, C. Kündig, P. Frossard and L. A. Dunbar, "Privacy-Preserving Image Acquisition for Neural Vision Systems," in IEEE Transactions on Multimedia, vol. 25, pp. 6232-6244, 2023, doi: 10.1109/TMM.2022.3207018.
P. Jokic, E. Azarkhish, R. Cattenoz, E. Türetken, L. Benini and S. Emery, "A Sub-mW Dual-Engine ML Inference System-on-Chip for Complete End-to-End Face-Analysis at the Edge," 2021 Symposium on VLSI Circuits, Kyoto, Japan, 2021, pp. 1-2, doi: 10.23919/VLSICircuits52068.2021.9492401.
E. Türetkin, S. Saeedi, S. Bigdeli, P. Stadelmann, N. Cantale, L. Lutnyk, M. Raubal, and and L. A. Dunbar, "Real-time eye gaze tracking for human machine interaction in the cockpit," Proc. SPIE 12019, AI and Optical Data Sciences III, 1201904 (2 March 2022), doi:10.1117/12.2607434.