DARQ

Challenge

AI is rapidly moving onto embedded platforms – from medical devices to drones and beyond. With such widespread adoption on the horizon, especially with critical systems, it is important that deep neural network (DNN) models can run correctly and efficiently across vastly different hardware environments.

In short: the results of calculations performed on an algorithm engineer’s development hardware may not match those executed on the target hardware – and that mismatch creates real-world risk.

Algorithm engineers typically design and verify AI models on high-precision computers, assuming standard mathematical properties and numerical behaviors will hold. But current software and hardware approaches for deploying DNNs are designed for increasing throughput, not safety, correctness, nor fault tolerance. For neural networks, built on layers of many nonlinear functions, even tiny numerical differences can have outsized effects. Even a small deviation between the intended and actual results of a function can push an input "across the boundary” of a decision function, leading to a drastically incorrect model output. That incorrect model, in turn, can lead to different-than-intended system behavior. 

In other words, imprecise math deployed in an AI model in an embedded system can make that system malfunction. And in a medical device, a drone on a high-risk mission, or a nuclear control system, predictable performance isn’t a “nice-to-have” feature, it can be the difference between life and death.

Solution

Galois’s DARQ (Dynamic Adaptive Real-time Quantization) project introduces a new method for doing math on neural networks to resolve numerical implementation issues. “Lazy arithmetic” not only ensures correctness, it enables unprecedented extreme efficiency for AI models running on edge devices.

How DARQ Works

Where other correctness approaches rely on compute-heavy, aggressive quantization, DARQ is the first system to run a neural network lazily, answering the question “is a data point to the left or right of a decision boundary line?” using the least amount of bits possible. 

A DNN implementation in an embedded system involves many thousands of calculations, the vast majority of which can be definitively determined to be to the left or right of a decision point - the determination of which is relatively low effort, only requiring a few bits. But every once in a while there is a dicey calculation that requires more bits to achieve confidence. 

‍

Instead of calculating all of a numerical value upfront, DARQ computes only the initial bits required to begin processing a decision. Thus, if we have a data point of 1.376, we first compute 1 and see if we can definitively learn its position relative to the decision line. If the answer is a definitive “yes” or “no,” we move on without using any additional bits. If more precision is needed to cross a critical decision boundary, the system adapts – requesting the necessary additional bits on demand, testing 3, then 5, then 7, and so on until confidence is achieved. Crucially, this can happen at any layer of the network, and the required precision can be retrieved without recomputing all prior math.

The strength of this dynamic, adaptive approach is that it only uses the bits that are needed for each calculation. And because the vast majority of data points have clear positions that are easy to determine, few bits are needed overall. Thus, DARQ provides a dramatically more efficient path to guaranteed correctness while also serving as a reliable method for identifying pain points.

This method guarantees that the model is doing exactly what it was programmed to do by maintaining correctness even when aggressive quantization is used and provides a super-efficient process that can reduce computational burden by up to 99%. Best of all, DARQ’s core math building blocks are generalizable and are intended to ensure a wide variety of embedded systems are functional and correct.

Currently, DARQ is capable of providing offline system analysis, allowing users to evaluate how susceptible a model is to numerical issues and pinpoint problematic areas. While DARQ is currently a software prototype, in the project’s next stage, we aim to deploy DARQ to optimize real-world embedded systems for correctness, efficiency, and security. 

Value Add

• Guaranteed Correctness and Assurance: Ensures the algorithm functions precisely as intended by the developer, mitigating risks in critical applications where incorrect numerical output can have serious consequences.
• Dramatically Increased Efficiency: Reduces computational burden by up to 99% through adaptive, lazy computation.‍
• Enhanced Security: Addresses hardware-level vulnerabilities, such as undetectable bit flip attacks or malicious data hidden in the model.