Quantitative Cross-Layer Evaluation of Transient-Fault Injection Techniques for Algorithm Comparison

In the wake of the soft-error problem, fault injection (FI) is a standard methodology to measure fault resilience of programs and to compare algorithm variants. As detailed, e.g. gate-level machine models are often unavailable or too slow to simulate, FI is usually carried out in fast simulators based on abstracted system models, using e.g. ISA-level register injection. However, the literature deems such injection techniques too inaccurate and yielding wrong conclusions about analyzed programs. In this paper, we empirically challenge this assumption by applying gate-, flip-flop-and ISA-level FI techniques on an Arm® Cortex®-M0 processor. Analyzing FI results from 18 benchmark programs, we initially confirm related work by reporting SDC-rate discrepancies of up to an order of magnitude between a gate-level baseline and injection techniques on higher machine-model levels, suggesting gate-level injection should be used e.g. to select a specific sorting algorithm. We discuss why these discrepancies are, however, to be expected, and show that the extrapolated absolute failure-count metric combined with relative inter-benchmark measurements yield a significantly better cross-layer alignment of algorithm-resilience rankings. Our results indicate that ISA-level injection techniques suffice for evaluating and selecting program and algorithm variants on low-end processors.