Robust Conformal Prediction for STL Runtime Verification under Distribution Shift
2024 ACM/IEEE 15th International Conference on Cyber-Physical Systems (ICCPS)(2023)
Abstract
Cyber-physical systems (CPS) designed in simulators behave differently in the
real-world. Once they are deployed in the real-world, we would hence like to
predict system failures during runtime. We propose robust predictive runtime
verification (RPRV) algorithms under signal temporal logic (STL) tasks for
general stochastic CPS. The RPRV problem faces several challenges: (1) there
may not be sufficient data of the behavior of the deployed CPS, (2) predictive
models are based on a distribution over system trajectories encountered during
the design phase, i.e., there may be a distribution shift during deployment. To
address these challenges, we assume to know an upper bound on the statistical
distance (in terms of an f-divergence) between the distributions at deployment
and design time, and we utilize techniques based on robust conformal
prediction. Motivated by our results in [1], we construct an accurate and an
interpretable RPRV algorithm. We use a trajectory prediction model to estimate
the system behavior at runtime and robust conformal prediction to obtain
probabilistic guarantees by accounting for distribution shifts. We precisely
quantify the relationship between calibration data, desired confidence, and
permissible distribution shift. To the best of our knowledge, these are the
first statistically valid algorithms under distribution shift in this setting.
We empirically validate our algorithms on a Franka manipulator within the
NVIDIA Isaac sim environment.
MoreTranslated text
Key words
Predictive runtime verification,stochastic system verification,signal temporal logic,conformal prediction
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined