Evolution-Aware Runtime Verification
The risk posed by software bugs has increased significantly as software is now essential to many areas of our daily lives. Runtime verification can help find bugs by monitoring program executions against formally specified properties. Over the last two decades, tremendous research progress has improved the performance of runtime verification. However, there has been very little focus on the benefits and challenges of using runtime verification during software testing. Yet, testing generates many executions on which properties can be monitored. In this talk, I will describe my work on studying and improving runtime verification during testing. My large-scale study was the first to show that runtime verification during testing is beneficial for finding many important bugs from tests that developers already have. However, my study also showed that runtime verification still incurs high overhead, both in machine time to monitor properties and in developer time to inspect violations of the properties. Moreover, all prior runtime verification techniques consider only one program version and would wastefully re-monitor unaffected properties and code as software evolves. To reduce the overhead across multiple program versions, I proposed the first evolution-aware runtime verification techniques. My techniques exploit the key insight that software evolves in small increments and reduce the accumulated runtime verification overhead by up to 10x, without missing new violations.