Paper accepted @ QuASoQ 2018!

Konrad Fögen and Horst Lichter: A Case Study on Robustness Fault Characteristics for Combinatorial Testing – Results and Challenges

Abstract

Combinatorial testing is a well-known black box testing approach. Empirical studies suggest the effectiveness of combinatorial coverage criteria. So far, the research focuses on positive test scenarios. But, robustness is an important characteristic of software systems and testing negative scenarios is crucial. Therefore, combinatorial strategies are extended to generate invalid test inputs. Since effectiveness of negative test scenarios is yet unclear, we conducted a case study and analyzed 434 failures reported as bugs of an financial enterprise application.