Scenario-Based Testing of Cyber-Physical Systems

Description

In recent years, the development of automated and autonomously operating cyber-physical systems such as cars or drones is on the rise. Testing the safe behavior of these complex systems in all relevant situations is a challenge that is not easily solved.
Due to the high amount of needed test samples, the verification and validation of cyber-physical systems only with real-world testing is infeasible. To complement these efforts, we propose the use of scenario-based testing. In this method, the cyber-physical system is tested in different scenario types, which describe situations in which we aim to test the system's behavior. For cars, examples of these scenario types are "Lane Change" or "Cut-In". Each of these scenario types includes parameters to describe its various aspects, e.g., starting position of the system under test. When generating test cases for these scenario types, concrete values are assigned to all parameters of the considered scenario type.
 

Since it is not feasible to test the cyber-physical system in all possible scenario instances of one scenario type, we need to select "good" test cases that challenge the system under test. A test case is a "good" test case if it can reveal potential faults in the system. When generating these "good" test cases, different optimization techniques such as search-based techniques can be used.

Publications

  • Pretschner, Alexander; Hauer, Florian; Schmidt, Tabea: Tests für automatisierte und autonome Fahrsysteme. Informatik Spektrum, 2021 more… BibTeX Full text ( DOI )
  • Florian Hauer, Bernd Holzmüller: A sound approach to scenario-based testing as the basis for safety argumentations. Automotive Testing Technology Magazine Special Issue, 2020 more… BibTeX
  • Calo, Alessandro; Arcaini, Paolo; Ali, Shaukat; Hauer, Florian; Ishikawa, Fuyuki: Generating Avoidable Collision Scenarios for Testing Autonomous Driving Systems. 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST), IEEE, 2020 more… BibTeX Full text ( DOI )
  • Calò, Alessandro; Arcaini, Paolo; Ali, Shaukat; Hauer, Florian; Ishikawa, Fuyuki: Simultaneously searching and solving multiple avoidable collisions for testing autonomous driving systems. Proceedings of the 2020 Genetic and Evolutionary Computation Conference, ACM, 2020 more… BibTeX Full text ( DOI )
  • Florian Hauer, Ilias Gerostathopoulos, Tabea Schmidt, Alexander Pretschner: Clustering Traffic Scenarios Using Mental Models as Little as Possible. 2020 IEEE Intelligent Vehicles Symposium (IV), IEEE, 2020 more… BibTeX Full text ( DOI ) Full text (mediaTUM)
  • Florian Hauer, Alexander Pretschner, Bernd Holzmüller: Re-Using Concrete Test Scenarios Generally Is a Bad Idea. 2020 IEEE Intelligent Vehicles Symposium (IV), IEEE, 2020 more… BibTeX Full text ( DOI ) Full text (mediaTUM)
  • Florian Hauer, Bernd Holzmüller: How Many Test Scenarios Do We Need for Testing Automated and Autonomous Driving Systems? TÜV / TUM Fachtagung "Automatisiertes Fahren" 2019 more… BibTeX Full text (mediaTUM)
  • Florian Hauer, Tabea Schmidt, Bernd Holzmuller, Alexander Pretschner: Did We Test All Scenarios for Automated and Autonomous Driving Systems? 2019 IEEE Intelligent Transportation Systems Conference (ITSC) , IEEE , 2019 more… BibTeX Full text ( DOI ) Full text (mediaTUM)
  • Florian Hauer, Alexander Pretschner, Bernd Holzmüller: Fitness Functions for Testing Automated and Autonomous Driving Systems. 38th International Conference on Computer Safety, Reliability and Security, Springer International Publishing , 2019 more… BibTeX Full text ( DOI ) Full text (mediaTUM)
  • Florian Hauer, Bernd Holzmüller: Szenario-Optimierung für die Absicherung von automatisierten und autonomen Fahrsystemen – PrePrint für den Tagungsband der FKFS AutoTest Fachtagung 2018. Chair of Software and Systems Engineering, 2018, more… BibTeX Full text (mediaTUM)