Automated Unit Test Generation for Python—SSBSE 2020

Recording of our presentation at SSBSE 2020 (http://ssbse2020.di.uniba.it/) on “Automated Unit Test Generation for Python”.

Pre-print: https://arxiv.org/abs/2007.14049
Final publication: https://doi.org/10.1007/978-3-030-59762-7_2
Slides: https://www.lukasczyk.me/downloads/ssbse2020-slides.pdf

Abstract:
Automated unit test generation is an established research field, and mature test generation tools exist for statically typed programming languages such as Java. It is, however, substantially more difficult to automatically generate supportive tests for dynamically typed programming languages such as Python, due to the lack of type information and the dynamic nature of the language. In this paper, we describe a foray into the problem of unit test generation for dynamically typed languages. We introduce Pynguin, an automated unit test generation framework for Python. Using Pynguin, we aim to empirically shed light on two central questions: (1) Do well-established search-based test generation methods, previously evaluated only on statically typed languages, generalise to dynamically typed languages? (2) What is the influence of incomplete type information and dynamic typing on the problem of automated test generation? Our experiments confirm that evolutionary algorithms can outperform random test generation also in the context of Python, and can even alleviate the problem of absent type information to some degree. However, our results demonstrate that dynamic typing nevertheless poses a fundamental issue for test generation, suggesting future work on integrating type inference.