Specification testing of agent-based simulation using property-based testing

Thaler, Jonathan; Siebers, Peer-Olaf

Abstract

The importance of Agent-Based Simulation (ABS) as scientific method to generate data for scientific models in general and for informed policy decisions in particular has been widely recognised. However, the important technique of code testing of implementations like unit testing has not generated much research interested so far. As a possible solution, in previous work we have explored the conceptual use of property-based testing. In this code testing method, model specifications and invariants are expressed directly in code and tested through automated and randomised test data generation.

This paper expands on our previous work and explores how to use property based testing on a technical level to encode and test specifications of ABS. As use case the simple agent-based SIR model is used, where it is shown how to test agent behaviour, transition probabilities and model invariants. The outcome are specifications expressed directly in code, which relate whole classes of random input to expected classes of output. During test execution, random test data is generated automatically, potentially covering the equivalent of thousands of unit tests, run within seconds on modern hardware. This makes property-based testing in the context of ABS strictly more powerful than unit testing, as it is a much more natural fit due to its stochastic nature.

Citation

Thaler, J., & Siebers, P.-O. (2020). Specification testing of agent-based simulation using property-based testing. Autonomous Agents and Multi-Agent Systems, 34(2), Article 47. https://doi.org/10.1007/s10458-020-09473-8

• Journal Article Type: Article
• Acceptance Date: Jun 9, 2020
• Online Publication Date: Jun 19, 2020
• Publication Date: 2020-10
• Deposit Date: Jun 22, 2020
• Publicly Available Date: Jun 20, 2021
• Journal: Autonomous Agents and Multi-Agent Systems
• Print ISSN: 1387-2532
• Electronic ISSN: 1573-7454
• Publisher: Springer Verlag
• Peer Reviewed: Peer Reviewed
• Volume: 34
• Issue: 2
• Article Number: 47
• DOI: https://doi.org/10.1007/s10458-020-09473-8
• Keywords: agent-based simulation testing, code testing, test driven development, model specification
• Public URL: https://nottingham-repository.worktribe.com/output/4701803
• Publisher URL: https://link.springer.com/article/10.1007/s10458-020-09473-8
• Additional Information: First Online: 19 June 2020