Paul Knott
Evaluating the Robustness of Collaborative Agents
Knott, Paul; Carroll, Micah; Devlin, Sam; Ciosek, Kamil; Hofmann, Katja; Dragan, Anca; Shah, Rohin
Authors
Micah Carroll
Sam Devlin
Kamil Ciosek
Katja Hofmann
Anca Dragan
Rohin Shah
Abstract
Artificial agents trained by deep reinforcement learning will likely encounter novel situations after deployment that were never seen during training. Our agent must be robust to handle such situations well. However, if we cannot rely on the average training or validation reward as a metric, then how can we effectively evaluate robustness? We take inspiration from the practice of unit testing in software engineering. Specifically, we suggest that when designing AI agents that collaborate with humans, designers should search for potential edge cases in possible partner behavior and possible states encountered, and write tests which check that the behavior of the agent in these edge cases is reasonable. We apply this methodology to build a suite of unit tests for the Overcooked-AI environment, and use this test suite to evaluate three proposals for improving robustness. We find that the test suite provides significant insight into the effects of these proposals that were generally not revealed by looking solely at the average validation reward. For our full paper, see arxiv.org/abs/2101.05507.
Citation
Knott, P., Carroll, M., Devlin, S., Ciosek, K., Hofmann, K., Dragan, A., & Shah, R. (2021, May). Evaluating the Robustness of Collaborative Agents. Presented at 20th International Conference on Autonomous Agents and Multiagent Systems, Virtual Event United Kingdom
Presentation Conference Type | Edited Proceedings |
---|---|
Conference Name | 20th International Conference on Autonomous Agents and Multiagent Systems |
Start Date | May 3, 2021 |
End Date | May 7, 2021 |
Acceptance Date | Feb 17, 2020 |
Publication Date | May 3, 2021 |
Deposit Date | Feb 5, 2021 |
Publicly Available Date | Feb 5, 2021 |
Publisher | Association for Computing Machinery (ACM) |
Pages | 1560-1562 |
Book Title | AAMAS '21: Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems |
ISBN | 9781450383073 |
Public URL | https://nottingham-repository.worktribe.com/output/5291583 |
Publisher URL | https://dl.acm.org/doi/10.5555/3463952.3464159 |
Related Public URLs | https://aamas2021.soton.ac.uk/ |
Files
Knott AAMAS Ext Abs 2021
(1.3 Mb)
PDF
Downloadable Citations
About Repository@Nottingham
Administrator e-mail: discovery-access-systems@nottingham.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search