Contrastive Explanations for Reinforcement Learning in terms of Expected Consequences
Machine Learning models become increasingly proficient in complex tasks. However, even for experts in the field, it can be difficult to understand what the model learned. This hampers trust and acceptance, and it obstructs the possibility to correct the model. There is therefore a need for transparency of machine learning models. The development of transparent classification models has received much attention, but there are few developments for achieving transparent Reinforcement Learning (RL) models. In this study we propose a method that enables a RL agent to explain its behavior in terms of the expected consequences of state transitions and outcomes. First, we define a translation of states and actions to a description that is easier to understand for human users. Second, we developed a procedure that enables the agent to obtain the consequences of a single action, as well as its entire policy. The method calculates contrasts between the consequences of a policy derived from a user query, and of the learned policy of the agent. Third, a format for generating explanations was constructed. A pilot survey study was conducted to explore preferences of users for different explanation properties. Results indicate that human users tend to favor explanations about policy rather than about single actions.
Authors

Are you an author of this paper? Check the Twitter handle we have for you is correct.

Jasper van der Waa (add twitter)
Jurriaan van Diggelen (add twitter)
Karel van den Bosch (add twitter)
Mark Neerincx (add twitter)
Ask The Authors

Ask the authors of this paper a question or leave a comment.

Read it. Rate it.
#1. Which part of the paper did you read?

#2. The paper contains new data or analyses that is openly accessible?
#3. The conclusion is supported by the data and analyses?
#4. The conclusion is of scientific interest?
#5. The result is likely to lead to future research?

Github
User:
None (add)
Repo:
None (add)
Stargazers:
0
Forks:
0
Open Issues:
0
Network:
0
Subscribers:
0
Language:
None
Youtube
Link:
None (add)
Views:
0
Likes:
0
Dislikes:
0
Favorites:
0
Comments:
0
Other
Sample Sizes (N=):
Inserted:
Words Total:
Words Unique:
Source:
Abstract:
None
07/23/18 06:50PM
4,986
1,402
Tweets
Memoirs: Contrastive Explanations for Reinforcement Learning in terms of Expected Consequences. https://t.co/4wtWNduPQv
BrundageBot: Contrastive Explanations for Reinforcement Learning in terms of Expected Consequences. Jasper van der Waa, Jurriaan van Diggelen, Karel van den Bosch, and Mark Neerincx https://t.co/UKZbqdATwd
Images
Related