Causal Discovery with Reinforcement Learning
Discovering causal structure among a set of variables is a fundamental problem in many empirical sciences. Traditional score-based casual discovery methods rely on various local heuristics to search for a directly acyclic graph (DAG) according to a predefined score function. While these methods, e.g., greedy equivalence search (GES), may have attractive results with infinite samples and certain model assumptions, they are less satisfactory in practice due to finite data and possible violation of assumptions. Motivated by recent advances in neural combinatorial optimization, we propose to use reinforcement learning (RL) to search for the DAG with the best scoring. Our encoder-decoder model takes observable data as input and generates graph adjacency matrices that are used to compute corresponding rewards. The reward incorporates both the predefined score function and two penalty terms for enforcing acyclicity. In contrast with typical RL applications where the goal is to learn a policy, we use RL as a search strategy and our final output would be the graph, among all graphs generated during training, that achieves the best reward. We conduct experiments on both synthetic and real data, and show that the proposed approach not only has an improved search ability but also allows for a flexible score function under the acyclicity constraint.
NurtureToken New!

Token crowdsale for this paper ends in

Buy Nurture Tokens

Authors

Are you an author of this paper? Check the Twitter handle we have for you is correct.

Shengyu Zhu (edit)
Zhitang Chen (add twitter)
Ask The Authors

Ask the authors of this paper a question or leave a comment.

Read it. Rate it.
#1. Which part of the paper did you read?

#2. The paper contains new data or analyses that is openly accessible?
#3. The conclusion is supported by the data and analyses?
#4. The conclusion is of scientific interest?
#5. The result is likely to lead to future research?

Github
Stargazers:
39
Forks:
8
Open Issues:
1
Network:
8
Subscribers:
6
Language:
Jupyter Notebook
DAGs with NO TEARS: Continuous Optimization for Structure Learning
Youtube
Link:
None (add)
Views:
0
Likes:
0
Dislikes:
0
Favorites:
0
Comments:
0
Other
Sample Sizes (N=):
Inserted:
Words Total:
Words Unique:
Source:
Abstract:
None
06/11/19 06:03PM
6,081
1,860
Tweets
unsorsodicorda: RT @StatsPapers: Causal Discovery with Reinforcement Learning. https://t.co/xjuZzLU9wt
puneethmishra: RT @StatsPapers: Causal Discovery with Reinforcement Learning. https://t.co/xjuZzLU9wt
MarcosMatabuena: RT @StatsPapers: Causal Discovery with Reinforcement Learning. https://t.co/xjuZzLU9wt
dizzy_my_future: RT @StatsPapers: Causal Discovery with Reinforcement Learning. https://t.co/xjuZzLU9wt
RexDouglass: RT @StatsPapers: Causal Discovery with Reinforcement Learning. https://t.co/xjuZzLU9wt
cd_fuller: RT @StatsPapers: Causal Discovery with Reinforcement Learning. https://t.co/xjuZzLU9wt
ShirotaShin: RT @StatsPapers: Causal Discovery with Reinforcement Learning. https://t.co/xjuZzLU9wt
minsuk_chang: RT @StatsPapers: Causal Discovery with Reinforcement Learning. https://t.co/xjuZzLU9wt
StatsPapers: Causal Discovery with Reinforcement Learning. https://t.co/xjuZzLU9wt
arxiv_cs_LG: Causal Discovery with Reinforcement Learning. Shengyu Zhu and Zhitang Chen https://t.co/hXBIGoY4OS
BrundageBot: Causal Discovery with Reinforcement Learning. Shengyu Zhu and Zhitang Chen https://t.co/LOFOajqSDJ
Images
Related