ACE: An Actor Ensemble Algorithm for Continuous Control with Tree Search
In this paper, we propose an actor ensemble algorithm, named ACE, for continuous control with a deterministic policy in reinforcement learning. In ACE, we use actor ensemble (i.e., multiple actors) to search the global maxima of the critic. Besides the ensemble perspective, we also formulate ACE in the option framework by extending the option-critic architecture with deterministic intra-option policies, revealing a relationship between ensemble and options. Furthermore, we perform a look-ahead tree search with those actors and a learned value prediction model, resulting in a refined value estimation. We demonstrate a significant performance boost of ACE over DDPG and its variants in challenging physical robot simulators.
Authors

Are you an author of this paper? Check the Twitter handle we have for you is correct.

Shangtong Zhang (edit)
Hao Chen (add twitter)
Hengshuai Yao (add twitter)
Ask The Authors

Ask the authors of this paper a question or leave a comment.

Read it. Rate it.
#1. Which part of the paper did you read?

#2. The paper contains new data or analyses that is openly accessible?
#3. The conclusion is supported by the data and analyses?
#4. The conclusion is of scientific interest?
#5. The result is likely to lead to future research?

Github
Stargazers:
914
Forks:
196
Open Issues:
2
Network:
196
Subscribers:
46
Language:
Python
Modularized Implementation of Deep RL Algorithms in PyTorch
Youtube
Link:
None (add)
Views:
0
Likes:
0
Dislikes:
0
Favorites:
0
Comments:
0
Other
Sample Sizes (N=):
Inserted:
Words Total:
Words Unique:
Source:
Abstract:
None
11/07/18 06:04PM
7,922
1,953
Tweets
arxiv_pop: 2018/11/06 投稿 4位 LG(Machine Learning) ACE: An Actor Ensemble Algorithm for Continuous Control with Tree Search https://t.co/LdLc6OkU6P 4 Tweets 8 Retweets 45 Favorites
whi_rl: "ACE: An Actor Ensemble Algorithm for Continuous Control with Tree Search" - Shangtong Zhang, Hao Chen, Hengshuai Yao https://t.co/b6uYdaeHls
arxivml: "ACE: An Actor Ensemble Algorithm for Continuous Control with Tree Search", Shangtong Zhang, Hao Chen, Hengshuai Yao https://t.co/RaSrZeQM64
ShangtongZhang: Our paper ACE: An Actor Ensemble Algorithm for Continuous Control with Tree Search at #AAAI2019 is available now https://t.co/N47AdqqW0y. We extend TreeQN to continuous action and provide the option-critic theorem for deterministic intro-option policies.
pranjaltandon2: RT @Miles_Brundage: "ACE: An Actor Ensemble Algorithm for Continuous Control with Tree Search," Zhang et al.: https://t.co/qDDADDFIC5
Images
Related