Financial Trading as a Game: A Deep Reinforcement Learning Approach
An automatic program that generates constant profit from the financial market is lucrative for every market practitioner. Recent advance in deep reinforcement learning provides a framework toward end-to-end training of such trading agent. In this paper, we propose an Markov Decision Process (MDP) model suitable for the financial trading task and solve it with the state-of-the-art deep recurrent Q-network (DRQN) algorithm. We propose several modifications to the existing learning algorithm to make it more suitable under the financial trading setting, namely 1. We employ a substantially small replay memory (only a few hundreds in size) compared to ones used in modern deep reinforcement learning algorithms (often millions in size.) 2. We develop an action augmentation technique to mitigate the need for random exploration by providing extra feedback signals for all actions to the agent. This enables us to use greedy policy over the course of learning and shows strong empirical performance compared to more commonly used epsilon-greedy exploration. However, this technique is specific to financial trading under a few market assumptions. 3. We sample a longer sequence for recurrent neural network training. A side product of this mechanism is that we can now train the agent for every T steps. This greatly reduces training time since the overall computation is down by a factor of T. We combine all of the above into a complete online learning algorithm and validate our approach on the spot foreign exchange market.
Author

Are you an author of this paper? Check the Twitter handle we have for you is correct.

Chien Yi Huang (add twitter)
Ask The Authors

Ask the authors of this paper a question or leave a comment.

Read it. Rate it.
#1. Which part of the paper did you read?

#2. The paper contains new data or analyses that is openly accessible?
#3. The conclusion is supported by the data and analyses?
#4. The conclusion is of scientific interest?
#5. The result is likely to lead to future research?

Github
User:
None (add)
Repo:
None (add)
Stargazers:
0
Forks:
0
Open Issues:
0
Network:
0
Subscribers:
0
Language:
None
Youtube
Link:
None (add)
Views:
0
Likes:
0
Dislikes:
0
Favorites:
0
Comments:
0
Other
Sample Sizes (N=):
Inserted:
Words Total:
Words Unique:
Source:
Abstract:
None
07/09/18 06:53PM
4,796
1,803
Tweets
Grananqvist: #100DaysOfMLCode Day 93 - Reading Financial Trading as a Game: A Deep Reinforcement Learning Approach: https://t.co/6OqRNa46Gg
RiskDataScience: Financial #Trading as a Game: A Deep #Reinforcement Learning Approach. https://t.co/CFNTUhweAf via @QFinancePapers
RiskDataScience: Financial #Trading as a Game: A Deep #Reinforcement Learning Approach. https://t.co/CFNTUhweAf
saeedamenfx: Financial Trading as a Game: A Deep Reinforcement Learning Approach https://t.co/p22b6f9ks9 #QuantLinkADay
ericmjohn: RT @QFinancePapers: Financial Trading as a Game: A Deep Reinforcement Learning Approach. https://t.co/Wp8gZYM0VY
Justin_Goheen: RT @QFinancePapers: Financial Trading as a Game: A Deep Reinforcement Learning Approach. https://t.co/Wp8gZYM0VY
sussenglish: RT @QFinancePapers: Financial Trading as a Game: A Deep Reinforcement Learning Approach. https://t.co/Wp8gZYM0VY
Images
Related