Researchers
Anran Wang, M.S. in Financial Engineering, Graduated December 2019
Jieding He, M.S. in Financial Engineering, Graduated May 2020
Advisor:
Dr. Khaldoun Khashanah
Abstract
The application of machine learning methods in the financial field has become one of the hot topics, and the construction of investment port- folio is one of the key issues in the financial field. In this project, we use supervised learning and reinforcement learning to construct a portfo- lio respectively. In the supervised learning section, we combine wavelet decomposition with Long Short-Term Memory (LSTM) neural networks (WNN) to predict stock time series, and compare with the predicted value using FFT and original time series, where WNN has better prediction re- sults. The mean-variance portfolio constructed based on the prediction results has a superior performance than the benchmark. In the reinforce- ment learning section, we use deep reinforcement learning (DRL) to build a single stock trading model that can get more profit than a passive posi- tion. Moreover, we also use a deep reinforcement learning framework that combines different structures of agent, state and reward to build a portfo- lio. The results show that the DRL using Multilayer Perceptrons, wavelet coefficients and Sharpe ratios performs better. The portfolios based on DRL also outperform benchmarks. The application of deep reinforcement learning in the financial field still has a lot of energy waiting to be tapped.
Keywords: Portfolio Construction, Stocks Time Series, Wavelet Decomposition, Deep Learn- ing, Reinforcement Learning
Main Results
Wavelet neural network portfolio
We train some LSTM networks which use wavelet coefficients as input on 50 stocks separately. The trained networks can return predictions of the closing price of t + 1 for each stock and we construct a daily-adjusted mean-variance portfolio based on that. Which can be seen from figure 1 and 2, the portfolio have higher returns and less risk than the benchmark SPY. One million dollar in the portfolio can grows steadily.
Deep reinforcement learning portfolio
The performances of portfolios based on various deep reinforcement learning frameworks, which contain different observations, network structures and re- wards, are not the same. As shown in figure 3, 5 portfolios have higher return and all have less volatility than SPY. If we used σμ as an indicator, 7 portfolios are better than SPY. In figure 4, we can see the best three portfolios compete with SPY in the same time period directly.
Conclusion
In the section on building portfolios using supervised learning, we found that using wavelet decomposition combined with neural networks (we use LSTM) has a good effect in predicting time series. And compared with the use of FFT and original time series, there are higher accuracy and smaller errors. Based on the accurate time series prediction results provided by WNN, our mean-variance portfolio constructed using 50 S&P500 constituent stocks has a good performance. Compared with S&P500, the investment portfolio has higher yield and lower risk. WNN’s ability in stock time series prediction is a way should be considered to build a portfolio based on supervised learning.
In building a portfolio using deep reinforcement learning, we’ve tried some different structures. By comparing the yield results, it is shown that MLP as an agent, wavelet coefficient as state, and sharpe ratio as reward have advantages in such problems. DRL portfolios do have ability to surpass the benchmark but returns are not stable. Keep in mind that the construction of deep reinforcement learning structures is much more complicated than a supervised learning with labels. Many factors need to be improved and our model is still rough and simple. There is no doubt that the application of deep reinforcement learning in the portfolio still has a lot of energy to be developed, and as of now, this is a potential way to create an AI fund manager.