DEGREE PROJECT, IN COMPUTER SCIENCE , FIRST LEVEL STOCKHOLM, SWEDEN 2015
Stock Trading with Neural Networks
ERIK IHRÉN, SEAN WENSTRÖM
Stock Trading with Neural Networks
ERIK IHRÉN SEAN WENSTRÖM
Degree Project in Computer Science, DD143X Supervisor: Alexander Kozlov
Examiner: Örjan Ekeberg
Abstract
Stock trading is increasingly done pseudo-automatically or fully automatically, using algorithms which make day-to-day or even moment-to-moment decisions.
This report investigates the possibility of creating a virtual stock trader, using a method used in Artificial Intelligence, called Neural Networks, to make intelligent decisions on when to buy and sell stocks on the stock market.
Referat
Aktiehandel genomförs till allt större grad automatiskt el-ler halvautomatiskt, med algoritmer som fattar beslut på daglig basis eller över ännu kortare tidsintervall.
Denna rapport undersöker möjligheten att göra en virtu-ell aktiehandlare med hjälp av en metod inom artificivirtu-ell intelligens kallad neurala nätverk, och fatta intelligenta be-slut om när aktier på aktiemarknaden ska köpas eller säljas.
CONTENTS
Appendices 24
A Stock data 25
1.
Introduction
Artificial Intelligence (AI) is starting to have an increasingly large impact on our lives. It is starting to be used in more advanced ways. The stock market is chaotic and hard to predict. [12] Using computers and algorithms to make sense of the patterns and simplify the stock market is not a new idea. Algorithmic trading is already used around the world and stands for more than 50 percent of the stocks traded in the US and EU today. [11]
The purpose of this study is to investigate the effectiveness of neural networks on making stock sale decisions based on previous history. Specifically, this study looks at a neural network that examines only the history of the given stock in order to make a decision, while not taking into account any external factors. For example, actions during a stock crash would have to be learned as a consequence of stock movements, as it should not be able to "predict" specifically that a crash is coming. The neural network is an AI concept based on mimicking the mechanics of the brain on the neuron level. A neural network is a network consisting of input, output and intermediary nodes, where all the nodes are connected with different weights. If these weights are tweaked just right, entering the input should result in the desired output.
1.1
Problem statement
This thesis aims to answer the following question: Can an AI agent trained with neural networks reliably earn money on the stock market?
2.
Background
There are several studies on automatic trading agents using algorithms to figure out when to buy and sell, but we did not manage to find any that uses neural networks for the decision making. Instead, many of them study the use of trading indicators such as moving average convergence/divergence (MACD) [4][15][3] and Linear Re-gression Trend Channel. [3]
Instead a lot of research has been done on the application of neural networks for finance forecasting. This effectively means guessing what the future stock prices will be. [12][8] These studies use neural networks to make those educated guesses about the future. Usually, the previous day’s stock price is entered as input and the neural network gives the next day’s price as output.
2.1
Neural networks
To understand how neural networks can be applied to stock trading, a basic in-troduction of the subject is needed. In our case, we want the neural network to take the recent stock history and return a stock decision. A stock decision in our algorithm is represented as one of three choices: "buy", "sell" or "do nothing". Neural networks can be seen as a function that takes a set of inputs and gives a set of outputs. A simple example of this with four inputs and one output can be seen below in figure 1.
2.1. NEURAL NETWORKS
Inside the function there’s a network of nodes.The nodes have connections be-tween them. Some of these nodes are input nodes and some are output nodes. When the input is inserted into the input nodes it propagates forward in the network and results in output in the output nodes.
As shown in the figure above, a simple neural network is often divided into three kinds of layers. One input layer, a number of hidden layers, and one output layer. Each node takes the incoming connections from each node in the previous layer and multiplies them with different weights to add them together to a sum. This sum is entered into the node’s activation function, which determines what the node outputs. [7] A visual representation can be seen in figure 2.
Figure 2: Weights entering a node that adds them together into a threshold
CHAPTER 2. BACKGROUND
One common activation function is a simple binary threshold function which is displayed in figure 3.
Figure 3: The threshold function is 0 until it reaches a threshold T. Then it
changes to 1.[6]
If the sum is above a threshold T, it outputs 1, otherwise it outputs 0.
In many applications, it can be preferable to have a continuous function to allow for output that is not either 0 or 1. For this, there are a couple of common candidates. One is the logistic sigmoid function 1+e1−x and another one is the hyperbolic tangent
function tanh(x). [7] Both are continuous and have a S-shape. An important differ-ence is that the logistic sigmoid function goes from to 0 to 1 while the hyperbolic tangent function goes from -1 to 1. Depending on the application, one can be prefer-able over the other. In figure 4 you can see the shape of the activation function tanh.
2.2. GENETIC ALGORITHMS
The weights in the network need to be set in a manner to enable it to make the correct decision based on the provided input. Doing this is non-trivial and depends completely on the task the network is to perform. This is called training the network. Normally, neural networks are trained using supervised learning. This means that there is a set of training data and the neural network is left to process the input and produce output. The output is then compared to the known result and used to tweak the weights.
In some cases, there is no known result for a certain input. It might be the case of trying to maximize some score, in which case it is not feasible to tweak weights towards a certain value. One way to solve this, is to train the network using genetic algorithms.
2.2
Genetic Algorithms
Genetic algorithms can be used as a way to train a neural network, or any other algorithm with variable values that affect its performance. Since we are using neural networks, which will have to be trained, we can use genetic algorithms to perform this training. Just as neural networks are based on how brains work in nature, genetic algorithms are based on how natural selection works in nature. [1]
Evolution in nature is largely based on natural selection. The idea is that the goal of organisms is to reproduce, and mutations that make it easier for the organ-ism to reproduce increases the chance of passing on its mutations. [2]
CHAPTER 2. BACKGROUND
3.
Method
We decided to create a simulation of a world to train the agents in. The world has a list of stocks and how those stocks will perform over time. It also has a set of agents in the world that are able to buy and sell stocks in the world.
3.1
Stocks
We wanted to have a long and varied history of stocks to train the agents on. We decided on a history span of five years because a longer history span would limit the companies we could pick from. A longer history span would also come to include the large economy crisis in 2008 which is a large external factor that we did not want. We divided the stocks into training data and reference data. The agents are scored based on their performance with the training data, which will evolve them to per-form better on these stocks. However, they are also tested on a separate set of data, our reference data, which they have never seen before. This was used to see how well the agents performed on data it had not been trained for.
We decided to train the neural networks on 13 different stocks. Having a lower number can cause overtraining where the neural networks become trained to just perform well on that specific data instead of generally adapting to stock trends. Having a higher number would increase the iteration time and lead to slower train-ing. We found 13 to be a good compromise. An equal number of different stocks were used as reference data.
The stocks were picked from some of NASDAQ’s most active stocks. [10] For infor-mation on the stocks used in the simulation, see Appendix A.
The recent history was used as input for the neural network. The weekly data uses the last 20 data points as input. This means 20 weeks of input. We decided that this would be enough to see the recent trends.
3.2
World
The world is just a simulation of the stock market. It contains stocks and agents buying and selling them. There are 300 agents and each gets to make individual decisions on how to trade the stocks in the world. The stock prices change each tick based on real historical data.
CHAPTER 3. METHOD
To simplify the simulation, agents’ actions do not impact each other. There is an unlimited amount of each stock and buying or selling a stock does not affect the stock value, which always follows the real history.
3.3
Agent
An agent is a virtual individual trading on our stock market. Each agent has a single neural network that it uses to make decisions on all stocks in the world. The neural network has randomized start weights.
Each agent starts with $10000. We used American dollars as currency to match the currency that our stocks are traded with. Each week, all agents go through the stocks and their neural networks make a decision on whether to buy or sell that stock or not.
The agent has no control over how much to buy or sell. The initial funds are divided into 13 equal parts. Each part is allocated for a particular stock. When an agent decides to buy a stock, it uses all of the allocated money for that stock. If the agents decides to sell a stock, it sells all of it. In addition, deciding to sell a stock that the agent doesn’t have does nothing. The same is true for buying a stock that the agent already owns.
We rate an agent’s performance as the funds it has at the end of the simulation and the value of all unsold stocks at the end of the simulation at their value at that point in time.
3.4
Neural Networks
As mentioned before, 20 points of data are used as the recent history. This means the neural network has 20 input nodes. The rest of the network consists of a hidden layer with 20 nodes and an output node with one node.
The agents will use the output from the single output node to decide to buy or sell. If the output value is higher than 0, it buys, otherwise it sells.
For the activation function, we used a function similar to the hyperbolic tangent function. The function used, |x|+1x , also goes from -1 to 1 and has a similar S-shape. However, it is much faster to calculate. Figure 5 shows the similarity in shape be-tween tanh and the quicker activation used.
3.5. GENETIC ALGORITHM
Since the activation function is an integral part of the simulation, and runs thou-sands of times each tick, a fast activation function is preferred. This results in a faster training time.
The neural network library was written by us in Java. We decided to implement our own version instead of using a version written by someone else for several rea-sons. We already had a library that was nearly complete. Writing our own was also an interesting programming exercise in itself, and allowed us to get a greater understanding of the subject of our thesis. Finally, implementing our own ensures that it contains exactly the features we want and that it works well with the rest of our code.
3.5
Genetic Algorithm
The genetic algorithm is used to improve the agents. It repeatedly runs the world simulation while evolving the population of agents between each iteration.
The population evolution initially sorts the agents by fitness. The worst ones are removed immediately. The rest are then combined to create new agents. This is done with roulette wheel selection where the probability of an agent being selected is proportional to its fitness. The agents are combined to create new ones until the population is back to its original size.
The only difference between the neural networks is their individual weights be-tween nodes. To combine two agents into a new agent, we go through the weights in their networks and for each one, we take the average of their weights.
Finally each agent is considered for mutation. The mutation is small but has a chance to go through and slightly modify an agent’s weights.
algo-CHAPTER 3. METHOD
rithm. We removed 50% of the agents each generation based on their performance. When the population is recreated using combinations of old agents, each agent is considered for mutation and has a 50% change to mutate. Mutating an agent mod-ifies the weight of each connection between two nodes in the network by a random value between -0.5 and 0.5.
3.6
Reversed data
As part of our method to get a more interesting result, we decided to reverse all stocks in the training data. We did this because our preliminary results showed the agents abusing a very simple strategy.
Whatever we did, the agents always decided to buy stocks and never sell. This was because over our long time period, most of our stocks went up. This meant that our stock agents would often earn the most by buying early and keeping it for the entire duration. This is why we decided to reverse the training data. Keeping the stocks long-term is not ideal since most stocks now trend downwards over time.
3.7
Hardware
4.
Results
The result is divided into sections with graphs, visualizing different aspects of the result. The result was acquired after simulating the world repeatedly until perfor-mance plateaued off, resulting in 30 minutes of training. All graphs shown are the average of the three best agents in a population of 300 agents.
Two different approaches are covered in the result sections - training under the standard data, and training under reversed stock data where most stocks used for training have a total negative slope. These approaches are referred to as "standard data" and "reversed data", respectively.
All Y axes give values in US dollars.
4.1
Buy/Sell decisions
This shows graphs of stock data and when the agents decided to buy or sell the stock. Blue dots mean the agent bought the stock and red dots mean the agent sold the stock. Only dots which resulted in a stock action are shown. This means that if the agent tried to sell after already selling all its stocks, the second sell action will not be shown. The same thing is true for consecutive buy actions.
This section is divided into two different sets of results. The first shows the re-sult from using the stock data normally, while the second displays the rere-sult of the algorithm when trained on reversed stock data. Both sets of results contain the buy/sell decisions on two stocks from the training data and two stocks from the reference data.
The stocks displayed in the result are the first two stocks from the set of train-ing data and two stocks from the set of reference data. However, generally all stocks in our result show similar results.
4.1.1 Standard data
Training data
CHAPTER 4. RESULTS
American Airlines Group Inc.
Reference data
4.1. BUY/SELL DECISIONS
Gilead Science Inc.
4.1.2 Reverse data
Training data
CHAPTER 4. RESULTS
American Airlines Group Inc.
Reference data
4.2. TRAINING IMPROVEMENT
Gilead Science Inc.
4.2
Training improvement
CHAPTER 4. RESULTS
4.2.1 Standard data
4.2.2 Reverse data
4.3
Sales profits
4.3. SALES PROFITS
4.3.1 Standard data
Training data
CHAPTER 4. RESULTS
4.3.2 Reverse data
Training data
5.
Discussion
The result shows several interesting things. To give a better overview of the discus-sion, it is divided into one section for each section of results. At the end there is also some general discussion around the results and method.
5.1
Buy/Sell decisions
As mentioned in the method in section 3.6, if you use the training data normally, the agents end up barely selling anything. This can be seen in section 4.1.1. A reasonable theory for this is that over the period of 4-5 years, most stocks go up above their initial price. Evolving the agents to mostly buy, regardless of input is easy since you only have to get the weights to mostly output over 0 regardless of input which causes the agent to buy. Since it turns out to be a very good strategy, the agents quickly converge towards it.
A partial explanation lies in our data selection. We went with the relatively narrow selection method of picking NASDAQ’s most traded stocks, by dollar value or share volume, at the time of writing, and further restricted these to include only those which have been listed for 5 years. Because of this, the stocks our agents train on and subsequently are applied to will predominantly be companies that have been increasing in value for some time. This may skew data by over-adapting the algo-rithm for companies which are already known to become successful some time down the line, which is information a live algorithm would naturally not have access to. While buying stocks as a long-term investment is a valid strategy, we wanted some-thing a bit more interesting that also tries to buy and sell often, by seeing some kind of pattern. We tried a lot to make it less monotone and less prone to buying all the time. For example we tried not rewarding agents for having unsold stock left at the simulation and we also added a transaction cost for purchases. None of these changes had the intended effect, and we removed them and went another direction We decided to see what would happen if we trained it on reversed data instead. This would mean that the stock data went backwards, which in turn means that many stocks generally go downwards over a longer period of time. We decided to only reverse the training data, which would let us observe how it would perform in real life after being trained when used on non-reversed data.
CHAPTER 5. DISCUSSION
5.2
Training improvement
Now the question is how these agents perform. This can be seen in section 4.2 which shows how the agents’ performance evolve over time. There are several interesting conclusions to draw from this.
Despite our efforts to make the agents buy and sell more, it turns out that the original strategy is more efficient. Using the standard, non-reversed data results in an average of $25, 000 after it reaches the end. This is $15, 000 more than it started with, which makes for an average increase of 150%. It reaches approximately the same values on the reference data that it was not trained for.
For the reversed data, this value is much lower. After some initial training, the end amount seems to hover around $14, 000 for the training data, meaning an aver-age increase of 40%. The reference data is very spiky and sometimes goes up high, but it stays below $20, 000 and averages $13, 000 (30% profit)
The NASDAQ 100 index increases by 85%[9] during the time period investigated. This means that although our agents trained on the reverse data are earning some money reliably, they are earning less than the index. This means that a better op-tion would be to invest in an investment fund and earn 85% profit instead of using the agent’s suggestions earning 30% profit.
The algorithm trained on standard data manages to earn a lot more than the in-dex. A likely partial explanation is found in our data selection, as mentioned earlier. It’s also worth noting that the reversed data algorithm earns money on its training data - and this is under circumstances where most stocks fall in value rather dra-matically over the entire period of time. This would seem to indicate that while this algorithm isn’t effective compared to some approaches during healthy economic growth, it seems to perform decently even when the economy is not doing well, which is not true for approaches like the index fund.
5.3
Sales profits
Section 4.3 shows the profits of each sale. As expected, the algorithm trained on standard data has very few sales, but the ones it has are often very profitable, since the agent waited a long time before selling it. Conversely, when trained on reverse data, the algorithm seems mostly random. It has a lot of positive sales and a lot of negative ones. It seems to be doing a lot of both, although we already know it ends up earning more than it loses.
5.4
General Discussion
5.4. GENERAL DISCUSSION
5.4.1 Data selection
There is a lot to be said about our choice of stocks. Our data, taken specifically from 2010-2015, encompasses a time without a major worldwide economic crash, and thus may fail to adapt to poor economic conditions. This could lead to worse performance outside the chosen target area. The target data and training data are taken from the same time period in this paper, but an algorithm for real-world application would have to look at past data, .
Additionally, training an algorithm on the same time period it is applied to brings the risk of overtraining - accidentally training the agent to respond to very specific stimuli that exists across the market at a given time instead of responding to more general movement. We sought to avoid this through a large pool of different stocks (which should help us avoid overspecialization to reactions on specific data profiles) and the selection of stocks which are heavily traded (which should reduce the indi-vidual stock’s volatility, making it somewhat more predictable).
Our results show that the algorithm performs the same on the training data as on the reference data, both when the data is normal and when it is reversed. This seems to show that having a large pool of different stocks allowed us to avoid over-training, at least to the point where there’s not a specific trigger that happens to exist across most of the training data, which the algorithm can react to.
5.4.2 Fast Evolution
Performance in our evolutionary process plateaus off rather quickly - usually within 10 minutes - into something a local maximum, after which little or no progress is made by further evolutionary cycles. (It’s worth noting that we have on several occasions allowed the training to run for up to 24 hours, still giving similar results.) Our breeding procedures may be partly responsible for this fast convergence. Ge-netic algorithms often splice traits when breeding, mixing and matching discrete values or behaviours to cross-breed useful traits into the offspring. In contrast, our breeding procedure averages out the weights of the two involved networks, which may be inefficient for producing fit offspring.
CHAPTER 5. DISCUSSION
5.5
Conclusion
Our question concerned whether or not a neural network evolved by a hands-off genetic algorithm could be used as a stock trader, and achieve positive results. While our results are clearly non-comprehensive and are only tested under lim-ited circumstances, we claim to have shown that this method holds potential to "reliably" earn money. The first result turned out to abuse our choice of stocks and keep stocks for several years before selling. This "easy" strategy resulted in 150% profit over 5 years. Our second attempt with the reversed training data ended up earning significantly less (40% over 5 years).
References
[1] M. Schoenauer A.E. Eiben. “Evolutionary Computing”. In: Information
Pro-cessing Letters 82.1 (2002), pp. 1–6. doi: http://dx.doi.org/10.1016/
S0020-0190(02)00204-1.
[2] Robert Brandon. “Natural Selection”. In: The Stanford Encyclopedia of
Phi-losophy. Ed. by Edward N. Zalta. url: http : / / plato . stanford . edu /
archives/spr2014/entries/natural-selection/.
[3] Dennis Ekström. “Automated Foreign Exchange Trading Strategies:Improving Performance Without Strategy Modification”. B.S. Thesis. KTH, School of Computer Science and Communication (CSC), 2014.
[4] Andreas Falk and Johannes Moberg. “Algorithmic trading using MACD sig-nals”. B.S. Thesis. KTH, School of Computer Science and Communication (CSC), 2014, p. 39.
[5] Kiyoshi Kawaguchi. url: http://wwwold.ece.utep.edu/research/webfuzzy/ docs/kk-thesis/kk-thesis-html/img18.gif.
[6] Kiyoshi Kawaguchi. url: http://wwwold.ece.utep.edu/research/webfuzzy/ docs/kk-thesis/kk-thesis-html/img17.gif.
[7] David Kriesel. A Brief Introduction to Neural Networks. 2007. url: http: //www.dkriesel.com.
[8] Chi-Jie Lu and Jui-Yu Wu. “An efficient CMAC neural network for stock index forecasting”. In: Expert Systems with Applications 38.12 (2011), pp. 15194 – 15201. issn: 0957-4174. doi: http://dx.doi.org/10.1016/j.eswa.2011. 05 . 082. url: http://www.sciencedirect.com/science/article/pii/ S0957417411008669.
[9] NASDAQ 100 Index. https://www.avanza.se/index/om- indexet.html/
155541/nasdaq-100-index. Accessed: 2015-04-14.
[10] NASDAQ Most Active. http://www.nasdaq.com/markets/most- active.
aspx. Accessed: 2015-03-12.
[11] G. Nuti et al. “Algorithmic Trading”. In: Computer 44.11 (2011), pp. 61–69. issn: 0018-9162. doi: 10.1109/MC.2011.31.
[12] Kyong Joo Oh and Kyoung jae Kim. “Analyzing stock market tick data us-ing piecewise nonlinear model”. In: Expert Systems with Applications 22.3 (2002), pp. 249 –255. issn: 0957-4174. doi: http://dx.doi.org/10.1016/ S0957-4174(01)00058-6. url: http://www.sciencedirect.com/science/ article/pii/S0957417401000586.
[13] Ciumac Sergiu. url: http://www.codeproject.com/KB/dotnet/predictor/ network.jpg.
[14] Stock historical data. http://www.stockhistoricaldata.com/. Accessed:
REFERENCES
[15] Numan Ülkü and Eugeniu Prodan. “Drivers of technical trend-following rules’ profitability in world stock markets”. In: International Review of Financial
Analysis 30.0 (2013), pp. 214 –229. issn: 1057-5219. doi: http://dx.doi.
A.
Stock data
This section lists the different stocks used in the simulation as well as a graph showing their stock history from the last five years. The stock data was provided by [14]. The X-axis is time and the Y-axis is price in US dollars. The data is divided into two parts, the set of training data and the set of reference data. Each set contains 13 stocks.
A.1
Training data
Alcoa Inc.APPENDIX A. STOCK DATA
Almaden Minerals Ltd.
AeroCentury Corp.
A.1. TRAINING DATA
Amazon.com Inc.
APPENDIX A. STOCK DATA
Bank of America Corporation
A.1. TRAINING DATA
Chesapeake Energy Corporation
APPENDIX A. STOCK DATA
Catamaran Corporation
A.2. REFERENCE DATA
A.2
Reference data
Frontier Communications Corporation
APPENDIX A. STOCK DATA
Intel Corporation
A.2. REFERENCE DATA
Microsoft Corporation
APPENDIX A. STOCK DATA
Office Depot, Inc.
A.2. REFERENCE DATA
The Priceline Group Inc.
APPENDIX A. STOCK DATA
QUALCOMM Incorporated
A.2. REFERENCE DATA
