Det finns mycket man skulle kunna experimentera med för att förbättra modellen, mer meningsfulla data är dock det absolut viktigaste.
Maskininlärningen skulle kunna ske genom ett RNN [14], som skulle möjliggöra att ordning och sekvenser har betydelse för modellens output. Det skulle dock kräva mer förståelse och kunskap att sätta upp det rätt i Tensorflow, så att det inte blir en försämring jämfört med CNN.
Aktiveringsfunktionen skulle kunna bytas ut, exempelvis mot Exponential Linear Unit (ELU) [43] för både snabbare träning och eventuellt högre träffsäkerhet. Det kräver dock mer data än vad som fanns tillgängligt i denna studie.
Weight decay skulle kunna introduceras för att förbättra skyddet mot överträning på data [44]. Med weight decay menas att noders relevans minskar om de inte ändrats under en längre tid, vid träning. Detta gör att noder inte lika länge bli orörda under träning.
När det över tid finns fler och fler kvitton i databasen att träna på, så kommer ersättningen av dåligt OCR-tolkade ord, beskrivet i 4.2 bli bättre. Gränsen på 10 förekomster av ord kanske behöver ändras till något mer dynamiskt.
I sista delen i MoSCoW – Won’t så står det att maskininlärningsmodellen skulle kunna automatiskt uppdateras över tid. Maskininlärningsmodeller jobbar vanligtvis i isolation. Det gör att det blir svårt att uppdatera en modell, man brukar behöva träna helt nya från grunden för att lägga till nya data i träningsprocessen. Det finns en teknik som heter transfer learning [45].
Transfer learning har förmågan att bygga vidare på en redan tränad modell. Det skulle kunna vara en lösning på det problemet.
32
33
7 Referenser
[1] “Centsoft - Automatisera era leverantörsfakturor med vårt EFH.”
[Online]. Available: https://centsoft.se/. [Accessed: 03-Mar-2019].
[2] “Kvitton - Centsoft.” [Online]. Available: https://centsoft.se/kvitton/.
[Accessed: 26-May-2019].
[3] A. Chaudhuri, K. Mandaviya, P. Badelia, and S. K Ghosh, Optical Character Recognition Systems for Different Languages with Soft Computing, vol. 352. Cham: Springer International Publishing, 2017.
[4] Z. Nagy, “An Introduction to Regular Expressions,” in Regex Quick Syntax Reference, Berkeley, CA: Apress, 2018, pp. 1–8.
[5] “PyCharm: the Python IDE for Professional Developers by JetBrains.”
[Online]. Available: https://www.jetbrains.com/pycharm/. [Accessed:
18-Aug-2019].
[6] “GitHub.” [Online]. Available: https://github.com/. [Accessed: 18-Aug-2019].
[7] “Welcome to Python.org.” [Online]. Available: https://www.python.org/.
[Accessed: 17-Aug-2019].
[8] J. Whang and Z. Maurer, “Python Review, Presentation in course CS224n (Natural Language Processing with Deep Learning).”
[9] “Natural Language Toolkit — NLTK 3.4.4 documentation.” [Online].
Available: https://www.nltk.org/. [Accessed: 17-Aug-2019].
[10] L. Dolamic and J. Savoy, “When stopword lists make the difference,” J.
Am. Soc. Inf. Sci. Technol., vol. 61, no. 1, pp. 200–203, Jan. 2010.
[11] “Python Data Analysis Library — pandas: Python Data Analysis Library.”
[Online]. Available: https://pandas.pydata.org/. [Accessed: 19-May-2019].
[12] “Matplotlib: Python plotting — Matplotlib 3.1.1 documentation.”
[Online]. Available: https://matplotlib.org/. [Accessed: 17-Aug-2019].
[13] I. Idris, Python data analysis : learn how to apply powerful data analysis techniques with popular open source Python modules. Packt Pub, 2014.
[14] E. Alpaydin, Introduction to machine learning. .
[15] U. Michelucci, Applied Deep Learning. Berkeley, CA: Apress, 2018.
[16] J. E. Dayhoff and J. M. DeLeo, “Artificial neural networks,” Cancer, vol.
91, no. S8, pp. 1615–1635, Apr. 2001.
[17] X. Li and X. Wu, “Constructing Long Short-Term Memory based Deep Recurrent Neural Networks for Large Vocabulary Speech Recognition,”
Oct. 2014.
[18] “TensorFlow.” [Online]. Available: https://www.tensorflow.org/.
[Accessed: 03-Mar-2019].
[19] R. ALRashdi and S. O’Keefe, “Deep Learning and Word Embeddings for Tweet Classification for Crisis Response,” Mar. 2019.
[20] M. Cogswell, F. Ahmed, R. Girshick, L. Zitnick, and D. Batra,
“REDUCING OVERFITTING IN DEEP NETWORKS BY
DECORRELATING REPRESENTATIONS.”
[21] N. Srivastava, G. Hinton, A. Krizhevsky, and R. Salakhutdinov, “Dropout:
A Simple Way to Prevent Neural Networks from Overfitting,” 2014.
[22] A. Munoz, “Machine Learning and Optimization.”
[23] Z. Zhang, “Improved Adam Optimizer for Deep Neural Networks,” in 2018 IEEE/ACM 26th International Symposium on Quality of Service
34 (IWQoS), 2018, pp. 1–2.
[24] “Report on Text Classification using CNN, RNN & HAN - Jatana - Medium.” [Online]. Available: https://medium.com/jatana/report-on-text-classification-using-cnn-rnn-han-f0e887214d5f. [Accessed: 29-Aug-2019].
[25] “Keras | TensorFlow.” [Online]. Available:
https://www.tensorflow.org/guide/keras. [Accessed: 03-Mar-2019].
[26] Y. Kim, “Convolutional Neural Networks for Sentence Classification.”
[27] S. Gorard, How do I choose between different research methods? . [28] M. Denscombe, The good research guide : for small-scale social research
projects. McGraw-Hill/Open University Press, 2010.
[29] K. Malterud, “Qualitative research: standards, challenges, and guidelines,” Lancet, vol. 358, no. 9280, pp. 483–488, Aug. 2001.
[30] A. Håkansson, “Portal of Research Methods and Methodologies for Research Projects and Degree Projects, in Proceedings Of The International Conference On Frontiers In Education, 2013, pp. 67–73.”
[31] J. D. Farquhar, Case study research for business. .
[32] M. Bunge, Epistemology & Methodology I : Exploring the World.
Springer Netherlands, 1983.
[33] S. Eklund, Arbeta i projekt : individen, gruppen, ledaren.
Studentlitteratur, 2011.
[34] E. Miranda and Eduardo, “Time boxing planning,” ACM SIGSOFT Softw.
Eng. Notes, vol. 36, no. 6, p. 1, Nov. 2011.
[35] V. L.-S. physics doklady and undefined 1966, “Binary codes capable of correcting deletions, insertions, and reversals,” nymity.ch.
[36] dongjun-Lee, “Text Classification Models with Tensorflow.” [Online].
Available: https://github.com/dongjun-Lee/text-classification-models-tf. [Accessed: 15-Jun-2019].
[37] D. Gopinath, A. Taly, H. Converse, and C. S. Pasareanu, “Finding Invariants in Deep Neural Networks,” Apr. 2019.
[38] M. R. Karim, M. Sewak, and P. Pujari, Practical Convolutional Neural Networks : Implement advanced deep learning models using Python.
Packt Publishing, 2018.
[39] W. Liu, Y. Wen, Z. Yu, and M. Yang, “Large-Margin Softmax Loss for Convolutional Neural Networks,” 2016.
[40] Y. Tang, “Deep Learning using Linear Support Vector Machines.”
[41] D. P. Kingma and J. Lei Ba, “ADAM: A METHOD FOR STOCHASTIC OPTIMIZATION.”
[42] Q. F. Hassan, A. ur R. Khan, and S. A. Madani, Internet of things : challenges, advances, and applications. .
[43] D.-A. Clevert, T. Unterthiner, and S. Hochreiter, “Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs),” Nov. 2015.
[44] J. Schmidhuber, “Deep Learning in Neural Networks: An Overview,” Apr.
2014.
[45] D. Sarkar, R. Bali, and T. Ghosh, Hands-on transfer learning with Python : implement advanced deep learning and neural network models using TensorFlow and Keras.
35
TRITA-EECS-EX-2019:600
www.kth.se