Hostname: page-component-78c5997874-v9fdk Total loading time: 0 Render date: 2024-11-05T21:27:53.233Z Has data issue: false hasContentIssue false
  • English
  • Français

RANDOM NEURAL NETWORK METHODS AND DEEP LEARNING

Published online by Cambridge University Press:  30 January 2019

Yonghua Yin Open the ORCID record for Yonghua Yin [Opens in a new window]
Yonghua Yin*
Affiliation:
Intelligent Systems and Networks Group Department of Electrical and Electronic Engineering Imperial College, London SW7 2BT, UK E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

The random neural network (RNN) is a mathematical model for an “integrate and fire” spiking network that closely resembles the stochastic behavior of neurons in mammalian brains. Since its proposal in 1989, there have been numerous investigations into the RNN's applications and learning algorithms. Deep learning (DL) has achieved great success in machine learning. Recently, the properties of the RNN for DL have been investigated, in order to combine their power. Recent results demonstrate that the gap between RNNs and DL can be bridged and the DL tools based on the RNN are faster and can potentially be used with less energy expenditure than existing methods.

Keywords

computational probabilityqueueing theorystochastic modeling
Type
Research Article
Information
Probability in the Engineering and Informational Sciences , Volume 35 , Special Issue 1: Learning, Optimization, and Theory of G-Networks , January 2021 , pp. 6 - 36
Copyright
Copyright © Cambridge University Press 2019

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1.Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., & Zheng, X. (2015). TensorFlow: Large-scale machine learning on heterogeneous systems, software available from tensorflow.org. [Online]. Available: http://tensorflow.org/.Google Scholar
2.Abdelbaki, H.E. (1999). Random neural network simulator for use with matlab. Technical report.Google Scholar
3.Abdelbaki, H., Gelenbe, E., & El-Khamy, S.E. (1999). Random neural network decoder for error correcting codes. Neural Networks, 1999. IJCNN'99. International Joint Conference on. Vol. 5. IEEE, 3241–3245.CrossRefGoogle Scholar
4.Abdelbaki, H., Gelenbe, E., & El-Khamy, S.E. (2000). Analog hardware implementation of the random neural network model. Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium. Vol. 4, 197–201.Google Scholar
5.Abdelbaki, H.M., Hussain, K., & Gelenbe, E. (2001). A laser intensity image based automatic vehicle classification system. Intelligent Transportation Systems, 2001. Proceedings. 2001 IEEE. IEEE, 460–465.CrossRefGoogle Scholar
6.Adeel, A., Larijani, H., & Ahmadinia, A. (2015). Resource management and inter-cell-interference coordination in lte uplink system using random neural network and optimization. IEEE Access 3: 1963–1979.CrossRefGoogle Scholar
7.Aguilar, J. & Gelenbe, E. (1997). Task assignment and transaction clustering heuristics for distributed systems. Information Sciences 97(1–2): 199–219.CrossRefGoogle Scholar
8.Anthony, M., Bartlett, P.L. (2009) Neural network learning: theoretical foundations. New York: Cambridge University Press.Google Scholar
9.Atalay, V., Gelenbe, E., & Yalabik, N. (1992). The random neural network model for texture generation. International Journal of Pattern Recognition and Artificial Intelligence 6(01): 131–141.CrossRefGoogle Scholar
10.Bakircioglu, H. & Gelenbe, E. (1998). Random neural network recognition of shaped objects in strong clutter. Applications of artificial neural networks in image processing III. Vol. 3307. International Society for Optics and Photonics, 22–29.Google Scholar
11.Bakırcıoğlu, H. & Koçak, T. (2000). Survey of random neural network applications. European Journal of Operational Research 126(2): 319–330.CrossRefGoogle Scholar
12.Basterrech, S., Mohammed, S., Rubino, G., & Soliman, M. (2009). Levenberg-marquardt training algorithms for random neural networks. The Computer Journal 54(1): 125–135.Google Scholar
13.Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L.D., Monfort, M., Muller, U., Zhang, J., Zhang, X., Zhao, J., & Zieba, K. (Apr. 2016). End to End Learning for Self-Driving Cars, ArXiv e-prints.Google Scholar
14.Bousquet, O. & Bottou, L. (2008). The tradeoffs of large scale learning. Advances in neural information processing systems, 161–168.Google Scholar
15.Brun, O., Wang, L., & Gelenbe, E. (2016). Big data for autonomic intercontinental overlays. IEEE Journal on Selected Areas in Communications 34(3): 575–583.CrossRefGoogle Scholar
16.Brun, O., Yin, Y., Gelenbe, E., Kadioglu, Y.M., Augusto-Gonzalez, J., & Ramos, M. (2018). Deep learning with dense random neural networks for detecting attacks against iot-connected home environments. Security in Computer and Information Sciences: First International ISCIS Security Workshop 2018, Euro-CYBERSEC 2018, London, UK, February 26–27, 2018. Lecture Notes CCIS No. 821, Springer Verlag.Google Scholar
17.Brun, O., Yin, Y., Gelenbe, E., Kadioglu, Y., Augusto-Gonzalez, J., & Ramos, M. (2018). Deep learning with dense random neural networks for detecting attacks against iot-connected home environments. In Gelenbe, E., Campegiani, P., Czachorski, T., Katsikas, S., Komnios, I., Romano, L., & Tzovaras, D., (eds.), Recent Cybersecurity Research in Europe: Proceedings of the 2018 ISCIS Security Workshop, Imperial College London. Lecture Notes CCIS No. 821, Springer Verlag.Google Scholar
18.Brun, O., Yin, Y., & Gelenbe, E. Deep learning with dense random neural network for detecting attacks against iot-connected home environments, Procedia Computer Science. Vol. 134, 458 – 463, 2018, the 15th International Conference on Mobile Systems and Pervasive Computing (MobiSPC 2018) / The 13th International Conference on Future Networks and Communications (FNC-2018) / Affiliated Workshops. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S1877050918311487.CrossRefGoogle Scholar
19.Burks, A.W., Goldstine, H.H., & von Neumann, J. (1946). Preliminary discussion of the logical design of an electronic computing instrument. Report to the US Army Ordenance Department.Google Scholar
20.Cai, D., He, X., Hu, Y., Han, J., & Huang, T. (2007). Learning a spatially smooth subspace for face recognition. 2007 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1–7.CrossRefGoogle Scholar
21.Cambria, E., Huang, G.-B., Kasun, L.L.C., Zhou, H., Vong, C.M., Lin, J., Yin, J., Cai, Z., Liu, Q., Li, K., Leung, V.C.M., Feng, L., Ong, Y.-S., Lim, M.-H., Akusok, A., Lendasse, A., Corona, F., Nian, R., Miche, Y., Gastaldo, P., Zunino, R., Decherchi, S., Yang, X., Mao, K., Oh, B.-S., Jeon, J., Toh, K.-A., Teoh, A.B.J., Kim, J., Yu, H., Chen, Y., & Liu, J. (2013). Extreme learning machines [trends & controversies]. IEEE Intelligent Systems 28(6): 30–59. [Online]. Available: http://dx.doi.org/10.1109/MIS.2013.140.CrossRefGoogle Scholar
22.Carnevale, N.T. & Hines, M.L. (2006). The NEURON book. New York: Cambridge University Press.CrossRefGoogle Scholar
23.çerkez, C., Aybay, I., & Halici, U. (1997). A digital neuron realization for the random neural network model. Neural Networks, 1997. International Conference on. Vol. 2. IEEE, 1000–1004.Google Scholar
24.Chang, C.-C. & Lin, C.-J. (2011). Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST) 2(3): 27.Google Scholar
25.Cheng, H.-P., Wen, W., Song, C., Liu, B., Li, H., & Chen, Y. (2016). Exploring the optimal learning technique for ibm truenorth platform to overcome quantization loss. Nanoscale Architectures (NANOARCH), 2016 IEEE/ACM International Symposium on. IEEE, 185–190.Google Scholar
26.Cramer, C. & Gelenbe, E. (2000). Video quality and traffic qos in learning-based subsampled and receiver-interpolated video sequences. IEEE Journal on Selected Areas in Communications 18(2): 150–167. [Online]. Available: https://doi.org/10.1109/49.824788.CrossRefGoogle Scholar
27.Cramer, C., Gelenbe, E., & Bakircloglu, H. (1996). Low bit-rate video compression with neural networks and temporal subsampling. Proceedings of the IEEE 84(10): 1529–1543.CrossRefGoogle Scholar
28.Cramer, C., Gelenbe, E., & Gelenbe, P. (1998). Image and video compression. IEEE Potentials 17(1): 29–33.10.1109/45.652854CrossRefGoogle Scholar
29.Cramer, C., Gelenbe, E., & Bakircioglu, H. (1996). Video compression with random neural networks. Neural Networks for Identification, Control, Robotics, and Signal/Image Processing, 1996. Proceedings., International Workshop on. IEEE, 476–484.CrossRefGoogle Scholar
30.Davison, A., Brüderle, D., Kremkow, J., Muller, E., Pecevski, D., Perrinet, L., & Yger, P. (2009). Pynn: a common interface for neuronal network simulators.Google Scholar
31.Diehl, P.U., Pedroni, B.U., Cassidy, A., Merolla, P., Neftci, E., & Zarrella, G. (2016). Truehappiness: Neuromorphic emotion recognition on truenorth. Neural Networks (IJCNN), 2016 International Joint Conference on. IEEE, 4278–4285.CrossRefGoogle Scholar
32.Ding, C., Li, T., Peng, W., & Park, H. (2006). Orthogonal nonnegative matrix t-factorizations for clustering. Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 126–135.CrossRefGoogle Scholar
33.Ding, C.H., He, X., & Simon, H.D. (2005). On the equivalence of nonnegative matrix factorization and spectral clustering. SDM. Vol. 5. SIAM, 606–610.Google Scholar
34.Dominguez-Morales, J.P., Jimenez-Fernandez, A., Rios-Navarro, A., Cerezuela-Escudero, E., Gutierrez-Galan, D., Dominguez-Morales, M.J., & Jimenez-Moreno, G. (2016). Multilayer spiking neural network for audio samples classification using spinnaker. International Conference on Artificial Neural Networks. Springer, 45–53.CrossRefGoogle Scholar
35.Esser, S.K., Andreopoulos, A., Appuswamy, R., Datta, P., Barch, D., Amir, A., Arthur, J., Cassidy, A., Flickner, M., Merolla, P. et al. (2013). Cognitive computing systems: Algorithms and applications for networks of neurosynaptic cores. Neural Networks (IJCNN), The 2013 International Joint Conference on. IEEE, 1–10.CrossRefGoogle Scholar
36.Esser, S.K., Appuswamy, R., Merolla, P., Arthur, J.V., & Modha, D.S. (2015). Backpropagation for energy-efficient neuromorphic computing. Advances in Neural Information Processing Systems, 1117–1125.Google Scholar
37.Esser, S.K., Merolla, P.A., Arthur, J.V., Cassidy, A.S., Appuswamy, R., Andreopoulos, A., Berg, D.J., McKinstry, J.L., Melano, T., Barch, D.R. et al. (2016). Convolutional networks for fast, energy-efficient neuromorphic computing. Proceedings of the National Academy of Sciences 113: 201604850.CrossRefGoogle ScholarPubMed
38.Fourneau, J.-M. & Gelenbe, E. (2017). G-networks with adders. Future Internet 9(3), 34.CrossRefGoogle Scholar
39.François, F. & Gelenbe, E. (2016). Towards a cognitive routing engine for software defined networks. ICC 2016. IEEE, 1–6.CrossRefGoogle Scholar
40.François, F. & Gelenbe, E. (2016). Optimizing secure sdn-enabled inter-data centre overlay networks through cognitive routing. MASCOTS 2016, IEEE Computer Society. IEEE, 283–288.CrossRefGoogle Scholar
41.Furber, S.B., Galluppi, F., Temple, S., & Plana, L.A. (2014). The spinnaker project. Proceedings of the IEEE 102(5): 652–665.CrossRefGoogle Scholar
42.Gelenbe, E. (1989). Random neural networks with negative and positive signals and product form solution. Neural Computation 1(4): 502–510.CrossRefGoogle Scholar
43.Gelenbe, E. (1989). Réseaux stochastiques ouverts avec clients négatifs et positifs, et réseaux neuronaux. Comptes-Rendus Acad. Sciences de Paris, Série 2 309: 979–982.Google Scholar
44.Gelenbe, E. (1990). Reseaux neuronaux aléatoires stables. Comptes Rendus de l'Académie des Sciences. Série 2 310(3): 177–180.Google Scholar
45.Gelenbe, E. (1990). Stability of the random neural network model. Neural Computation 2(2): 239–247.CrossRefGoogle Scholar
46.Gelenbe, E. (1991). Product-form queueing networks with negative and positive customers. Journal of Applied Probability 28(3): 656–663.CrossRefGoogle Scholar
47.Gelenbe, E. (1993). G-networks by triggered customer movement. Journal of Applied Probability 30(3): 742–748.CrossRefGoogle Scholar
48.Gelenbe, E. (1993). G-networks with signals and batch removal. Probability in the Engineering and Informational Sciences 7(3): 335–342.CrossRefGoogle Scholar
49.Gelenbe, E. (1993). Learning in the recurrent random neural network. Neural Computation 5: 154–164.CrossRefGoogle Scholar
50.Gelenbe, E. (1994). G-networks: a unifying model for neural and queueing networks. Annals of Operations Research 48(5): 433–461.CrossRefGoogle Scholar
51.Gelenbe, E. (2007). Steady-state solution of probabilistic gene regulatory networks. Physical Review E 76: 031903-1–031903-8.CrossRefGoogle ScholarPubMed
52.Gelenbe, E. (2009). Steps toward self-aware networks. Communications of the ACM 52(7): 66–75.CrossRefGoogle Scholar
53.Gelenbe, E. (2012). Natural computation. The Computer Journal 55(7): 848–851.CrossRefGoogle Scholar
54.Gelenbe, E. & Abdelrahman, O.H. (2018). An energy packet network model for mobile networks with energy harvesting. Nonlinear Theory and Its Applications, IEICE 9(3): 1–15.CrossRefGoogle Scholar
55.Gelenbe, E. & Ceran, E.T. (2016). Energy packet networks with energy harvesting. IEEE Access 4: 1321–1331.CrossRefGoogle Scholar
56.Gelenbe, E. & Cramer, C. (1998). Oscillatory corticothalamic response to somatosensory input. Bio Systems 48(1–3): 67–75.CrossRefGoogle ScholarPubMed
57.Gelenbe, E. & Fourneau, J.-M. (1999). Random neural networks with multiple classes of signals. Neural Computation 11(4): 953–963.CrossRefGoogle ScholarPubMed
58.Gelenbe, E. & Hussain, K.F. (2002). Learning in the multiple class random neural network. IEEE Transactions on Neural Networks 13(6): 1257–1267.CrossRefGoogle ScholarPubMed
59.Gelenbe, E. & Kazhmaganbetova, Z. (2014). Cognitive packet network for bilateral asymmetric connections. IEEE Trans. Industrial Informatics 10(3): 1717–1725.CrossRefGoogle Scholar
60.Gelenbe, E. & Marin, A. (2015). Interconnected wireless sensors with energy harvesting. Analytical and Stochastic Modelling Techniques and Applications - 22nd International Conference, ASMTA 2015, Albena, Bulgaria, May 26–29, 2015. Proceedings, 87–99.CrossRefGoogle Scholar
61.Gelenbe, E. & Morfopoulou, C. (2010). A framework for energy-aware routing in packet networks. The Computer Journal 54(6): 850–859.CrossRefGoogle Scholar
62.Gelenbe, E. & Pujolle, G. (1987). Introduction to Networks of Queues. Translation of “Introduction aux Réseaux de Files d'Attente”, Eyrolles, Paris, 1982, published by John Wiley Ltd, New York and Chichester.Google Scholar
63.Gelenbe, E. & Schassberger, R. (1992). Stability of product form g-networks. Probability in the Engineering and Informational Sciences 6(3): 271–276.CrossRefGoogle Scholar
64.Gelenbe, E. & Sungur, M. (1994). Random network learning and image compression. Neural Networks, 1994. IEEE World Congress on Computational Intelligence., 1994 IEEE International Conference on. Vol. 6. IEEE, 3996–3999.CrossRefGoogle Scholar
65.Gelenbe, E. & Timotheou, S. (2008). Random neural networks with synchronized interactions. Neural Computation 20(9): 2308–2324.CrossRefGoogle ScholarPubMed
66.Gelenbe, E. & Yin, Y. (2016). Deep learning with random neural networks. 2016 International Joint Conference on Neural Networks (IJCNN), 1633–1638.CrossRefGoogle Scholar
67.Gelenbe, E. & Yin, Y. (2016). Deep learning with random neural networks. SAI Intelligent Systems Conference 2016, 907–912.CrossRefGoogle Scholar
68.Gelenbe, E. & Yin, Y. (2017). Deep learning with dense random neural networks. International Conference on Man–Machine Interactions. Springer, 3–18.Google Scholar
69.Gelenbe, E., Glynn, P., & Sigman, K. (1991). Queues with negative arrivals. Journal of Applied Probability 28(1): 245–250.CrossRefGoogle Scholar
70.Gelenbe, E., Stafylopatis, A., & Likas, A. (1991). Associative memory operations of the random neura network. Proceedings of the International Conference on Artificial Neural Networks, 307–312.Google Scholar
71.Gelenbe, E., Feng, Y., & Krishnan, K.R.R. (1996). Neural network methods for volumetric magnetic resonance imaging of the human brain. Proceedings of the IEEE 84(10): 1488–1496.CrossRefGoogle Scholar
72.Gelenbe, E., Sungur, M., Cramer, C., & Gelenbe, P. (1996). Traffic and video quality with adaptive neural compression. Multimedia Systems 4(6): 357–369. [Online]. Available: https://doi.org/10.1007/s005300050037.CrossRefGoogle Scholar
73.Gelenbe, E., Ghanwani, A., & Srinivasan, V. (1997). Improved neural heuristics for multicast routing. IEEE Journal on Selected Areas in Communications 15(2): 147–155.CrossRefGoogle Scholar
74.Gelenbe, E., Mao, Z., & Li, Y. (Aug 1999). Approximation by random networks with bounded number of layers. Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468), 166–175.Google Scholar
75.Gelenbe, E., Mao, Z., & Li, Y. (1999). Function approximation with spiked random networks. IEEE Transactions on Neural Networks 10(1): 3–9.CrossRefGoogle ScholarPubMed
76.Gelenbe, E., Hussain, K.F., & Abdelbaki, H. (2000). Random neural network texture model. Applications of Artificial Neural Networks in Image Processing V . Vol. 3962. International Society for Optics and Photonics, 104–112.Google Scholar
77.Gelenbe, E., Koçak, T., & Wang, R. (2004). Wafer surface reconstruction from top–down scanning electron microscope images. Microelectronic Engineering 75(2): 216–233.CrossRefGoogle Scholar
78.Gelenbe, E., Mao, Z.-H., & Li, Y.-D. (2004). Function approximation by random neural networks with a bounded number of layers. Differential Equations and Dynamical Systems 12(1): 143–170.Google Scholar
79.Georgiopoulos, M., Li, C., & Kocak, T. (2011). Learning in the feed-forward random neural network: a critical review. Performance Evaluation 68(4): 361–384, g-Networks and their Applications. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0166531610000970.CrossRefGoogle Scholar
80.Gewaltig, M.-O. & Diesmann, M. (2007). Nest (neural simulation tool). Scholarpedia 2(4): 1430.CrossRefGoogle Scholar
81.Glorot, X. & Bengio, Y. (2010). Understanding the difficulty of training deep feedforward neural networks. Aistats. Vol. 9, 249–256.Google Scholar
82.Glorot, X., Bordes, A., & Bengio, Y., Deep sparse rectifier neural networks, In Gordon, G. J. & Dunson, D. B., (eds.), Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics (AISTATS-11), vol. 15. Journal of Machine Learning Research - Workshop and Conference Proceedings, 2011, 315–323. [Online]. Available: http://www.jmlr.org/proceedings/papers/v15/glorot11a/glorot11a.pdf.Google Scholar
83.Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative adversarial nets. In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N. D., & Weinberger, K. Q., (eds.), Advances in Neural Information Processing Systems 27, Curran Associates, Inc., 2672–2680. [Online]. Available: http://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf.Google Scholar
84.Goodman, D. & Brette, R. (2008). Brian: a simulator for spiking neural networks in python.CrossRefGoogle Scholar
85.Grenet, I., Yin, Y., Comet, J.-P., & Gelenbe, E. (2018). Machine learning to predict toxicity of compounds. 27th Annual International Conference on Artificial Neural Networks, ICANN18, accepted for publication. Springer Verlang.CrossRefGoogle Scholar
86.He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770–778.CrossRefGoogle Scholar
87.Heeger, D. (2000). Poisson model of spike generation. Handout, University of Standford 5: 1–13.Google Scholar
88.Hinton, G.E. & Salakhutdinov, R.R. (2006). Reducing the dimensionality of data with neural networks. Science 313(5786): 504–507.CrossRefGoogle ScholarPubMed
89.Hinton, G.E., Osindero, S., & Teh, Y.-W. (2006). A fast learning algorithm for deep belief nets. Neural Computation 18(7): 1527–1554.CrossRefGoogle ScholarPubMed
90.Hornik, K. (1991). Approximation capabilities of multilayer feedforward networks. Neural Networks 4(2): 251–257.CrossRefGoogle Scholar
91.Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Networks 2(5): 359–366.CrossRefGoogle Scholar
92.Hoyer, P.O. (2002). Non-negative sparse coding. Neural Networks for Signal Processing, 2002. Proceedings of the 2002 12th IEEE Workshop on. IEEE, 557–565.CrossRefGoogle Scholar
93https://en.wikipedia.org/wiki/go_(game).Google Scholar
94.Huang, G.-B., Zhu, Q.-Y., & Siew, C.-K. (2006). Extreme learning machine: theory and applications. Neurocomputing 70(1): 489–501.CrossRefGoogle Scholar
95.Hussain, K.F. & Moussa, G.S. (2005). Laser intensity vehicle classification system based on random neural network. Proceedings of the 43rd annual Southeast regional conference-Volume 1. ACM, 31–35.CrossRefGoogle Scholar
96.Javed, A., Larijani, H., Ahmadinia, A., & Emmanuel, R. (2017). Random neural network learning heuristics. Probability in the Engineering and Informational Sciences 31(4): 436–456.CrossRefGoogle Scholar
97.Javed, A., Larijani, H., Ahmadinia, A., & Gibson, D. (2017). Smart random neural network controller for hvac using cloud computing technology. IEEE Transactions on Industrial Informatics 13(1): 351–360.CrossRefGoogle Scholar
98.Jo, S., Yin, J., & Mao, Z.-H. (2005). Random neural networks with state-dependent firing neurons. IEEE Transactions on Neural Networks 16(4): 980–983.CrossRefGoogle ScholarPubMed
99.Kadioglu, Y.M. & Gelenbe, E. (2018). Product form solution for cascade networks with intermittent energy. IEEE Systems Journal. doi: 10.1109/JSYST.2018.2854838.Google Scholar
100.Kasun, L.L.C., Zhou, H., & Huang, G.-B. (2013). Representational learning with extreme learning machine for big data. IEEE Intelligent Systems 28(6): 31–34.Google Scholar
101.Kim, H. & Gelenbe, E. (2012). Stochastic gene expression modeling with hill function for switch-like gene responses. IEEE/ACM Trans. Comput. Biology Bioinform. 9(4): 973–979. [Online]. Available: https://doi.org/10.1109/TCBB.2011.153.Google ScholarPubMed
102.Knight, J., Voelker, A.R., Mundy, A., Eliasmith, C., & Furber, S. (2016). Efficient spinnaker simulation of a heteroassociative memory using the neural engineering framework. Neural Networks (IJCNN), 2016 International Joint Conference on. IEEE, 5210–5217.CrossRefGoogle Scholar
103.Kocak, T., Seeber, J., & Terzioglu, H. (2003). Design and implementation of a random neural network routing engine. IEEE Transactions on Neural Networks 14(5): 1128–1143.CrossRefGoogle ScholarPubMed
104.Krizhevsky, A. & Hinton, G. (2009). Learning multiple layers of features from tiny images.Google Scholar
105.LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11): 2278–2324.CrossRefGoogle Scholar
106.LeCun, Y., Huang, F.J., & Bottou, L. (2004). Learning methods for generic object recognition with invariance to pose and lighting. Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on. Vol. 2. IEEE, II–97–104.CrossRefGoogle Scholar
107.LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature 521(7553): 436–444.CrossRefGoogle ScholarPubMed
108.Lee, D.D. & Seung, H.S. (1999). Learning the parts of objects by non-negative matrix factorization. Nature 401(6755): 788–791.CrossRefGoogle ScholarPubMed
109.Leshno, M., Lin, V.Y., Pinkus, A., & Schocken, S. (1993). Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural Networks 6(6): 861–867.CrossRefGoogle Scholar
110.Lichman, M. (2013). UCI machine learning repository. [Online]. Available: http://archive.ics.uci.edu/ml.Google Scholar
111.Likas, A. & Stafylopatis, A. (2000). Training the random neural network using quasi-newton methods. European Journal of Operational Research 126(2): 331–339. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0377221799004828.CrossRefGoogle Scholar
112.Liu, X., Yan, S., & Jin, H. (2010). Projective nonnegative graph embedding. Image Processing, IEEE Transactions on 19(5): 1126–1137.Google ScholarPubMed
113.Lu, R. & Shen, Y. (2006). Image segmentation based on random neural network model and gabor filters. Engineering in Medicine and Biology Society, 2005. IEEE-EMBS 2005. 27th Annual International Conference of the. IEEE, 6464–6467.Google Scholar
114.Makhzani, A., Shlens, J., Jaitly, N., & Goodfellow, I.J. (2015). Adversarial autoencoders. CoRR, abs/1511.05644, 1–16. [Online]. Available: http://arxiv.org/abs/1511.05644.Google Scholar
115.McCulloch, W.S. & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics 5(4): 115–133. [Online]. Available: https://doi.org/10.1007/BF02478259.CrossRefGoogle Scholar
116.Metropolis, N. & Ulam, S. (1949). The monte carlo method. Journal of the American Statistical Association 44(247): 335–341.CrossRefGoogle ScholarPubMed
117.Öke, G. & Loukas, G. (2007). A denial of service detector based on maximum likelihood detection and the random neural network. The Computer Journal 50(6): 717–727. [Online]. Available: http://dx.doi.org/10.1093/comjnl/bxm066.CrossRefGoogle Scholar
118.Park, J. & Sandberg, I.W. (1991). Universal approximation using radial-basis-function networks. Neural Computation 3(2): 246–257.CrossRefGoogle ScholarPubMed
119.Park, J. & Sandberg, I.W. (1993). Approximation and radial-basis-function networks. Neural Computation 5(2): 305–316.CrossRefGoogle Scholar
120.Paudel, I., Pokhrel, J., Wehbi, B., Cavalli, A., & Jouaber, B. (Sept 2014). Estimation of video qoe from mac parameters in wireless network: A random neural network approach. 2014 14th International Symposium on Communications and Information Technologies (ISCIT), 51–55.Google Scholar
121.Pavlus, J. (2015). The search for a new machine. Scientific American 312(5): 58–63.CrossRefGoogle ScholarPubMed
122.Phan, H.T.T., Sternberg, M.J.E., & Gelenbe, E. (2012). Aligning protein-protein interaction networks using random neural networks. 2012 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2012, Philadelphia, PA, USA, October 4–7, 2012, 1–6. [Online]. Available: https://doi.org/10.1109/BIBM.2012.6392664.CrossRefGoogle Scholar
123.Preissl, R., Wong, T.M., Datta, P., Flickner, M., Singh, R., Esser, S.K., Risk, W.P., Simon, H.D., & Modha, D.S. (2012). Compass: a scalable simulator for an architecture for cognitive computing. Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis. IEEE Computer Society Press, 54.CrossRefGoogle Scholar
124.Qin, Z., Yu, F., Shi, Z., & Wang, Y. (2006). Adaptive inertia weight particle swarm optimization. International conference on Artificial Intelligence and Soft Computing. Springer, 450–459.CrossRefGoogle Scholar
125.Radford, A., Metz, L., & Chintala, S. (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. CoRR, abs/1511.06434, 1–16. [Online]. Available: http://arxiv.org/abs/1511.06434.Google Scholar
126.Radhakrishnan, K. & Larijani, H. (2011). Evaluating perceived voice quality on packet networks using different random neural network architectures. Performance Evaluation 68(4): 347–360, g-Networks and their Applications. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0166531611000101.CrossRefGoogle Scholar
127.Riedmiller, M. & Braun, H. (1993). A direct adaptive method for faster backpropagation learning: the rprop algorithm. IEEE International Conference on Neural Networks. Vol. 1, 586–591.Google Scholar
128.Rosenblatt, F. (1958). The perceptron: a probabilistic model for information storage and organization in the brain. Psychological Review 65(6), 386.CrossRefGoogle Scholar
129.Rosenblatt, F. (1961). Principles of neurodynamics. perceptrons and the theory of brain mechanisms, DTIC Document, Tech. Rep.Google Scholar
130.Rumelhart, D.E., Hinton, G.E., & Williams, R.J. (1985). Learning internal representations by error propagation, California Univ San Diego La Jolla Inst for Cognitive Science, Tech. Rep.Google Scholar
131.Rumelhart, D.E., Hinton, G.E., & Williams, R.J. (1986). Parallel distributed processing: Explorations in the microstructure of cognition. In Rumelhart, D. E., McClelland, J. L., & C. PDP Research Group, (eds.), ch. Learning Internal Representations by Error Propagation. Vol. 1, Cambridge, MA, USA: MIT Press, 318–362. [Online]. Available: http://dl.acm.org/citation.cfm?id=104279.104293.CrossRefGoogle Scholar
132.Sakellari, G. & Gelenbe, E. (2010). Demonstrating cognitive packet network resilience to worm attacks. 17th ACM conference on Computer and Communications Security, Proceedings of the. ACM, 636–638.CrossRefGoogle Scholar
133.Scarselli, F. & Tsoi, A.C. (1998). Universal approximation using feedforward neural networks: A survey of some existing methods, and some new results. Neural Networks 11(1): 15–37.CrossRefGoogle Scholar
134.Serrano, W. & Gelenbe, E. (2018). The random neural network in a neurocomputing application for web search. Neurocomputing 280: 123–134.CrossRefGoogle Scholar
135.Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M. et al. (2016). Mastering the game of go with deep neural networks and tree search. Nature 529(7587): 484–489.CrossRefGoogle ScholarPubMed
136.Springenberg, J.T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014). Striving for simplicity: The all convolutional net, arXiv preprint arXiv:1412.6806.Google Scholar
137.Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 15(1), 1929–1958.Google Scholar
138.Stinchcombe, M. & White, H. (1989). Universal approximation using feedforward networks with non-sigmoid hidden layer activation functions. International 1989 Joint Conference on Neural Networks. Vol. 1, 613–617.Google Scholar
139.Storn, R. & Price, K. (1997). Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization 11(4): 341–359. [Online]. Available: https://doi.org/10.1023/A:1008202821328.CrossRefGoogle Scholar
140.Tang, J., Deng, C., & Huang, G.-B. (2016). Extreme learning machine for multilayer perceptron. IEEE Transactions on Neural Networks and Learning Systems 27(4): 809–821.CrossRefGoogle ScholarPubMed
141.Teke, A. & Atalay, V. (2006). Texture classification and retrieval using the random neural network model. Computational Management Science 3(3): 193–205.CrossRefGoogle Scholar
142.Timotheou, S. (2008). Nonnegative least squares learning for the random neural network. International Conference on Artificial Neural Networks. Springer, 195–204.CrossRefGoogle Scholar
143.Timotheou, S. (2009). A novel weight initialization method for the random neural network. Neurocomputing 73(1–3): 160–168.CrossRefGoogle Scholar
144.Timotheou, S. (2010). The random neural network: a survey. The Computer Journal 53(3): 251–267.CrossRefGoogle Scholar
145.Vlontzos, A. (May 2017). The rnn-elm classifier. 2017 International Joint Conference on Neural Networks (IJCNN), 2702–2707.Google Scholar
146.Wachsmuth, E., Oram, M., & Perrett, D. (1994). Recognition of objects and their component parts: responses of single units in the temporal cortex of the macaque. Cerebral Cortex 4(5): 509–522.CrossRefGoogle ScholarPubMed
147.Wang, L. & Gelenbe, E. (2016). Real-time traffic over the cognitive packet network, 3–21.CrossRefGoogle Scholar
148.Wang, L. & Gelenbe, E. (2018). Adaptive dispatching of tasks in the cloud. IEEE Transactions on Cloud Computing 6(1): 33–45.CrossRefGoogle Scholar
149.Wang, Y.-X. & Zhang, Y.-J. (2013). Nonnegative matrix factorization: a comprehensive review. IEEE Transactions on Knowledge and Data Engineering 25(6): 1336–1353.CrossRefGoogle Scholar
150.Wen, W., Wu, C., Wang, Y., Nixon, K., Wu, Q., Barnell, M., Li, H., & Chen, Y. (2016). A new learning method for inference accuracy, core occupation, and performance co-optimization on truenorth chip. Design Automation Conference (DAC), 2016 53nd ACM/EDAC/IEEE. IEEE, 1–6.CrossRefGoogle Scholar
151Wikipedia contributors, “Perceptron,” 2018, [Online; accessed 25-Sep-2018]. [Online]. Available: https://en.wikipedia.org/wiki/Perceptron.Google Scholar
152.Wilson, D.R. & Martinez, T.R. (1996). Heterogeneous radial basis function networks. Proceedings of the International Conference on Neural networks (ICNN 96), 1263–1267.CrossRefGoogle Scholar
153.Yin, Y. (2018). Random neural networks for deep learning, Imperial College London, PhD Thesis, available in https://san.ee.ic.ac.uk/publications/PhDThesis_Yonghua_Yin_v31.pdf.Google Scholar
154.Yin, Y. & Gelenbe, E. (2016). Deep learning in multi-layer architectures of dense nuclei, arXiv preprint arXiv:1609.07160.Google Scholar
155.Yin, Y. & Gelenbe, E. (Sept. 2016). Nonnegative autoencoder with simplified random neural network, ArXiv e-prints.Google Scholar
156.Yin, Y. & Gelenbe, E. (May 2017). Single-cell based random neural network for deep learning. 2017 International Joint Conference on Neural Networks (IJCNN), 86–93.Google Scholar
157.Yin, Y. & Gelenbe, E. (2018). A classifier based on spiking random neural network function approximator, Preprint available in ResearchGate.net.Google Scholar
158.Yin, Y. & Zhang, Y. (2012). Weights and structure determination of chebyshev-polynomial neural networks for pattern classification. Software 11, 048.Google Scholar
159.Yin, Y., Wang, L., & Gelenbe, E. (May 2017). Multi-layer neural networks for quality of service oriented server-state classification in cloud servers. 2017 International Joint Conference on Neural Networks (IJCNN), 1623–1627.Google Scholar
160.Yunong, Z., Kene, L., & Ning, T. (2009). An rbf neural network classifier with centers, variances and weights directly determined. Computing Technology and Automation 3, 002.Google Scholar
161.Zeiler, M.D. (2012). ADADELTA: an adaptive learning rate method. CoRR abs/1212.5701, 1–6. [Online]. Available: http://arxiv.org/abs/1212.5701.Google Scholar
162.Zhang, Y., Yin, Y., Yu, X., Guo, D., & Xiao, L. (2012). Pruning-included weights and structure determination of 2-input neuronet using chebyshev polynomials of class 1. Intelligent Control and Automation (WCICA), 2012 10th World Congress on. IEEE, 700–705.CrossRefGoogle Scholar
163.Zhang, Y., Yin, Y., Guo, D., Yu, X., & Xiao, L. (2014). Cross-validation based weights and structure determination of chebyshev-polynomial neural networks for pattern classification. Pattern Recognition 47(10): 3414–3428.CrossRefGoogle Scholar
164.Zhang, Y., Yu, X., Guo, D., Yin, Y., & Zhang, Z. (2014). Weights and structure determination of multiple-input feed-forward neural network activated by chebyshev polynomials of class 2 via cross-validation. Neural Computing and Applications 25(7–8): 1761–1770.CrossRefGoogle Scholar
165.Zhu, X. (2005). Semi-supervised learning literature survey.Google Scholar