The week's pick
Random Articles
Reseach Article
Training Recurrent Neural Networks by a Hybrid PSO-Cuckoo Search Algorithm for Problems Optimization
| International Journal of Computer Applications |
| Foundation of Computer Science (FCS), NY, USA |
| Volume 159 - Number 3 |
| Year of Publication: 2017 |
| Authors: Ruba Talal Ibrahim, Zahraa Tariq Mohammed |
10.5120/ijca2017912891
|
Ruba Talal Ibrahim, Zahraa Tariq Mohammed . Training Recurrent Neural Networks by a Hybrid PSO-Cuckoo Search Algorithm for Problems Optimization. International Journal of Computer Applications. 159, 3 ( Feb 2017), 32-38. DOI=10.5120/ijca2017912891
Abstract
Because of computational drawbacks of conventional numerical methods in solving complex optimization problems, researchers may have to rely on meta-heuristic algorithms. Particle swarm optimization (PSO) is one of the most widely used algorithms due to its simplicity of implementation and fast convergence speed. Also, the cuckoo search algorithm is a recently developed meta-heuristic optimization algorithm, which is suitable for solving optimization problems .Normally, the parameters of the cuckoo search are kept constant. This may make algorithm suffering from slow convergence rate. To overcome with this issue, a hybrid algorithm called (PSO-CS classifier)for adjusting the cuckoo search parameters is presented to improved cuckoo search algorithm by particle swarm optimization (PSO) for training recurrent neural network which its weights and bias trained using the (PSO-CS classifier) to deviate from being stuck in local minima) for two benchmark classification problems. Moreover, to combine the ability of social communication in PSO with the local search capability of CS .Finally, the performance of the proposed algorithm is compared with that of the standard cuckoo search and PSO Algorithms. The simulation results show that the proposed (PSO-CS classifier) algorithm performs better than other algorithms in decrease number of training errors with a fast convergence rate and high accuracy.
References
- Paliwal, M.; Kumar, U.A. Neural networks and statistical techniques: A review of applications. Expert Syst. Appl. 2009, 36, 2–17.
- Nazri Mohd Nawi, Abdullah khan, M.Z. Rehman, Maslina Abdul Aziz, Tutut Herawan, and Jemal H. Abawajy,” Neural Network Training by Hybrid Accelerated Cuckoo Particle Swarm Optimization Algorithm”, Springer International Publishing Switzerland 2014 .
- Jeng-Fung Chen , Quang Hung Do , and Ho-Nien Hsieh,” Training Artificial Neural Networks by a Hybrid PSO-CS Algorithm”, Algorithms 2015, 8, 292-308 .
- C. L. Giles, S. Lawrence, and A. C. Tsoi, “ Rule inference for financial prediction using recurrent neural networks ,”in Proceedings of the IEEE/IAFE Conference on Computational Intelligence for Financial Engineering ,pp .253–259 ,IEEE, March 1997.
- S.Li,D.C . Wunsch, E. O’Hair ,and M. G. Giessel mann,“Wind turbine power estimation by neural networks with Kalman filter training on a SIMD parallel machine,” in Proceedings of the International Joint Conference on Neural Networks(IJCNN'99), pp.3430–3434,Washington,DC,USA,July1999.
- N. M. Nawi, A. Khan, and M. Z. Rehman, “CSBPRNN: a new hybridization technique using cuckoo search to train back propagation recurrent neural network,” in Proceedings of the First International Conference on Advanced Data and information engineering(2013)vol.285 of lecture notes in Electrical Engineering,pp.111–118,2014.
- L.Fauseeti, Fundamental of Neural Network Architecture, Algorithm and Application, Prentice Hall, Englewood Cliffs, NJ, USA, 1994.
- Nazri Mohd Nawi, Abdullah Khan,M. Z. Rehman, Haruna Chiroma,and Tutut Herawan,” Weight Optimization in Recurrent Neural Networks with Hybrid Metaheuristic Cuckoo Search Techniques for Data Classification”, Hindawi Publishing Corporation Mathematical Problems in Engineering Volume 2015, Article ID 868375, 12 pages.
- Rumelhart, D. E., Hinton, G. E., and Williams, R. J.: Learning internal representations by error propagation, in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Rumelhart, D. E. McClelland, J. L., Eds., MIT Press, Cambridge, 45. (1986) .
- Goldberg, E. Genetic Algorithms in Search, Optimization and Machine Learning; Addison Wesley: Boston, MA, USA, 1989.
- Kennedy, J.; Eberhart, R.C. Particle swarm optimization. In Proceedings of the 1995 IEEE International Conference on Neural Networks, Perth, Australia, 27 November–1 December 1995; Volume 4, pp. 1942–1948.
- Dorigo, M.; Maniezzo, V.; Golomi, A. Ant system: Optimization by a colony of cooperating agents. IEEE Trans. Syst. Man Cybernet. 1996, 26, 29–41.
- Travis Desell, Sophine Clachar, James Higgins, Brandon Wild,” Evolving Deep Recurrent Neural Networks Using Ant Colony Optimization”, The 15th European Conference on Evolutionary Computation in Combinatorial Optimization .
- Beatriz A. Garroand Roberto A.Vázquez,” Designing Artificial Neural Networks Using Particle Swarm Optimization Algorithms”, Computational Intelligence and Neuroscience Volume 2015, Article ID 369298, 20 page .
- Ala'a Abu-Srhan and Essam Al Daoud,” A Hybrid Algorithm Using a Genetic Algorithm and Cuckoo Search Algorithm to Solve the Traveling Salesman Problem and its Application to Multiple Sequence Alignment”, International Journal of Advanced Science and Technology Vol.61, (2013), pp.29-38 .
- Ehsan Valian, Shahram Mohanna and Saeed Tavakoli,” IMPROVED CUCKOO SEARCH ALGORITHM FOR FEEDFORWARD NEURAL NETWORK TRAINING”, International Journal of Artificial Intelligence & Applications (IJAIA), Vol.2, No.3, July 2011 .
- Nazri Mohd. Nawi, Abdullah, M. Z. Rehman,” A NEW OPTIMIZED CUCKOO SEARCH RECURRENT NEURAL NETWORK (CSRNN) ALGORITHM”, Springer-Verlag Berlin Heidelberg 2011 .
- Peng Xiao, Ganesh K. Venayagamoorthy, and Keith A. Corzine,” Combined Training of Recurrent Neural Networks with Particle Swarm Optimization and Backpropagation Algorithms for Impedance Identification”, Proceedings of the 2007 IEEE Swarm Intelligence Symposium (SIS 2007) .
- Yang XS, Deb S, (2009) Cuckoo search via Lévy flights, Proceeings of World Congress on Nature & Biologically Inspired Computing, India, pp 210-214.
- Yang XS, Deb S, (2010) Engineering Optimisation by Cuckoo Search, Int. J. Mathematical Modelling and Numerical Optimisation. Vol. 1, No. 4, pp 330–343.
- H. Jaeger (2002): Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the "echo state network" approach. GMD Report 159, German National Research Center for Information Technology, 2002 (48 pp.) .
- Aseel Ismael Ali, Ruba Talal,” UCTP based on Hybrid PSO with Tabu Search Algorithm using Mosul University Dataset”, International Journal of Computer Applications (0975 – 8887) Volume 91 – No.9, April 2014 .
- S. Kitayama, K. Yamazaki, M. Arakawa " adaptive range particle swarm optimization". Springer Science + Business Media, Journal: Optimization and Engineering ISSN: 13894420 Year: 2009 Volume: 10 Issue: 4 pages:575-597 DOI:10.1007/s11081-009-9081-7.
- D. Bratton, J. Kennedy "defining a standard for particle swarm optimization", IEEE, Journal: 2007 IEEE Swarm Intelligence Symposium ISBN: 1424407087 Year: 2007 Pages: 120-127 Provider: IEEE Publisher: IEEE DOI: 10.1109/SIS.2007.368035 .
- D. P. Rini, S. M. Shamsuddin, S. S. yuhaniz "Particle Swarm Optimization: Technique, System and Challenges" International Journal of Computer Applications (0975 – 8887) Volume 14– No.1, January 2011.
- Engelbrecht A. P., (2007):”Computational Intelligence An Introduction”, Second Edition, John Wiley & Sons Ltd, West Sussex, England .
- E.Valian, S. Mohanna and S. Tavakoli, “Improved Cuckoo Search Algorithm for Global Optimization”, International Journal of Communications and Information Technology, IJCIT, vol. 1, no. 1, (2011), pp. 1-62 .
- R. G. Babukartik and P. Dhavachelvan, “Hybrid Algorithm using the advantage of ACO and Cuckoo Search for Job Scheduling”, International Journal of Information Technology Convergence and Services (IJITCS), vol. 2, no. 4, (2012).
Index Terms
Keywords