Search for abstracts of CN`99-18
Home > Description
Topic: Intelligent systems with shallow and deep architectures
Abstract: Shallow networks are trained with various gradient based methods or by using newer Support Vector Machines (SVM), or Extreme Learning Machines (ELM) technologies. These networks are using a single hidden layer with limited capabilities, so even for medium size problems require excessive numbers of processing units commonly called neurons. At the same time too many neurons may lead to poor generalization abilities. On the other hand a deep network architecture exhibits tremendous capabilities, but it is almost impossible to train them with a supervised learning algorithm because of the “vanishing gradient” problem.
We have already shown that our NBN algorithm with Fully Connected Cascade (FCC) architecture applied to MRI data significantly outperforms results obtained with SVMs. We have proved that capabilities of networks with connections across layers may increase over 100 times. As we continue on this path, we may face more challenges with training deep architectures. We have already succeeded in training deep architectures by using Bridged MLP. Of course a combination of all listed above approaches is also possible. In order to better understand the deep nonlinear network operation, we have to carry on numerous experiments with training these networks. Analyzing the results should lead us to suggestions on how to modify learning algorithms.
Notice that with a new training approach humans may not be required to understand all details of complex systems, and they still can get an acceptable solution. The key point is for the successful solution of many complex problems with this new technology is to develop new powerful learning algorithms. One may notice that humans already are capable of building in hardware very complex nonlinear systems. Notice that the core of an artificial neuron with sigmoid activation function can be built out of a single differential pair of two transistors and our civilization is already producing solid state systems with over 1012 transistors; in the near future we should be also able to build very large artificial neural networks. Unfortunately currently known algorithms for training are very far behind the hardware development. People may start to believe that it is possible to develop systems which surpass human intelligence in solving certain projects. This can make the current “Leonardo da Vinci” approach, which requires detailed understanding of various problems, not that essential. With the “training approach” we may not able to understand detailed neither design of such complex systems, but we should be able to train them to our needs.
Key Speaker: Bogdan M. WILAMOWSKI (IEEE Fellow)
Bogdan M. Wilamowski received his MS in computer engineering in 1966, PhD in neural computing in 1970, and Dr. Habil. in integrated circuit design in 1977. He received the title of full professor from the President of Poland in 1987. He was the Director of the Institute of Electronics (1979-1981) and the Chair of the Solid State Electronics Department (1987-1989) at the Technical University of Gdansk. He was Professor at the University of Wyoming (1989-2000). From 2000 to 2003 he was the Associate Director of Microelectronics Research and Telecommunication Institute at University of Idaho. Currently he is the Director of ANMSTC (Alabama Nano/Micro Science and Technology Center) and Alumna Professor of Electrical and Computer Engineering Department at Auburn University. Dr. Wilamowski was with the Communication Institute at Tohoku University, Japan (1968-1970), and he spent one year at the Semiconductor Research Institute, Sendai, Japan as a JSPS Fellow (1975-1976). He was a visiting scholar at Auburn University (1981-1982 and 1995-1996), and a visiting professor at the University of Arizona, Tucson (1982-1984). He is the author of 9 books, more than 300 refereed publications, and has over 30 patents. He was the major professor for about 160 graduate students. His main areas of interest are: semiconductor devices and sensors, mixed signal and analog signal processing, network programming, and computational intelligence.
Dr. Wilamowski was one of four founders of the IEEE Computational Intelligence Society then the President of the IEEE Industrial Electronics Society (2004-2005). From 2012 to 2014 he was member of the IEEE Board of Directors.
He has served as an associate editor in several journals and more recently he was the Editor-in-Chief of IEEE Trans. on Industrial Electronics (2007-2010) and the Editor-in-Chief of IEEE Trans. on Industrial Informatics (2011-2013). Both are one of the highest ranked IEEE journals (#3 with IF=6.5 and #1 with IF=8.8).
Prof. Wilamowski is an IEEE Fellow; Fellow of the Kosciuszko Foundation, and an Honorary Member of the Hungarian Academy of Science. In 2008 the President of Poland awarded him with the Commander Cross of the Order of Merit of the Republic of Poland for outstanding service in proliferation of international scientific collaborations and for achievements in areas of microelectronics and computer science.http://www.eng.auburn.edu/~wilambm/