VLSI -- Compatible Implementations for Artificial Neural Networks

This book introduces several state-of-the-art VLSI implementations of artificial neural networks (ANNs). It reviews various hardware approaches to ANN implementations: analog, digital and pulse-coded. The analog approach is emphasized as the main one taken in the later chapters of the book. The area of VLSI implementation of ANNs has been progressing for the last 15 years, but not at the fast pace originally predicted. Several reasons have contributed to the slow progress, with the main one being that VLSI implementation of ANNs is an interdisciplinaly area where only a few researchers, academics and graduate students are willing to venture. The work of Professors Fakhraie and Smith, presented in this book, is a welcome addition to the state-of-the-art and will greatly benefit researchers and students working in this area. Of particular value is the use of experimental results to backup extensive simulations and in-depth modeling. The introduction of a synapse-MOS device is novel. The book applies the concept to a number of applications and guides the reader through more possible applications for future work. I am confident that the book will benefit a potentially wide readership. M. I. Elmasry University of Waterloo Waterloo, Ontario Canada Preface Neural Networks (NNs), generally defined as parallel networks that employ a large number of simple processing elements to perform computation in a distributed fashion, have attracted a lot of attention in the past fifty years. As the result. many new discoveries have been made.

Introduces an approach to biologically-inspired and VLSI-compatible definition, simulation, and implementation of artificial neural networks. This book develops a set of guidelines for general hardware implementation of artificial neural networks. It provides a geometrical interpretation of the behavior of different variants of these networks.

1 Introduction and Motivation.- 1.1 Introduction.- 1.2 Motivation.- 1.3 Objectives of this Work.- 1.4 Organization of the Book.- 2 Review of Hardware-Implementation Techniques.- 2.1 Introduction.- 2.2 Taxonomies of Neural Hardware.- 2.3 Pulse-Coded Implementations.- 2.4 Digital Implementations.- 2.5 Analog Implementations.- 2.6 Comparison of Some Existing Systems.- 2.7 Summary.- 3 Generalized Artificial Neural Networks (GANNs).- 3.1 Introduction.- 3.2 Generalized Artificial Neural Networks (GANNs).- 3.3 Nonlinear MOS-Compatible Semi-Quadratic Synapses.- 3.4 Networks Composed of Semi-Quadratic Synapses.- 3.5 Training Equations.- 3.6 Simulation and Verification of the Approach.- 3.7 Summary.- 4 Foundations: Architecture Design.- 4.1 Introduction.- 4.2 Feedforward Networks with Linear Synapses.- 4.3 Feedforward Networks with Quadratic Synapses.- 4.4 Single-Transistor-Synapse Feedforward Networks.- 4.5 Intelligent MOS Transistors: SyMOS.- 4.6 Performance of Simple SyMOS Networks.- 4.7 Architecture Design in Neural-Network Hardware.- 4.8 The Resource-Finding Exploration.- 4.9 The Current-Source-Inhibited Architecture (CSIA).- 4.10 The Switchable-Sign-Synapse Architecture (SSSA).- 4.11 Digital-Analog Switchable-Sign-Synapse Architecture (DASA).- 4.12 Simulation Results and Comparison.- 4.13 Our Choice of the Way to Go.- 4.14 Summary.- 5 Design, Modeling, and Implementation of a Synapse-MOS Device.- 5.1 Introduction.- 5.2 Design of a SyMOS Device in a CMOS Technology.- 5.3 Implementation.- 5.4 Reliability Issues.- 5.5 Summary.- 6 Synapse-MOS Artificial Neural Networks (SANNs).- 6.1 Introduction.- 6.2 Overview of the Work Leading to Hardware Implementation.- 6.3 Guidelines for Neural-Network Hardware Design.- 6.4 Design and VLSI Implementation.- 6.5 Structure of an SSSA Chip.-6.6 Training Algorithm.- 6.7 Experimental Results.- 6.8 Summary.- 7 Analog Quadratic Neural Networks (AQNNs).- 7.1 Introduction.- 7.2 Background.- 7.3 Design of an "Analog Quadratic Neural Network (AQNN)".- 7.4 Training.- 7.5 VLSI Implementation.- 7.6 Test Results.- 7.7 Applications.- 7.8 Summary and Future Work.- 8 Conclusion and Recommendations for Future Work.- 8.1 Summary of the Work.- 8.2 Contributions of this work.- 8.3 Recommendations for Future Work.- Appendix A Review of Nonvolatile Semiconductor Memory Devices.- A.1 Background.- A.2 Device Review.- A.2.1 Charge-Trapping Devices.- A.2.2 Floating-Gate Devices.- A.3 Conclusion.- Appendix B Scaling Effects.- B.1 The Effect of the Scaling of CMOS Technology on SANNs.- Appendix C Performance Evaluation.- C.1 Speed.- C.2 Power Consumption.- C.2.1 Detailed Calculation of Power Dissipation.- C.3 Area.- C.4 Overall Performance.- References.