원문정보
초록
영어
Neural networks have been reborn as a Deep Learning thanks to big data, improved processor, and some modification of training methods. Neural networks used to initialize weights in a stupid way, and to choose wrong type activation functions of non-linearity. Weight initialization contributes as a significant factor on the final quality of a network as well as its convergence rate. This paper discusses different approaches to weight initialization. MNIST dataset is used for experiments for comparing their results to find out the best technique that can be employed to achieve higher accuracy in relatively lower duration.
목차
Abstract
1. INTRODUCTION
2. MULTI-LAYER NEURAL NETWORKS TRAINING
2. 1. Structure of Multi-layer Neural Networks
2. 2. Neural Networks Training
3. WEIGHTS INITIALIZATION OF NEURAL NETWORKS
3. 1. Random Initialization
3. 2. Xavier Initialization
3. 3. He-at-al Initialization
3. 4. Batch Normal Initialization
4. EMPIRICAL RESULTS AND OBSERVATION
5. CONCLUSION
ACKNOWLEDGEMENT
1. INTRODUCTION
2. MULTI-LAYER NEURAL NETWORKS TRAINING
2. 1. Structure of Multi-layer Neural Networks
2. 2. Neural Networks Training
3. WEIGHTS INITIALIZATION OF NEURAL NETWORKS
3. 1. Random Initialization
3. 2. Xavier Initialization
3. 3. He-at-al Initialization
3. 4. Batch Normal Initialization
4. EMPIRICAL RESULTS AND OBSERVATION
5. CONCLUSION
ACKNOWLEDGEMENT