[置頂] Learning RNN from scratch (ＲＮＮ神經網絡參數推導)
來源：程序員人生 發布時間：2016-07-05 14:08:30 閱讀次數：5373次
從上1篇原創的文章到現在，已有1年多了，目前終究有1些新的總結分享個大家。
本文主要講了RNN神經網絡的基本原理，并給出了RNN神經網絡參數更新的詳細推導進程（back propagation），對想知道RNN的參數是如果推導的，可以仔細瀏覽本文。
由于時間有限，下面的總結難免有疏漏的地方，請大家指正。
本文結合了1個非常經典的RNN的例子代碼，進行了詳細的說明，RNN的代碼和注釋請見：https://github.com/weixsong/min-char-rnn
并且，本文給出了驗證為何在output layer采取sigmoid激活函數的時候應當采取cross entropy error作為cost function。
本文目錄：
1.Introduction
2.Simple RNN Theory
3. Using RNN to predict next character
4. Loss Function
4.1 Sum of Squared error (Quadratic error)
4.2 Cross Entropy Error
5. Forward Propagation
6. Quadratic Error VS Cross Entropy Error
6.1 Derivative of error with regard to the output of output layer
6.2 Derivative of error with regard to the input of output layer
7. Error Measure of RNN for Character Prediction
8. Back Propagation
8.1 compute the derivative of error with regard to the output of outputlayer
8.2 compute the derivative of error with regard to the input of outputlayer
8.3 compute the derivative of error with regard to the weight betweenhidden layer and output layer
8.4 compute the derivative of error with regard to the output of hiddenlayer
8.5 Compute the derivative of error with regard to the input of hiddenlayer
8.6 compute the derivative of error with regard to the weight between inputlayer and hidden layer
8.7 compute the derivative of error with regard to the weight betweenhidden layer and hidden layer
生活不易，碼農辛苦
如果您覺得本網站對您的學習有所幫助,可以手機掃描二維碼進行捐贈
