(18) 第三节课：从零起步在框架中编码实现Back Propagation功能

4.2 盘古Back Propagation功能

        在前面两节我们已完成深度学习的骨架、神经元网络结构、Forward Propagation的实现，本章节实现Back Propagation。Forward propagation 和Back Propagation之间的一来一回，形成了深度学习的神经网络之所以能够学习的最重要的部分及最重要的过程，无论是TensorFlow还是PyTorch，或者其他的一些深度学习框架，其思路都是首先从左到右运行一下，输入端（Input Layer）到输出端（Output Layer）运行一次，这个过程就叫Forward propagation，然而运行的时候，往往会有一定的误差。产生误差怎么办？这个时候需要我们回顾过去，Deep Learning的神经网络跟人的行为模式是一样的，它可以回顾过去，回顾过去的时候要发现哪些因素导致这些误差。

       本节内容：

1)    DeepLearning是具体是如何学习的？

2)    实现Back Propagation算法。

3)    测试并分析计算结果。

本章节我们将实现自研盘古人工智能框架的Back Propagation算法，将新增BackPropagation.py代码，要实现的目录结构代码如图所示：

                           

图 1- 29 Create_AI_Framework_In5Classes(Day3)

 

我们先看一下上一节Create_AI_Framework_In5Classes(Day2)版本中Neuron_Network_Entry.py的运行结果：

 +1      V1      V2      Hidden layer creation: 1        N[1][1]         N[1][2]         N[1][3]         N[1][4]         N[1][5]         N[1][6]         N[1][7]         N[1][8]         Hidden layer creation: 2        N[2][1]         N[2][2]         N[2][3]         N[2][4]         Hidden layer creation: 3        N[3][1]         N[3][2]         
Output layer:  OutputThe weight from 1 at layers[0] to 4 at layers[1] : -0.6912940261140725The weight from 1 at layers[0] to 5 at layers[1] : -0.545954211041835The weight from 1 at layers[0] to 6 at layers[1] : -0.07196256901813092The weight from 1 at layers[0] to 7 at layers[1] : -0.1716847532542054The weight from 1 at layers[0] to 8 at layers[1] : 0.5436794344410698The weight from 1 at layers[0] to 9 at layers[1] : 0.0927510938379803The weight from 1 at layers[0] to 10 at layers[1] : -0.2025580886147379The weight from 1 at layers[0] to 11 at layers[1] : -0.3900140803181851The weight from 2 at layers[0] to 4 at layers[1] : 0.8513281607647141The weight from 2 at layers[0] to 5 at layers[1] : -0.6656985531444581The weight from 2 at layers[0] to 6 at layers[1] : 0.8308516529438015The weight from 2 at layers[0] to 7 at layers[1] : -0.06993813902896084The weight from 2 at layers[0] to 8 at layers[1] : -0.38947318555312527The weight from 2 at layers[0] to 9 at layers[1] : -0.7918003384016561The weight from 2 at layers[0] to 10 at layers[1] : 0.8884015561041609The weight from 2 at layers[0] to 11 at layers[1] : 0.19292212264493425The weight from 4 at layers[1] to 13 at layers[2] : -0.11960722294594706The weight from 4 at layers[1] to 14 at layers[2] : -0.10380085578174647The weight from 4 at layers[1] to 15 at layers[2] : -0.5529776266236945The weight from 4 at layers[1] to 16 at layers[2] : 0.9233144278756291The weight from 5 at layers[1] to 13 at layers[2] : -0.32656221428721066The weight from 5 at layers[1] to 14 at layers[2] : -0.9875314106348981The weight from 5 at layers[1] to 15 at layers[2] : 0.7324986925523662The weight from 5 at layers[1] to 16 at layers[2] : -0.7415684507491214The weight from 6 at layers[1] to 13 at layers[2] : 0.05058287752320001The weight from 6 at layers[1] to 14 at layers[2] : 0.835976536842689The weight from 6 at layers[1] to 15 at layers[2] : -0.5429146921382267The weight from 6 at layers[1] to 16 at layers[2] : 1.0028073498164036The weight from 7 at layers[1] to 13 at layers[2] : -0.7718682631713544The weight from 7 at layers[1] to 14 at layers[2] : -0.9068957126977204The weight from 7 at layers[1] to 15 at layers[2] : -0.3841691125756408The weight from 7 at layers[1] to 16 at layers[2] : 0.7715497546889127The weight from 8 at layers[1] to 13 at layers[2] : -0.14727982579911536The weight from 8 at layers[1] to 14 at layers[2] : -0.38628365513998897The weight from 8 at layers[1] to 15 at layers[2] : -0.227950879833562The weight from 8 at layers[1] to 16 at layers[2] : -0.652077921521375The weight from 9 at layers[1] to 13 at layers[2] : 0.7781741568206197The weight from 9 at layers[1] to 14 at layers[2] : 0.3510754998234846The weight from 9 at layers[1] to 15 at layers[2] : 0.2309293743704257The weight from 9 at layers[1] to 16 at layers[2] : -0.18835571066223The weight from 10 at layers[1] to 13 at layers[2] : -0.2213382006227218The weight from 10 at layers[1] to 14 at layers[2] : -0.9508277637903745The weight from 10 at layers[1] to 15 at layers[2] : -0.02559879696579548The weight from 10 at layers[1] to 16 at layers[2] : -0.30284783410815697The weight from 11 at layers[1] to 13 at layers[2] : 0.5930853471509221The weight from 11 at layers[1] to 14 at layers[2] : -0.5793678025960891The weight from 11 at layers[1] to 15 at layers[2] : -0.12170995553670871The weight from 11 at layers[1] to 16 at layers[2] : 0.9262342193773581The weight from 13 at layers[2] to 18 at layers[3] : 0.9389925802670069The weight from 13 at layers[2] to 19 at layers[3] : -0.6634631642356033The weight from 14 at layers[2] to 18 at layers[3] : 0.022740002232284784The weight from 14 at layers[2] to 19 at layers[3] : 0.8546860432679304The weight from 15 at layers[2] to 18 at layers[3] : -0.5185523547696405The weight from 15 at layers[2] to 19 at layers[3] : -0.33566596595949894The weight from 16 at layers[2] to 18 at layers[3] : 0.7470756546708803The weight from 16 at layers[2] to 19 at layers[3] : 0.7830098033920143The weight from 18 at layers[3] to 20 at layers[4] : 0.042524157398006635The weight from 19 at layers[3] to 20 at layers[4] : -0.516532532247196Prediction: 0.43422777315602007Prediction: 0.43033577269868234Prediction: 0.43479114233210986Prediction: 0.4306899358038518

 这里预测的结果有4条记录，或者是4个输入的对象，每个输入对象对两个Features（即instances的第1列，第2列）构成的关系进行了预测：

instances =    [[0,0,0],             [0,1,1],             [1,0,1],             [1,1,0]] 

       instances第1条记录[0,0,0]真实的结果是0，第2条记录[0,1,1]真实的结果是1，第3条记录[1,0,1]真实的结果是1，第4条记录[1,1,0]真实的结果是0；但这里预测的结果分别是Prediction: 0.43422777315602007，Prediction:0.43033577269868234，Prediction: 0.43479114233210986，Prediction: 0.4306899358038518，这显然是一个很大的误差，是我们上一节中实现的结果。 这里Neuron_Network_Entry.py代码运行时的hidden_layers =[8,4,2]，网络层一共有5层，输入层1层，隐藏层有3层（其[8,4,2]的隐藏层分别包括8个神经元、4个神经元、2个神经元），输出层1层，一共是5层。