Rest of questions

A training pattern, consisting of an input vector x = [x1, x2, x3]T and desired outputs t = [t1, t2, t3]T, is presented to the following neural network. What is the usual sequence of events for training the network using the backpropagation algorithm?

After some training, the units in the neural network of question 22 have the following weight vectors: v1 = ⎡ ⎣ −0.7 1.8 2.3 ⎤ ⎦, v2 = ⎡ ⎣ −1.2 −0, 6 2.1 ⎤ ⎦, w1 = 1.0 −3.5 , w2 = 0.5 −1.2 and w3 = 0.3 0.6 . Assume that all units have sigmoid activation functions given by f(x) = 1 1 + exp(−x) and that each unit has a bias θ = 0 (zero). If the network is tested with an input vector x = [2.0, 3.0, 1.0]T then the output of the first hidden neuron y1 will be (Hint: on some calculators, exp(x) = ex where e = 2.7182818)

For the same neural network described in questions 22 and 23, the output of the second hidden neuron y2 will be (Assume exactly the same weights, activation functions, bias values and input vector as described in the previous question.)

For the same neural network described in questions 22 and 23, the output of the first output neuron z1 will be (Assume exactly the same weights, activation functions, bias values and input vector as in question 23.)

For the same neural network described in questions 22 and 23, the output of the third output neuron z3 will be (Assume exactly the same weights, activation functions, bias values and input vector as in question 23.)

The following figure shows part of the neural network described in questions 22 and 23. In this question, a new input pattern is presented to the network and training continues as follows. The actual outputs of the network are given by z = [0.35, 0.88, 0.57]T and the corresponding target outputs are given by t = [1.00, 0.00, 1.00]T. The weights w12, w22 and w32 are also shown below.For the output units, the Generalized Delta Rule can be written as wkj = ηδkyj where δk = f (Ik)(tk − zk) where wkj is the change to the weight from unit j to unit k, η is the learning rate, δk is the error for unit k, and f(net) is the derivative of the activation function f(net). For the sigmoid activation function given in question 23, the derivative can be rewritten as f (Ik) = f(Ik)[1 − f(Ik)]. What is the error for each of the output units?

For the hidden units of the same network, the Generalized Delta Rule can be written as vji = ηδjxi where δj = f (Hj)  k δkwkj where vji is the change to the weight from unit I to unit j, η is the learning rate, δj is the error for unit j, and f(net) is the derivative of the activation function f(net). For the sigmoid activation function given in question 23, the derivative can be rewritten as f (Hj) = f(Hj)[1 − f(Hj )]. What is the error for hidden unit 2 given that its activation for the pattern being processed is currently y2 = 0.74?

Which of the following techniques is NOT a strategy for dealing with local minima in the backpropagation algorithm?

Training with the “1-of-M” coding is best explained as follows:

Consider the following feedforward network with one hidden layer of units: The input vector to the network is x = [x1, x2, x3]T , the vector of hidden layer outputs is y = [y1, y2]T , the vector of actual outputs is z = [z1, z2, z3]T , and the vector of desired outputs is t = [t1, t2, t3]T. The network has the following weight vectors: v1 = ⎡ ⎣ 0.4 −0.6 1.9 ⎤ ⎦, v2 = ⎡ ⎣ −1.2 0.5 −0.7 ⎤ ⎦, w1 = 1.0 −3.5 , w2 = 0.5 −1.2 and w3 = 0.3 0.6 . Assume that all units have sigmoid activation functions given by f(x) = 1 1 + exp(−x) and that each unit has a bias θ = 0 (zero). If the network

Assuming exactly the same neural network and the same input vector as in the previous question, what is the activation I2 of the second output neuron?

For the hidden units of the network in question 31, the generalized Delta rule can be written as vji = ηδjxi where vji is the change to the weights from unit I to unit j, η is the learning rate, δj is the error term for unit j, and xi is the ith input to unit j. In the backpropagation algorithm, what is the error term δj?

For the output units of the network in question 31, the generalized Delta rule can be written as wkj = ηδkyj where wkj is the change to the weights from unit j to unit k, η is the learning rate, δk is the error term for unit k, and yj is the jth input to unit k. In the backpropagation algorithm, what is the error term δk?

Which of the following equations best describes the generalized Delta rule with momentum?

The following figure shows part of the neural network described in question 31. A new input pattern is presented to the network and training proceeds as follows. The actual outputs of the network are given by z = [0.32, 0.05, 0.67]T and the corresponding target outputs are given by t = [1.00, 1.00, 1.00]T. The weights w12, w22 and w32 are also shown below. For the output units, the derivative of the sigmoid function can be rewritten as f (Ik) = f(Ik)[1 − f(Ik)]. What is the error for each of the output units?

For the hidden units, the derivative of the sigmoid function can be rewritten as f (Hj) = f(Hj)[1 − f(Hj )]. What is the error for hidden unit 2 given that its activation for the pattern being processed is currently y2 = 0.50?

What is the biggest difference between Widrow & Hoff’s Delta Rule and the Perceptron Learning Rule for learning in a single-layer feed forward network?