## 2017-03-18

### DeepLearning:Linear Class解释

[code lang="python"]
class Linear(Node):
"""
Represents a node that performs a linear transform.
"""
def __init__(self, X, W, b):
# The base class (Node) constructor. Weights and bias
# are treated like inbound nodes.
Node.__init__(self, [X, W, b])

def forward(self):
"""
Performs the math behind a linear transform.
"""
X = self.inbound_nodes[0].value
W = self.inbound_nodes[1].value
b = self.inbound_nodes[2].value
self.value = np.dot(X, W) + b

def backward(self):
"""
Calculates the gradient based on the output values.
"""
# Initialize a partial for each of the inbound_nodes.
self.gradients = {n: np.zeros_like(n.value) for n in self.inbound_nodes}
# Cycle through the outputs. The gradient will change depending
# on each output, so the gradients are summed over all outputs.
for n in self.outbound_nodes:
# Get the partial of the cost with respect to this node.
# Set the partial of the loss with respect to this node's inputs.
# Set the partial of the loss with respect to this node's weights.
# Set the partial of the loss with respect to this node's bias.
[/code]

1. the loss with respect to inputs

[code]self.inbound_nodes[0]，self.inbound_nodes[1]，self.inbound_nodes[2][/code]

So, each node will pass on the cost gradient to its inbound nodes and each node will get the cost gradient from it's outbound nodes. Then, for each node we'll need to calculate a gradient that's the cost gradient times the gradient of that node with respect to its inputs.

If a node has multiple outgoing nodes, you just sum up the gradients from each node.

To find the gradient, you just multiply the gradients for all nodes in front of it going backwards from the cost. This is the idea behind backpropagation. The gradients are passed backwards through the network and used with gradient descent to update the weights and biases.

#### 4 条评论:

1. /(ㄒoㄒ)/~~一点都看不懂

2. 其实也没什么，都是些入门的知识。

3. 其实没什么含金量，只是为了充文章数。