Introduction to Neural Networks

Neural networks is the best alternative if a logistic/linear regression model is not enough.

In neural networks there are neurons, which are basically just computational units that takes inputs and gives an output (in terms of h(x)). If you remember our logistic regression hypothesis, we use the same hypothesis as we want all of our outputs and inputs to be <= 1.

A hidden layer is where inputs go to into this layer, computations happen, and then it either goes to another hidden layer or a hypothesis is made. This layer is not really that important because we only care a bout the inputs and the hypothesis. If you need help visualizing a hidden layer, the second layer below represents a hidden layer because it isn’t the hypothesis layer or the input layer.

Just to get some terminology out of the way let us rewrite the above graph in terms of “activations”. Hopefully the graphic below can clear it up.

So basically this model does not include x0 because this is often called the “bias node”because it always is always equal to 1 (we will learn more about this later). Each connection to a unit in the hidden layer from a unit in the input layer has a certain weight on it. To organize all this we create a 3 x 4 dimensional matrix called theta. In this matrix each row represents what node in the target layer it is, and each column represents the arrows from a specific node in the base layer. Remember that it is a 3 x 4 because there will be 4 base nodes with the addition of a base node. So now to compute the sum of weights on a particular unit in the hidden layer it would simply be:

Where

And the subscripts represent what element in the matrix you are getting. You multiply the unit by the corresponding weight and add it hence computing the sum of weights on a particular unit in the hidden layer. Then you apply logistic regression denoted by g(x) to squeeze the value between 0 to 1. After this the hypothesis (third layer, using the activations of the first layer) is simply:

Refer to the graphs and figures above if this doesn’t make sense, hopefully it will soon with the compliments of a few articles which will come up in a few months. I am going to take a break from blogging for a little bit. Corona is going crazy!