skewed class classificationn with neural network with not much data; procedure to follow
This is my first implementation (trial) for a Neural Network.
I use Octave with neural network to classify (3 classes).
of expected network output fall in class 3 (42 'non-class 3'
are 245 features at the input layer.
have 4300 examples to train, so I divided the Train, Cv, and
Test sets in a way that there are an equal number of 'non-class
3' results in each set.(so => in each about 17 ).
was fixed at 200)
1- I train the network with my X and y Train set and get the
2 theta's (1 hidden layer) back from fmincg.
2- Then I use those theta's from the Train set as initial
theta's for the fmincg function when doing the Cv training with
eg. 2 different number of nodes in the hidden layer (350 and 400)
and use 3 lambda's so I get 6 combinations (n° of hidd nodes,
lambda, cost, theta-pairs).
3- Those 6 are used on the Testset 'predict' function which
makes the hypothesis, applies it to each example of the 6
combinations on Testset and checks the accuracy op the prediction
in % correct results.
So I have not used fmincg on the Testset.
In order to have a less than 1 failure on the 42 non-class 3
predictions I need a accuracy of 99,99% since only 1% is non class
Is the procedure I followed the correct one, or have I