ABSTRACT

Recognition of

Handwriting by humans may seem as a very easy task but when done by a machine

is a very complex one. It is impossible for humans to expend so much of their

time to interpret characters apart from them analyzing the collected data. Our

main focus should be on analyzing the data rather than trying to interpret the

data in the first place. Apart from this, the manual interpretation of data may

not yield the right results since it may wary from person to person and hence it

is not accurate to a great extent and may take a lot of time and energy.

Algorithms using neural networks have made this task a lot easier and more

accurate. Therefore neural networks have been utilized with an aim to determine

the characters by recognizing them using algorithms, which use various rules

for prediction of handwritten characters. In this paper we discuss the recognition of handwritten

digits taken from the MNIST data set and calculate the efficiency of our

algorithm.

Keywords

Introduction

Handwriting is a form

of writing peculiar to a person which it varying in size, shape of letters,

spacing between letters. There are different styles of handwriting including

cursive, block letters, calligraphy, signature etc.It makes the task of

recognizing handwritten characters a bit complex for neural networks since it

has to predict the characters based on its ability to learn rather than being

explicitly programmed.

Arthur Samuel and Tom

Mitchell offer the definitions of machine learning. An informal definition given

by Arthur Samuel is “the field of study that gives computers the ability

to learn without being explicitly programmed.” The more modern definition

is given by Tom Mitchell states: “A computer program is said to learn from

experience E with respect to some class of tasks T and performance measure P,

if its performance at tasks in T, as measured by P, improves with experience

E.”

Proposed Methodology

Digit recognition is done by training a multi-layer

feedforward neural network by using mini batch stochastic gradient descent and

backpropagation algorithm.

The MNIST data set obtained from link1 contains a

modified version of the original training set of 60,000 images. The original

training set is split into a training set with 50,000 examples and a validation

set with 10,000 examples. This set is then used to train the neural network. Each image is represented

as numpy 1-dimensional array of 784 float values between 0 and 1. The labels

are numbers between 0 and 9 indicating which digit the image represents. link1

The

neural network is an artificial neural network with sigmoid neurons. Therefore,

the output of each neuron is calculated using the sigmoid function.

The output is given as. Where, w is the

weight, b is the bias and x is the input.

Initially, the weights and biases of the neural

network are initialized randomly using Gaussian distribution. They are later

adjusted by applying mini batch stochastic gradient descent.

The training data is split into a number of mini

batches. In each epoch, the training data is shuffled and split into mini

batches of a fixed size and gradient descent is applied. The neural network is

trained over a number of epochs. The gradient of the cost function is calculated

by using the backpropagation algorithm. This calculated gradient is then used

to update the weights and biases of the neural network

Results

The settings of hyerparameters,

On taking epochs=30, mini-batch size=10 and learning

rate=3.0, we get the following accuracy for 10,000 test images in the MNIST

data set.

On taking epochs=20, mini-batch size=15 and learning

rate=2.8, we get the following accuracy for 10,000 test images in the MNIST

data set.