Convolutional Neural Network (CNN)

One of the most popular deep neural networks is the Convolutional Neural Network (CNN). It takes this name from the linear mathematical process between matrices called convolution. 

In recent years, he has seen considerable success in the area of computer vision. Since CNN is particularly effective in processing large sets of image data, it has been successfully applied to the segmentation, classification, and identification of objects and other related fields. During the last decade, From image processing to speech recognition, the Convolutional Neural Network has had pioneering findings in different fields related to pattern recognition.

The most beneficial function of CNN is reducing the number of parameters in ANN. This achievement encouraged researchers and developers to look at layer models to solve complex tasks that classic ANNs were unable to perform.

Inspiration for the evolving neural network architecture comes from the structure of the mammalian visual system. In 1962 Hubel and Wiesel proposed visual structure model based on the cat visual cortex. First suggested was the idea of receptive field. In 1980 Fukushima proposed the first hierarchical structure Neocognition used to view images. The Neocognition pursues the local neuron-to-neuron relation, will invariant network translation. Yan LeCun is the first to implement a Convolutionary Neural Network. They have created a multi-layer artificial neural network called LeNet-5 that can count the number of handwriting. LeNet-5 has several layers like other neural networks, and can be equipped with the back propagation algorithm. However, at the time because of the lack of significant training data and processing resources. On more complex issues such as large-scale image and videorecognition, LeNet-5 can't perform well. Since 2006, several approaches have been developed to address the difficulties faced in training deep neural networks.

Krizhevsky proposes a classic Alexnet CNN architecture and shows considerable progress on the image classification task from previous methods. With Alexnet 's success, many works are proposed to improve its performance. It proposes the ZFNet, VGGNet and GoogleNet. Currently Convolutional Networks (ConvNets) are the most powerful deep models for classifying image images. The multi-stage architecture is influenced by biology theory. Invariant features are taught hierarchically and automatically through these models. First they classify features at low levels and then learn to recognize and combine these features to learn more complex patterns. These varying feature levels come from various network layers. And each layer has unique neuron numbers and is presented in three dimensions: height, width, depth.

To understand the structure of the Convolutional neural network we should view it as two separate parts. In the input images are represented as a pixel matrix. For a grayscale picture it has 2 dimensions. A third dimension reflects the colour, depth 3 to reflect the fundamental colours (Red, Green, Blue).

Reducing the number of parameters in ANN is the most advantageous feature of CNNs. This achievement has inspired both researchers and developers to seek broader models to solve complex problems, something that classical ANNs could not do. The most critical assumption regarding the issues that CNN addresses has no space-based characteristics. That is in a facial recognition application, for example, We don't have to give heed towards where the faces are in the pictures, the only concern is to detect them regardless of their location in the images given. Another essential feature of CNN is getting abstract features as the feedback spreads to next further layers. For reference the edge of the image can be found in the first layer and then the simplest shapes in the second layers, and then the features of the highest level.

Basic CNN Components

Convolutionary types of neural network layers comprise primarily four levels, namely the Convolutionary layer, the ReLU layer, the pooling layer and the completely connected layer. Figure Displays LeNet-5 architecture, introduced by Yann LeCun.

LeNet-5 Network [Tianamei Guo, Henjian Li, J. Dong ,Y. Gao , “Simple Convolutional Neural Network on Image Classification ”, 2017 ]

Convolutional layer

Convolutional layer is the central component of the Convolutionary Neural Network, which has local links and mutual property weights. The aim of Convolutional Layer is to learn input feature representations. Convolutional layer is, as shown above, made up of many feature maps. Each neuron of the same feature map is used to extract local characteristics of different positions in the former layer, but its extraction is local characteristics of the same positions in the former separate feature map for single neurons. To obtain a new feature, the maps of the input feature are first transformed with a trained kernel and then the results are translated into a nonlinear activation function. By adding different kernels, we will be having different function maps. The characteristic activation feature is sigmoid, tanh and Relu.

ReLU layer

After convolution, the next is nonlinearity layer. Non-linearity can be used to change or decrease the output produced. This layer is used to saturate the output or to limit the generated output. It is also known as the ReLU (rectified linear unit) layer.

Pooling layer

The method of sampling equals fuzzy filtering. The pooling layer has the effect of extracting the secondary feature, it can lower the dimensions of the feature maps and improve the robustness of extracting function. Typically it's put between two layers of Convolution. The size of feature maps in the pooling layer is determined by the kernel move phase. Typical pooling operations are average pooling and max pooling. By stacking multiple Convolutional layers and pooling layers, we can extract the high-level properties of inputs.

Fully-connected layer

Convolutional neural network classifier is, in general, one or more entirely connected layers. They take all the neurons in the previous layer, and connect them to each current layer neuron. No spatial information is kept in fully connected layers. An output layer is then followed by the last completely connected layer.