Convolutional Neural Networks - Part 2
February 15, 2019 — 12 min
Convolutional neural networks are widely used in addressing image-based problems, such as object/character detection and face recognition. In this article, we will focus on the most famous architectures from LeNet to Siamese networks, having for most of them the following architecture:
If you don’t have any knowledge about convolutional neural networks, I advise you to read the first part of this article, discussing the fundamentals of CNNs.
1 - Cross-Entropy
2 - Image classification
3 - Object detection - YOLOv3
4 - Face Recognition - Siamese Networks
When classifying an image, we often use a
softmax function at the last layer having the size where is the number of classes in question.
The line of the vector is the probability that the input image belongs to the class . The
predicted class is set to be the one corresponding to the
At the last layer , we have:
The network learns by using backpropagation and optimizes the
cross entropy defined as follows:
- is the reference probability and equals 1 if the object really belongs to the class filled out and 0 otherwise
- is the probability, learned by the network through the softmax, that the object x belongs to that class
For an input :
(General truth) (Predicted)
Thus, we set the
loss function as:
We average the loss, where is the size of the training set.
LeNet is an architecture developed by Yann Lecun in and it aims at detecting the digit present in the input.
gray-scale images of
hand-written digits from to , the convolutional neural network predicts the digit of the image.
The trainset is called
MNIST, which is a dataset containing more than images having pixels. The neural network has the following architecture counting more than parameters:
For more details, I advise you to read the official paper.
AlexNet is a famous architecture which won the ImageNet competition in 2012. It is similar to LeNet but has more layers, dropouts and ReLU activation function most of time.
The training set is a subset of the
ImageNet database, a million labeled images having a high resolution and representing more than categories.
AlexNet used more than million images in the training set, in the validation set and in the test set, which were all resized to . The architecture has more than
60 million parameters and thus was trained on
2 GPUs, it outputs a
softmax vector with size .
For more information, I advise you to read the official paper.
VGG-16 in a convolutional neural network for image classification, trained on the same dataset
ImageNetand has more than
138 million parameters trained on GPUs.
The architecture is the following:
It is more accurate and deeper than AlexNet since it replaced the large kernels and by successive kernels.
For more details, check the official paper of the VGG project.
Object detection is the task of detecting multiple objects in an image which comprehenses both object localization and object classification. A first rough approach would be sliding a window with customizable dimensions and predict each time the class of the content using a network trained on cropped images. This process has a high computational cost and can luckily be automized using convolutions.
YOLO stands for You Only Look Once and the basic idea consists on placing a grid on the image (usually ) where:
Only one cell, the one containing the center/midpoint of an object is responsible for detecting this object
Each cell of the grid is labelled as follows:
- : the probability of presence of an object
- : the
normalizedcoordinates of the object’s center
- : the
normalizedwidth and height of the object’s bounding box
- : the probability that the object belongs to the class
- : the number of classes
Hence, for each image the target output is of size:
Where is the size of the grid.
In order to evaluate the object localization, we use the Intersection Over Union which measures the
overlap between two bounding boxes:
When predicting the bounding box of a given object in a given cell of the grid, many outputs might be given,
Non-Max Suppression helps you detect the object
only once. It takes the highest probability and suppresses the other boxes having a high overlap (IOU).
For each cell of the grid, the algorithm is the following:
For each output prediction
- Discard all the boxes with (presence of an object)
- While there are any remaining boxes:
- Pick the box with the highest and output it as a prediction
- Discard any remaining box with with the previous output box
In most cases, a grid cell might contain
multiple objects, anchor boxes allow the detection of all of them.
In case of
2 anchor boxes, each cell of the grid is labeled as follows:
More general, the output target is of size:
Where is the number of classes and the number of anchor boxes.
YOLO was trained on coco dataset, a large-scale object detection, segmentation and captioning database with 80 object categories. YOLOv3 has a
Darknet-53 architecture as a feature extractor also called a
The training is carried out by minimizing a
loss function using gradient methods as well.
- Logistic regression loss on
- Squared error loss for
- Softmax loss(cross-entropy) for the probabilities
At each epoch, in each cell, we generate the output and
evaluate the loss function.
When making predictions, we check that is high enough and for each grid-cell, we get rid of low probability prediction and use non-max suppression for each class to generate the final output.
For more information, I advise you to read the official paper.
Siamese networks are neural networks, often convolutional, which allow to calculate the degree of similarity between two inputs, images in our case, as follows:
The purpose of the CNN module is to represent the information on the image in another space, called
embedding space thanks to a function. We then, compare the two embeddings using a certain distance.
Learning in Siamese networks is done by minimizing an objective function composed of a loss function called
triplet function takes 3 vector variables as an input: an Anchor , a positive (similar to ) and a last negative (different from ). Thus, we are looking to have:
Where for a given scalar product. To prevent the learned function from being null, we define the margin so that:
Thus, we define the
loss function as follows:
Starting from a learning database of size , the objective function to be minimized is:
Using gradient methods.
When training the architecture and for each epoch, we fixe the number of triplets and for each one:
- We randomly choose two images of the same class (Anchor & Positive)
- We randomly pick an image from another class (Negative)
A triplet can be:
- Easy negative, when
- Semi-hard negative, when
- Hard negative, when
We usually choose to focus on the semi-hard negatives to train the neural network.
Siamese networks can be used to develop a system capable of identifying faces.
Given an image taken by camera, the architecture compares it to all the images in the database.
Since we can not have multiple images of the same person in our database, we usually train the siamese network on an open-source imageset rich enough to create the triplets.
The convolutional neural network learns a similarity function which is the embedding of the image in , where is the size of the output layer.
Given a camera-picture, we compare it to each fo the database such that:
- if , both of the image represent the same person
- if , the images are of two different persons
We choose the face which is the closest to the in terms of the distance . The threshold is chosen in such a way that the -score is the highest for example.