FindPet: using AI to identify lost pets

La Paz. Deep Learning. 2021

In this article we will use Artificial Intelligence techniques to find lost pets. Those people who have lost a pet for different reasons know that although many return to their owners, a large percentage does not achieve the reunion. About 420,000 pets, mainly dogs and cats, are lost annually in Peru and very few are found or returned to their owners, because it is very difficult to identify them.

Pet identification is usually done through metal tags hung around their neck, with their name and the owner's phone number. Others choose to insert a chip under the skin and a few, because of the cost, prefer GPS and even tattoo them. When they lose their pets, people choose to look for them through posters posted on posts, through social networks, advertisements in the media or by offering rewards in money or with tickets to concerts or soccer games. That is why the objective of this project is to help people find their lost pets through Artificial Intelligence and Deep Learning to identify their pets quickly, accurately and economically.

The problem

Every day, it is estimated that approximately 30 to 40 domestic animals, including dogs and cats, are lost in Lima alone. However, it is estimated that this figure may be higher, not only when accounting for the rest of the cities in Peru, but also because many people do not know how or where they can report the loss or loss of their pet. Very few pets are found or returned to their owners, because it is difficult to identify them and do the correct tracking. As a consequence, many pets end up as stray animals, causing overpopulation that ends up contaminating the city with their feces and the parasites that are expelled through them.

Objective

Develop an application that helps people report and find lost pets quickly using Artificial Intelligence with Deep Learning techniques.

Datasets

We used openly available datasets. For breed classification, we used the dataset from Stanford containing images of 120 dog breeds from around the world and Thudogs , containing 130 breeds with bounding boxes for each dog’s body and head. To identify individual dogs, we used an image dataset from Flickr , containing the faces of pugs and huskies, grouped by each dog’s name.

Pet Identification Process

The process of identifying lost pets using Artificial Intelligence consists of four phases as shown in the following figure, each of them is detailed below.

Coarse stage: breed classification

First, the classifier takes a photo of the lost or found dog and returns the N most likely breeds. These predictions help select candidate images from the stored records for the next stage.

MIDDLE PHASE: Face Detection

Next, YOLOv5 processes the dog’s image and the selected candidate images. It returns bounding boxes around the faces so that this region can be cropped in each image.

FINE PHASE: Identification of the pet

The identification stage uses cropped face images of the dog being sought and the candidates. These images are used to train and run the model that distinguishes individual animals in the dataset.

SUPERVISED DECISION

Finally, we include details such as the dog’s age and sex to narrow down possible matches and support review of the results.

Models

We used convolutional neural networks, specifically Inception v3 and Xception, to classify breeds and identify individual dogs. Each task has N possible outputs: 120 breeds in Stanford, 130 in Thudogs and 42 dogs in Flickr. We used YOLOv5 for face detection.

Inception v3

Inception-v3 is a convolutional neural network architecture from the Inception family that makes several improvements, including the use of Label Smoothing, 7 x 7 factored convolutions, and the use of an auxiliary classifier to propagate label information at the bottom of the network, along with the use of batch normalization for layers in the head-end.

Below is the architecture diagram of Inception v3:

Xception

Xception means “xtreme inception”. This architecture rethinks the way we view neural networks, particularly convolutional networks. And, as the name suggests, it takes the principles of Inception to the extreme.

Conventional convolutions jointly learn spatial patterns and relationships between channels. Inception partly separates these operations into different branches. Xception takes this further with depthwise separable convolutions: a spatial operation on each channel, followed by a pointwise 1×1 convolution that combines information across channels.

Below is the architecture diagram of Xception, where the data first passes through the input stream, then through the middle stream that is repeated eight times, and finally through the output stream. Note that all convolution and separable convolution layers are followed by batch normalization.

Yolo v5

Yolo v5 is an object detection model, and its first official version was released by Ultralytics. As YOLO v5 is a single-stage object detector, it has three important parts like any other single-stage object detector.

Model Backbone: It is mainly used to extract important features from the given input image. In YOLO v5, CSPs — Cross Stage Partial Networks are used as a backbone to extract a large number of informative features from an input image.
Model Neck: generates feature pyramids to detect objects at different scales. This structure helps recognise objects of different sizes; other architectures use variants such as FPN, BiFPN or PANet.
Model Head: performs the final detection using the extracted features and anchor boxes. It produces class probabilities, objectness scores and bounding boxes.

Below is the architecture diagram of Yolo v5:

Results

Coarse stage: breed classification

MIDDLE PHASE: Face Detection

FINE PHASE: Identification of the pet

Conclusions and Recommendations

In the coarse breed classification phase, it is concluded that the most optimal model is Inception v3 with an accuracy of 83.35% using the Stanford dataset and 78.28% for the Thudogs dataset.
In the middle phase of face detection, only the Yolo v5 model was tested, obtaining a MAP of 98.5%.
In the fine phase of face identification, it is concluded that the most optimal model is Xception with an accuracy of 67.5% for the Flickr dataset once the data augmentation has been performed.
The team proposes that this combination of models could help identify lost pets. Determining whether it reduces the time needed to reunite them with their families would require evaluation in real cases.
In the fine phase it is recommended to try GANs for data augmentation.
In the coarse phase, it is suggested to combine the Stanford and Thudogs datasets, and test the developed models.
In the coarse phase, it is recommended to create your own dataset for identification.
In the middle phase, it is suggested to try new models additional to yolo v5.

References