Facial Expression Recognition Using Euclidean Distance Method

Facial expression recognition is found to be useful for emotion science, clinical psychology and pain assessment. In the proposed method, the face detection algorithm involves lighting compensation for getting uniformity on face and morphological operations for retaining the required face portion. After retaining the face portion in the image, the facial features like eyes, nose, and mouth are extracted using AAM (Active Appearance Model) method. For automatic facial expression recognition, simple Euclidean Distance method is used. In this method, the Euclidean distance between the feature points of the training images and that of the query image is compared. Based on minimum Euclidean distance, output image expression is decided.


Introduction
Face biometric alone has many applications like security, forensic and other commercial applications. Similarly facial expressions are means of communication which are more rapid than the language through which people can quickly infer the state of mind of their companions. In 1978, Ekman and Frisen [1] reported that, Happy, Sad, Anger, Fear, Disgust and Surprise are the six basic expressions which are readily recognized across very different cultures. A system designed for an automatic facial action analysis through a human-computer interaction, is called Facial Expression Recognition System (AFERS). An Automatic Facial Expression Recognition system has a wide range of applications. The robust AFER system [2] can be applied in many areas of science such as emotion detection, clinical psychology and pain assessment. There are three major steps in an AFERS; the first step is to detect the face in the scene the second step is to extract the facial features like eyes, nose, mouth from the detected face and the third step is to classify the facial expressions conveyed by the face. Face detection is a special case of object detection. Early methods focused only on the detection and localization of human faces facing towards camera. The recent work is focused in handling complex issues i.e., to identify the human face in multiple views with different poses, illumination variations and with different rotations.
In the proposed system, lighting compensation algorithm and morphological operations are performed on the given input image and then, only face is retained from the input image. Active Appearance Model i.e. AAM method is used for the feature extraction. Finally the expressions are recognized by comparing Euclidean Distance between training and the query image. In the next part of the paper, section II explains Lighting Compensation algorithm and face detection using morphological operations. Section III gives Feature extraction using AAM method. Section IV deals with expression recognition using Euclidean Distance method. Finally Section V gives result analysis and Section VI concludes the paper.

Lighting Compensation
Many times there are large variations in the real colors of skin in an image, since the skin color is often affected by light in the image. Due to different brightness levels on the facial skin, the appearance of skin in light is not uniform. This may tend to deviate from the real color of the skin in the image. So to make color correction in color images, lighting compensation algorithm [3] is used. (1) The R, G and B are the amount of stimulus of red, green and blue respectively in the recorded scenery. R average , G average & B average are the average of each color channel. After applying this lighting compensation algorithm on the image, the results can be obtained as shown in figure 1.

Original Image
Image after Color balance

Face Detection using Morphological Operations
The Morphological operations are used to transform the signals into simpler ones by removing irrelevant information. These operations can reserve essential shape features of an image. Dilation and Erosion are the most basic morphological operations used in image processing. Dilation adds pixels to the boundaries of objects in an image, and erosion removes pixels on object boundaries. Firstly, erosion function is used to get rid of some small pieces, compared with face area, which is unwanted fragment. After that step, dilation operation will help to recover face area. This procedure can be done several times to get good result. The effect of Morphological operation is shown in figure 2. Using morphological operations (majority operator and applying dilations followed by erosions until the image no longer changes); the number of these regions are reduced. To deal with faces of different orientations, firstly the best ellipse can be calculated fitting the face candidate. Due to the fact that the pixel value variations of other skin-like regions such as hands or legs are smaller than those of face regions because of the presence of features with different brightness, all face region candidates with pixel value variations smaller than a threshold are removed. In order to improve the detection speed and achieve high robustness, the symmetry of the face is checked and all the candidates are removed when the symmetry is verified but no facial features are detected.

Feature Extraction using Aam Method
Image-based methods have been applied in many areas of facial computing [4]. One of the most successful recent techniques, which, incorporates both shape and texture information from facial images is the Active Appearance Model (AAM) method. It was developed initially by Cootes and Taylor [5] and has shown strong potential in a variety of facial recognition technologies.

Expression Recognition using Euclidean Distance Method
In this method, the database consists of training data sets and testing data sets of images. For one particular subject, the training and testing data sets consist of images of different expressions like Neutral, Happy, Sad, Anger, Fear, Disgust, and Surprise. Using AAM method, the points on facial features are located for all these images and stored in the form of data file. The data file consists of the relative x-y co-ordinates of those located points. When any test image is given as an input, the system finds the Euclidean Distance between the points on the test image and the points on each training image. If (x1, y1) and (x2, y2) are the coordinates of points on training and testing images respectively, then the Euclidean Distance is calculated by the following formula [7]: The training image for which, the Euclidean Distance of the test image is minimum, is given as the output image expression. As shown in figures 6 and 7, figure 1 GUI shows the testing image and figure 2 GUI shows the training image output.

Result Analysis
In the proposed system, person-dependent dataset has been used. Using this dataset, the system trained each subject (person) with different facial expressions. So all subjects with their facial expressions exist in both training and testing datasets. Results of the proposed system are compared with the recognition rates of [6] and [7] where person-dependent dataset has been used.
In [6], Ira Cohen, Nicu Sebe and others used different classifiers and compared their performance based on expression recognition rate as shown in table1. After comparing the results of [6] and [7], with the results of our proposed system, the recognition rates are found to be 98 % to 99 % which gave a good accuracy in the system.

Conclusion
An automated Facial Expression Recognition System has a wide range of applications in psychological research and human-computer interaction applications. The system plays a communicative role in interpersonal relations because they can reveal the affective state, cognitive activity, personality, intention and psychological state of a person. The proposed Face Detection module is based on image segmentation technique where the given image is converted into a binary image and further used for face detection. AAM (Active Appearance Model) method incorporates both shape and texture information from facial images, so it is found to be convenient for feature extraction. For expression recognition, Euclidean distance method is useful for static images and requires a large amount of manual work. Due to its ambiguity for real time or robust images, ANFIS (Artificial Neuro-Fuzzy Inference System) will be used as a further improvement. In this system, the static images as well as video input can be given and person-independent dataset can be used.