Facial Expression Recognition Based on Facial Motion Patterns

Facial expression is one of the most powerful and direct mediums embedded in human beings to communicate with other individuals’ feelings and abilities. In recent years, many surveys have been carried on facial expression analysis. With developments in machine vision and artificial intelligence, facial expression recognition is considered a key technique of the developments in computer interaction of mankind and is applied in the natural interaction between human and computer, machine vision and psychomedical therapy. In this paper, we have developed a new method to recognize facial expressions based on discovering differences of facial expressions, and consequently appointed a unique pattern to each single expression.by analyzing the image by means of a neighboring window on it, this recognition system is locally estimated. The features are extracted as binary local features; and according to changes in points of windows, facial points get a directional motion per each facial expression. Using pointy motion of all facial expressions and stablishing a ranking system, we delete additional motion points that decrease and increase, respectively, the ranking size and strenghth. Classification is provided according to the nearest neighbor. In the conclusion of the paper, the results obtained from the experiments on tatal data of Cohn-Kanade demonstrate that our proposed algorithm, compared to previous methods (hierarchical algorithm combined with several features and morphological methods as well as geometrical algorithms), has a better performance and higher reliability.


Introduction
Recently, according to the interaction between human and computer and existing intelligent systems, this human-computer interaction is deemed as one of the important issues in human daily life.Gesture recognition is a process in which man-made gestures are recognized by a computer.Gestures include meaningful motions of body such as physical movements, fingers, hands, arms, head or the body.
Facial expressions is among the most powerful and direct.According to the broad application of this field, automatic recognition of facial expressions has attracted more attention in recent years.Despite the major developments in the field, facial expression recognition with high accuracy is really difficult, because of the complication and changability of facial expressions.Facial expression recognition may be a bit complicated due to the human age, race or sex, or whether the face is covered with cosmetics or other covering stuff such as glasses and hair.This paper focuses on the recognition of changed expressions in stable pixels, which is the basis for facial expression recognition in different media.
Generally, there are two ways for facial expression recognition: methods based on the model, and those based on the appearance.Their difference lies in the way to describe the information about the expression.Model-based methods describe expressions according to the anatomy of the face and shape of body organs.AAM is one of the favorite and common modelbased metods, while Cohn and Kanade mostly support combinations of AUs for facial expressions.In this paper, we detect facial expressions by considering pictures' appearance and determining the differences between various expressions.This facial expression recognition is carried out according to human detection, i.e. by appointing and destingishing the most difference in facial changes with other expressions.The tests on total C-K data introduced the validity of the proposed algorithm.
The paper is organized as follows: In Part I, the previous works are described.the proposed algorithm is introduced in Part II.In Part III, the proposed method is assessed with previous methods.In fourth part the conclusion and in fifth part future work are presented [1].A review of previous works.We generally intend to classify seven main categories of facial expressions: anger, disgust, fear, happiness, neutral, sadness, surprise.
In hiearchical algorithm with several features, the basis and framework of the algorithm is that studying previous works led to the understanding that the accuracy and precision of both happiness and surprise are always reported in total result, and also prototypes of other five categories are blended in all dimensions.In such circumstances, they first distinguish two "easy" categories from the rest of the cases, and then with less interference, pay more attention to other five cases.doing this has increased detection percentage [2].
Another study of neuro-fuzzy classification network is carried on the extraction of binary local numbers, in which it detects expressions by deviding various parts of the face into five sections.The results has a low standard deviation in detection, but the high complexity makes this method less ideal [3].

Proposed Method
First, we show the stages of proposed method in the diagram of Figure 1.

Choosing the face
First, we use the following algorith to locate the faces and cut out the face in the picture.The picture is normalized by rotation and leveling, and we cut the face from forehead to chin, and also from one ear to the other ear.Then, LBP is extracted for the face and is located in ageneral matrix for one picture that is presented in Figure 2 [4].
Normalizing pictures: 1. Rotate the images so that they are aligned with the the eyes by vertical rotation.2. scale the face image so that the distance between the eyes is the same for all face images (scaling the image is equalizing pixel of all face components) 3. cut (face) to remove the background and the hair 4. Apply equalization histogram on face to normalize photometric.After processing, the image size is 150 × 200. Figure 1 shows the normalized procedure.

Local binary pattern
LBP operator tags the pixels of an image by 3x3 neighborhood thresholding in each single pixel, and results in a binary figure.LBP feature has been frequently used in SEX classification [6].
In equations 4 and 5, f c is the value of pixel center and f p is the central pixel neighborhood.The LBP value in central pixel is obtained from equation 5.The symbol , is used invariably for LBP operator [6], in which the use of LBP operator in a neighborhood of P points is sampled on the circle of radius R. Figure 3 shows the LBP operator for neighborhood.And Figure 4 shows LBP results for R=1 and P=8.The use of these features aimed to the strength of feature extraction in facial expression detection which ahs been recently the most popular and had gained the higher detection percerntage [5].

Formation of a motion pattern for the face
To choose one motion in order to get informed of the changes in one part of the image, we divide the human face into 3*3 neighbor frames and convert the main parts of the face including eyes, mouth, nose and forehead, formed in section 2-1, into figure 5. [7]- [10] Figure 5. how to partition the face In Figure 5, face movement will be seen clearly with respect to the positioning of the human face in concordance with the formed partitions.Figure 6 shows how these partitions are embedded in the face.

Remove motion add-ons
According to extraction of rules for each facial expression motion, we extract several rules for the face which overlap between all motions, and hence, remove other motions.This is done as follows: 1. choose one facial expression in order (sadness, happiness, .., etc) ( we have 7 facial expressions) 2. Compare motion patterns of these facial expressions based on the motion angle.This means that we consider those motions in one angle together (in trial and error, the best rate for obtained angles, is 10 degree different).3. The motions located in the face, but undertaken no change, will be removed.4. Remained motion patterns will be organized normally, that is we organize angles of one direction equally, this will remove extra complexity [11].

Classifying the images:
To make a reasonable classification according to the structure of proposed algorithm, we choose the classification of neighbor nearest which has the best result in the existing classifications (neural networks, support vector machine, etc.).According to tests performed using the nearest neighbor with k = 1, the result obtained is revealved in Figure 9. Interestingly, the papers [12]- [13] have also obtained the same result for the value of K, according to the extraction of features in the face images.

Evaluating Proposed Method
To evaluate proposed method, we made use of Cohn-Kanade data set, which contains 123 persons with 7 different state of consequent images of one single facial expression.According to the Table 1, we have compared the strength of proposed method with previous methods.
Sixty percent of Comparisons are devoted to tain images and 40 percent to test data.Choosing test and train data is done on the basis of cross-validation, which avoids hunger and enclosure in classes, and the accuracy rate of recognition will be more precise.In compared papers, the ratio of test and train data is obtained similarly.(11) Table 3 shows confusion matrix for proposed method.

Conclusion
In this paper, we introduced a new method based on the detection of changes in the face by partitioning facial forms.The proposed method extracts feature by using binary local features, and then place the points changed in this classification in a matrix, after matching change formats of the face with human face.The proposed method summarizes extracted matrix, and maintains the most facial changes for classification.Results show that proposed method has better accuracy in facial expression recognition, compared to previous methods.

Future works
Since facial changes in humans is located in certain parts of the face, we will be able to detect the amount of changes in each facial expression by forming movable sections in these parts.And for well removing of add-ons in this human motion matrix, we would better use an efficient fuzzy system, which removes considerable amount of additional motions and avoids negative effect in classifying facial expressions.

Figure 4 .
Figure 4. Applying LBP operator for pictures with parameters R=1 and P=8

Figure 6 .
Figure 6.Locating templates on face

Figure 7 .Figure 8 .
Figure 7.The sequence of faces in happiness

Figure 9 .
Figure 9. Determining best rate for K-neighbor nearest

Figure 10 Figure 10 .Figure 11 .
Figure10presents distribution diagram of data set; as you can see, detecting some feelings are severely tough, but applying changes in seperation of datas in Figure11shows that data seperation has been done more clearly.

Table 1 .
Comparing proposed method with other methods

Table 2
Compares proposed method with other methods in all considered facial expressions.

Table 2 .
Comparing existing methods

Table 3 .
Confusion matrix for proposed method