Interactive Visualization System for Psychological Topology

Received Jun 9, 2018 Revised Nov 20, 2018 Accepted Dec 11, 2018 Recently, there is increasing interest in mental support activities, including mental health care, counseling, and mental training in workplaces, schools, and sports teams. As a background to these things, various analysis methods have been developed to clarify and visualize the subject’s mental state based on these data. We tried to reveal and visualize the transition patterns of the subjects’ mental states by analyzing their utterances. Furthermore, we developed an interactive system of visualization of psychological state to support visual understanding of psychological topology. Features have been implemented to enable multidimensional data to visualize the movements shown on the SOM map. In this paper, we describe the system that can interactively visualize psychological states.


INTRODUCTION
Recently, there is increasing interest in mental support activities, including mental health care, counseling, and mental training in workplaces, schools, and sports teams [1][2][3]. Following this, psychological assessment for employees, students, and athletes who receive such mental support is becoming more important. Psychological assessments are normally conducted through questionnaires, which can evaluate object people easily and quantitatively. It is also necessary, however, to acquire qualitative data through interviews in order to understand the mental states of groups and individuals.
We have worked on such visualization studies. So far, we have obtained some achievements regarding the introduction of psychological topology and the extraction and visualization of psychological transiting patterns. For the effective comprehension of emotional state and transition, an analysis procedure and recognition protocol has been proposed by using athletes' interviews. The verbal data was decomposed with morphological analysis into some units such as morphemes, words, and utterances. From those data, by extracting major words, TF-IDF [4] was estimated at first. After BLSOM [5] and fuzzy cluster analysis, consequently, we obtained an emotional state and transition for individual interview articles.
Qualitative verbal and text data such as narrations and descriptions can include additional information about a subject's spiritual response or deep psyche than questionnaires as quantitative data [6,7]. Therefore, in the early previous studies, various analysis methods have been developed to clarify and visualize the subject's mental state based on these data. Then we focused on the sequential transitions of the visualization results from our previous study [8,9].
The purpose of the study was to reveal and visualize the transition patterns of the subjects' mental states by analyzing their utterances. The mental changes were expressed with trajectories in two-dimensional space in which the relationship between various emotions was represented with self-organizing maps (SOM)  [8,9]. These patterns were visualized on the twodimensional space composed of psychological evaluation axes, thus the visualization results were interpreted in terms of psychology. So we could obtain conclusion that the group's tendencies in terms of mental changes can be comprehended by noting on the transition of the mental states. However, as for visualization, one direction from the creator to the user side is not enough, and functions such as sorting by the user's intention by interactive operation so that the user can easily understand the visualized objects are needed. Therefore, how to realize these visualization remained an issue for understanding psychological states more effectively.
In this study, we developed an interactive system of visualization of psychological state to support visual understanding of psychological topology. Features have been implemented to enable multidimensional data to visualize the movements shown on the SOM map. In order to deal with the above mentioned issues, this paper describes dynamic visualization and interactive operation functions in the system. Section 2 explains basic visualization methods and system design. Section 3 describes interactive visualization for understanding psychological states, Section 4 concludes the paper.

RESEARCH METHOD
In this section, we describe our system design, data formats and acquisition methods, experiment executions and results.

Overview of Our Interactive Visualization System
By quantifying the psychological features read from the language data and performing analysis by SOM, it enables to visually represent the psychological features of the group and the mental changes of the individual on a 2-D plane. In order to visualize the above analysis results more effectively and to facilitate psychological interpretation, we developed the system in Java-based visualization language Processing (ver. 2.2.1).
In the system, the analysis result by SOM is firstly read, and then a map including the classification result of the input vector and the update result of the representative vector is output. In addition, the input vector for each subject can represent the coordinate transition shown on the map of SOM by animation. Furthermore, we have realized an interactive visualization system that can selectively display the component of interest of the representative vector, or allow the user to replace the neuron on the map with an arbitrary arrangement. Details of the features and implementation of our system are described in the next section.

Input Data to the System
When the system is started, it is necessary to read CSV data as input data. The input data includes the SOM input vectors, and the classification results of the input vectors and the update results of the representative vectors as analysis results. Figure 1 shows a part of input data that can be obtained using R software. R provides a wide variety of statistical as free software, e.g., for simulation [10] and image analysis [11].
The input data was created through the results of interviews with a university student athlete (a rugby football player). The interviews were conducted one to one between the investigator and athletes. Athletes were free to comment on what they felt and thought during the game while watching the video of the game in which they played (20 scenes were extracted from the first 40 minutes). Based on the recorded voice data, the athlete's comments were converted to text data and divided into 20 scenes. In addition, among the comments of each athlete, the appearance frequency of keywords ("do it hard", "tension", "pressure" and "fear") indicating mental state was calculated for each scene.
Furthermore, TF-IDF values of each keyword were calculated, and vector data with these values as vector components was used as an input vector. Using the input vector obtained by the above procedure as an input, analysis by the SOM package of software R for statistical analysis was performed. Then we obtained results on classification of input vector and updating of representative vector. These analysis results can be read from the system side by putting them in the form of CSV data as shown in Figure 1.

RESULTS AND DISCUSSION
The issue is how the user can interactively capture psychological states. This section describes how our system realizes visualization and intaractive operations to support understanding of psychological states.

Basic visualization operations of our system
When the application is executed on Development Integrated Environment Processing, a control panel ( Figure 2) and a map (Figure 3) showing the analysis results of SOM are displayed. The control panel controls the visualization of analysis results. Figure 3 shows the display when input data is not read. The user can select and read CSV data to be used as input data by pressing the "Read File" button on the control panel. In the SOM adopted by us, since the input data is mapped to the 5✕5 output layer, so the application side also displays the 5✕5 map as a result. Figure 3 is a map of the initial state before reading data, but in each neuron, numbers 1 to 25 are described as ID, and the indication of "n = 0" is the input assigned to the neuron. It means that the number of vectors is 0.
When the input data is read by the "Read File" button, the read input data is processed, and an initial state corresponding to the data is displayed on a display screen for visualization as shown in Figure 4. As the result, some neurons will write numerical values. These numbers indicate the numbers of input vectors assigned to each neuron.
Once the data loading is complete, the user can use the control panel to select how to display the SOM analysis results. "Element 1" to "Element 10" in control panel shown in Figure 4 show the components of the representative vector displayed as the analysis result. Here, according to the number of components of the input vector (= 4), select "Element 1" to "Element 4" as shown in Figure 5. These correspond to the TF-IDF value of each keyword ("do your best", "tension", "pressure" and "fear") found in the athlete's comment. In this case, it is also possible to select and display only specific components in the representative vector.  In this system, it is also possible to select and display only specific components in the representative vector. Also, for the display method of analysis results, it is possible to select from multiple modes. In the following, we will illustrate "Display mode 1" to "Display mode 3" that are implemented in the application.
Firstly, in Display mode 1 (Figure 6), the components of the representative vector possessed by each neuron are displayed in numerical form. The numerical values described in the neuron correspond to the values of Element 1, 2, 3 and 4 in order from the top. As shown in Figure 6, the specific numerical values of the elements can be confirmed. Next, in Display mode 2 (Figure 7), the value of each component of the representative vector is indicated by the size of the fan that constitutes the pie chart. Since the number of components of the representative vector is 4, the pie chart is divided into four, and each fan represents the size of Elements 1, 2, 3 and 4 from the upper right. As shown in Figure 7, each element can be visually confirmed.

Animation and Interactive Visualization
In Display mode 2, if the number of Elements to be selected is less than 4 ( Figure 8, select Elements 1-3 here), the analysis result is part of the pie chart as shown in Figure 9. On the other hand, in Display mode 3, the pie chart is divided according to the number of selected elements. When Elements 1-3 are selected, the graphs are divided into three as shown in Figure 10, and each fan represents the size of each Element. As shown in these figures, by changing elements and display modes on the control panel, SOM analysis results can be visualized in any combination.
In the control panel, when the user presses the "Animation" button, an animation in which a light blue sphere moves on the map is reproduced on the displayed map ( Figure 11). The sphere moves the neuron to which each input vector is assigned on the map according to the number of the input vector. That is, by using the function of Animation, coordinate transition on the map of input vector can be represented dynamically. In Figure 11, the transition between the neurons 13 and 14 is performed as the transition of the input vector. It is expected that our animation representation will be used as an effective visualization method that emphasizes transition features more than general static diagrams that represent coordinate transitions. With this function, the overall state and change of the psychological state can be captured by animation. In addition, in this application, it is possible to select an arbitrary neuron on the map and move the arrangement freely (Figure 12), and the realization of this function takes advantages of features of Objectoriented language. At this time, using the above-mentioned Animation function, the sphere moves between the neurons whose arrangement has changed in the order of the input vector. By pressing the "Reset" button on the control panel, the neuron returns to its initial configuration. In Figure 12, neurons 4, 13, 14, and 22 are picked up on the right side of the screen. An animation is also executed according to these arrangements. In this way, an interactive analysis interface including animation is provided.
It has been reported that subject's statements and descriptions are related to their physical and mental health [12][13][14]. We have demonstrated the effectiveness of psychiatric changes in modal patterns [8,9]. Other studies have challenged to visualize sequential transitions of events described [15,16]. However, there is room for improvement as a visualization method. We have realized interactive dynamic visualization, and it became possible to interactively capture the characteristics of psychological transitions exhibited by each subject.

CONCLUSION
In this study, an application was developed to visualize the analysis results of language data. This application has mainly two features. One is the realization of dynamic visualization, and the other is the interactive operation according to the user's point of view. The visualizations presented in our previous work are all with static images, which represent only the coordinate transitions of the input vector on the map.
On the other hand, if the application developed this time is used, by representing the transition on the map in the form of animation, the characteristics of the transitions shown by each subject are emphasized, and psychological understanding can be promoted. Furthermore, by changing the arrangement on the map, specific neurons can be extracted and it became possible to consider the nature of the representative vector and it became possible to observe the characteristics of movement between neurons. In combination with the selection function that displays only specific components of the representative vector, it can be said that it is a function that brings about the visualization according to the purpose of analysis.
As mentioned above, by applying this application to the analysis result of language data, it was shown that a more effective visual representation, which is in line with the analyst's goals, could be realized. The issue for the future is the expansion of further functions. In the application developed this time, one can simultaneously visualize the coordinate transition indicated by the group. However, the problem is that it is difficult to find common patterns. In order to solve this task, that is, to reveal the tendency of psychological change in a specific group, it is necessary to add different visualization methods. Therefore, the future direction of this study will be interactive dynamic visualization to solve the problem.