Spatial Analysis of Moments of Stress Derived from Wearable Sensor Data

Over the last years, we have witnessed an increasing interest in urban health research using physiological sensors. There is a rich repertoire of methods for stress detection using various physiological signals and algorithms. However, most of the studies focus mainly on the analysis of the physiological signals and disregard the spatial analysis of the extracted geo-located emotions. Methodologically, the use of hotspot maps created through point density analysis dominates in previous studies, but this method may lead to inaccurate or misleading detection of high-intensity stress clusters. This paper proposes a methodology for the spatial analysis of moments of stress (MOS). In a first step, MOS are identified through a rule-based algorithm analysing galvanic skin response and skin temperature measured by low-cost wearable physiological sensors. For the spatial analysis, we introduce a MOS ratio for the geo-located detected MOS. This ratio normalises the detected MOS in nearby areas over all the available records for the area. Then, the MOS ratio is fed into a hot spot analysis to identify hot and cold spots. To validate our methodology, we carried out two real-world field studies to evaluate the accuracy of our approach. We show that the proposed approach is able to identify spatial patterns in urban areas that correspond to self-reported stress.


Introduction
Today, more than half of the world's population lives in cities, making the creation of a healthy urban environment a major priority for policy makers. Mood and anxiety disorders are more prevalent amongst city dwellers, so mental health is negatively affected by urban life (Lederbogen et al., 2011). For instance, people might experience emotional responses during their daily activities in the urban environment (Layeb & Hussein, 2016;Knöll et al., 2018;Helbich, 2018). Residents and city users are exposed to pollution, crowded areas, extreme temperature, degraded landscapes, a high level of noise, etc. When these exposures are beyond tolerance levels, a person may feel stress (Rishi & Khuntia, 2012). The cognitive representation of urban space started in 1960 when Lynch asked participants to sketch a map out of their mind, creating a "mental map" (Lynch, 1960). Many years later, Matei et al. introduced the idea of a digital mental map to understand the environment of Los Angeles city (Matei et al., 2001). Nowadays, the rapid development of high-performance sensor technology has led to small and flexible sensors, which are the basis for pervasive sensing approaches in urban areas (Bergner et al., 2013;Resch, 2013). Researchers have investigated stress in correlation with traffic load, noise and environmental pollution (Birenboim et al., 2019;Knöll et al., 2018). Some research efforts investigated the restorative influence of the natural environment, such as green and blue spaces, on mental stress (Helbich, 2018;Ulrich et al., 1991;Birenboim et al., 2019). However, they still endeavour to better understand the specific environmental elements that cause urban stress. Therefore, researchers are looking for tools that contribute to a more objective investigation of the moment-by-moment environmental exposure and its impact on health (Birenboim et al., 2019).
in urban research . Most notably, wearable biosensors enable continuous monitoring of physiological conditions with high temporal resolution (Healey & Picard, 2005). The resulting data can be used for basic research, clinical application or during daily routines in real-life situations with less bias than selfreported surveys (Birenboim et al., 2019). These biosensors may be valuable tools to detect emotions as they provide high-quality data which are accurate, complete, timely, detailed, adequately portrayed and retain contextual information to support a decision-making process (Can et al., 2019). However, there are some preconditions for obtaining reliable and useful measurements that can be used for emotion extraction. First, the sampling frequency needs to be sufficient to accurately depict the signal. Second, the proper placement of the sensor is essential to avoid ambiguities and to correctly record the physiological signal. Despite these preconditions, the physiological signals usually exhibit many small fluctuations caused by the oscillations of the physiological status of the human body. It is inevitable to record these fluctuations, which need to be eliminated through filtering processes such as using the Kalman filter, Butterworth filter, etc. (Alberdi et al., 2016). These filters aim to remove noise in measurements and to produce a dataset with high accuracy (Guo et al., 2013;Schmidt et al., 2018). The selection of the optimum filter depends on the nature of the signal, the features to be extracted, and the type of noise (Alberdi et al., 2016). The combination of these wearable biosensors with location tracking technologies such as Global Positioning System (GPS) allows the collection of high spatiotemporal resolution information about subjects' location and their environmental exposure (Birenboim et al., 2019). This combination allows us to better understand cities as complex systems.

Stress Detection Strategies Using Physiological Signals
Currently, there is a rich repertoire of methodologies to detect stress from physiological sensor data. Researchers have used mainly GSR combined with ST, ECG, BVP and respiration rate (Setz et al. 2010;Wijsman et al., 2013;Lee et al., 2004;Sharma & Gedeon, 2013). Some researchers have combined the aforementioned physiological signals with pupil dilation, EEG and EMG. After measuring the physiological signals, classifiers are trained to identify the emotional states. Machine learning algorithms have been widely used, the most popular being a Support Vector Machine (SVM). The achieved accuracies vary from 74.50% (Wijsman et al., 2013) to 96.67% (Lee et al., 2004). Several studies attempt to detect stress states and the associated stress levels (low, medium and high). Most of these studies utilise GSR and ECG combined with ST, EMG or EEG (Healey & Picard, 2005;de Santos Sierra et al., 2011;Cho et al., 2017;Zhang, 2018;Liao et al., 2005). The classification of an affective state is performed using SVM, random forest, Bayesian network and fuzzy logic. The accuracies vary from 88.2% (Keshan et al., 2015) to 99.5% (de Santos Sierra et al., 2011). Some researchers aimed to classify distinct emotional states (stress, neutral, anger, etc.). In this case, researchers combined many physiological signals such as GSR, EMG, ECG, BVP, etc (Picard et al., 2001). SVM is again used but a feature-based algorithm achieved the highest accuracy (86%). To sum up, various physiological signals and their combinations can be used for stress detection, but the use of many physiological signals does not ensure the highest accuracy. Wijsman et al. tried to detect stress using four different signals and achieved 74.5% accuracy, while Setz et al. used only GSR and achieved a higher accuracy. Methodologically, the use of SVM dominates previous studies for stress detection, but the highest accuracy has been achieved by an introduced algorithm which combines various methods. Concerning the studies for estimating stress levels, GSR and ECG were found to be most helpful. An introduced feature-based algorithm and fuzzy logic seem to contribute to more accurate stress level classification. Many researchers achieved accuracies over 95%. On the contrary, the achieved accuracy for emotional states classification does not exceed 90%. This phenomenon is reasonable, as it is generally a complex task to understand emotions (Picard et al., 2001).

Proposed Approach
In this paper, we propose a multi-faceted methodology for the spatial analysis of moments of stress (MOS). In a first step, MOS are identified through a rule-based algorithm analysing GSR and ST measured by low-cost wearable physiological sensors. For the spatial analysis, we introduce a MOS ratio for the detected geo-located MOS. This ratio normalises the detected MOS in nearby areas over all the available records for the area. Then, the MOS ratio is fed into a spatial hot spot analysis to identify stress hot and cold spots.

Related Work
In his Biomapping project, Nold firstly used physiological signals combined with GPS data to explore the subjects' emotions. He collected GSR and ST through wearable devices, and then he mapped the collected physiological signals based on their GPS locations to describe the areas in terms of emotional arousal (Nold, 2009). Huang and Gartner proposed a mobile crowdsourcing approach to acquire people's affective responses through smartphones in their "EmoMap" project. They developed an Affect-Space-Model in which users were asked to evaluate their "level of comfort" in their environment using a 7-point Likert scale (from 1 being uncomfortable to 7 being comfortable) and optionally provide further details about their affective response concerning the aspects of safety, attractiveness, diversity and relaxation. The Affect-Space-Model was implemented in an Android mobile application to enable users to report their affective responses. The experience showed that with this mobile crowdsourcing approach, it is feasible to acquire affective data aroused by realistic scenarios from numerous users (Huang & Gartner, 2016). In 2018, the "EmoMap" was used again to acquire people's emotional responses in Vienna and Siena exploring two different spatial approaches. In the first approach, the authors opted to study areas with different Advances in Cartography and GIScience of the International Cartographic Association, 2, 2019. 15th International Conference on Location Based Services, 11-13 November 2019, Vienna, Austria. This contribution underwent double-blind peer review based on the full paper | https://doi.org/10.5194/ica-adv-2-9-2019 | © Authors 2019. CC BY 4.0 License levels of traffic and vegetation in Vienna. In the second, they endeavoured to identify which urban environments in Siena stimulate the emotional response and the different levels of comfort and discomfort. They concluded that the volunteered geographic information might shed some light on the interaction between humans and the environment (Capineri et al., 2018). Bergner et al. investigated the potential use of human sensory assessment for urban planning on a promenade in Alexandria, Egypt, which is shared by pedestrians and vehicles. Seven persons participated in the experiment, carrying cameras, GPS trackers and the wristband sensor "BMS Smartband". The researchers analysed the collected data through the software "StressPhaseIdentifier", which identifies stress phases and provides statistical information and the average length of a stress phase. They presented all the individual stress spots through a "stress hotspot heatmap" by implementing a point density analysis (Bergner et al., 2013). Leveraging the "Urban Emotions" approach (Resch et al., 2015b), Zeile et al. carried out a similar study in the cities of Cambridge and Boston, MA (USA), to investigate the possibility of using crowdsourcing mechanisms with physiological data to enhance the urban planning of cycling infrastructures. They recruited twelve participants who were equipped with (1) the Zephyr Bioharness 3 chest belt to record ECG, (2) a smartband to collect GSR, (3) smartphones with an application for participants' feedback and (4) action cameras. The stress detection was implemented based on the assumption that when a negative experience occurs, skin conductance increases and skin temperature decreases. The authors created hotspot maps with the identified points of negative arousal, and they further correlated them with video tracks to understand the reason for the emotional arousal (Zeile et al., 2016). Chen et al. carried out a pilot experiment asking four participants to walk a predefined route in the same direction three times. They equipped the participants with cameras and biosensors, which record ECG, EMG, GSR, ST and respiration. Then, they made two assumptions: (1) "affective arousal" indicates biphasic levels of stress, calm/ neutral vs aroused, and (2) "affective valence" indicates biphasic valence of emotional reactions, unpleasant vs pleasant. The correlated increases or decreases in arousal and valence indicators show the emotional responses of pleasant or unpleasant feelings. The results were presented as a heat map (Chen et al., 2016). Shoval et al. used ambulatory sensing technologies to record the two types of GSR, namely the skin conductance level (SCL) and the skin conductance response (SCR) from sixty-eight tourists in the city of Jerusalem. The SCL is the baseline level of a recording during an experiment without any environmental events, while the SCR represents the body's reaction in the presence of a stimulus. They calculated z-scores for each measurement to compare the skin conductance level (SCL) measurement, taking into consideration the variations amongst the participants. The z-SCL score represents the extent to which a singular SCL measurement compares to the mean SCL score of an individual. Thus, the acquired values represented the extent to which a particular measurement of SCL is high or low relative to the other SCL measurements of the same individual. Then, the authors paired the SCL data with GPS data. They assumed that high SCL might characterise excitement or fear. They also collected subjective measurements asking the participants to state their perceived level of arousal from 1 (low) to 7 (high). The study area was divided into cells (20m * 20m) to implement a spatial analysis of stress points. They calculated the mean z-SCL score of all participants who passed through each cell and the proportion of participants who passed through each cell. Mean z-scores were calculated only for cells visited by at least 5% of participants to highlight the areas of significant activity and to enable a precise analysis of the results. The aggregation of subjective emotional data was done through a spatial interpolation technique to convert discrete measurements into a continuous distribution. The technique used was inverse distance weighting (IDW) spatial interpolation (Shoval et al., 2018). Fathullah et al. recorded participants' GSR and trajectories along a walking tour in Plymouth City Center. They recruited nine volunteers, and they defined a path which contained a park, urban pedestrianised areas, and roads. Then, they investigated the peaks of GSR to detect stressful places. The outcome was a map with the average of all the participants' stress levels (Fathullah & Willis, 2018).

Problem Formulation
The works which combine health and geography are prone to biases such as selection bias and spatial distribution bias. The study cases that were depicted in Section 2 are prone to spatial distribution bias which, has not been taken into consideration by all the researchers. There are a variety of approaches to spatially analyse the detected stress points in urban areas. However, most of the previous approaches used a binary classification scheme (stress vs no stress) -which is still state of the art for research efforts involving wearable sensors in mobile settings. This binary attribution does not allow for autocorrelation-based hot spot analysis, which performs attribute-wise clustering rather than density-based clustering. In fact, the binary classification scheme provides only density-based clustering. Also, a major shortcoming of previous approaches is that the applied methods, such as density estimation or point density analysis to create quasicontinuous hotspot maps, are usually not normalised over the entire physiological dataset for the sub-region of study. Consequently, an essential factor of the spatiotemporal nature of MOS (accounting for varying speeds of movement) is disregarded. Figure 1  velocity. On the contrary, there are fewer GPS points close to point p i+n and most of them have been detected as "no stress". Point p i+n is located on a road segment where participants passed through continuously at a higher speed without stopping. Thus, fewer data points were generated for this area. These cases give an attribute to the two detected MOS, the relation between the detected MOS and all GPS points. This relation has been neglected in most previous research efforts, but it has to be considered for the detection of spatial high-intensity stress clusters. Shoval et used a more advanced clustering method considering the number of participants who passed through their defined sub-regions, but they disregard the parameter of "spending time" (or speed of movement) at every single sub-region. So, sampling methodology and the related frequency leads to spatial distribution bias which may promote inaccurate conclusions. The source of this bias is different from the background population bias, but it can be dealt with methods for background population bias. However, these methods will not allow us to have an explicitly quantified attribute for every point.

Algorithm for Stress Detection
To address the shortcomings illustrated in Section 3, we developed a rule-based algorithm that detects MOS using physiological signals which are provided by the wearable sensor "Empatica E4". It is an unobtrusive and noninvasive wristband designed for research and clinical purposes (Empatica, 2019), including medical (FDA) and electronic certifications. The sensor measures BVP, GSR and ST in real-time, at sampling frequencies of 64Hz, 4Hz and 4Hz, respectively. The accuracy of GSR varies from 0 to 100μS, and the accuracy of ST is 0.02 o C. The device also contains a 3-axis accelerometer to capture the physical movement. In our study setup, the human sensor data are fused in an eDiary smartphone app, which connects the wearable sensor via Bluetooth. The eDiary app collects GSR and ST, plus the emotions subjectively perceived by the test persons (Resch et al., 2015a). It creates a database containing the collected physiological signals, and it automatically adds timestamps to all measurements based on the smartphone system. It also geolocates the data, making it possible to associate the detected MOS with their spatial context (Zeile et al., 2016). Our algorithm detects moments of stress in the following steps: First, we define criteria for stress detection and threshold values based on previous research results. Second, we perform a pairwise comparison to calculate weights for each rule based on the assumption that each rule has a different significance for representing the phenomenon of stress. Third, we adopt a ternary scoring system (0, 0.5 and 1) to assess the degree of rule fulfilment. We determine critical values to assess the partial fulfilment through an experimental process, by modifying the critical values aiming to detect as many as possible induced MOS. Table 1 presents the defined rules and the associated critical values for three levels of fulfilment: complete (condition for score 1), partial (condition for score 0.5) and no fulfilment (condition for score 0). We downsampled the collected physiological signals from 4.0 Hz to 1.0 Hz to obtain one value per second to establish comparability between the signals in the frequency domain. Thus, in 1 Hz intervals, the algorithm assesses the rules and allocates a score (0, 0.5 and 1) which is multiplied by the weight of each rule respectively. The Total Score (TS) for a second is calculated by: TS = ∑ (sc * ) 5 1 (1) where sc is the given score for the rule and wn is the associated weight of the rules and ∑ = 100 5 1 . The maximum achievable TS is 100, provided that all rules are scored with "1", so it follows that 0 ≤ TS ≤ 100. Through an empirical procedure, we defined 75 as the critical score (CS) to identify a MOS. In its current version, the algorithm works as a binary classifier. Therefore, if TS is greater than CS, a MOS is detected, and the value "1" is Advances in Cartography and GIScience of the International Cartographic Association, 2, 2019. 15th International Conference on Location Based Services, 11-13 November 2019, Vienna, Austria. This contribution underwent double-blind peer review based on the full paper | https://doi.org/10.5194/ica-adv-2-9-2019 | © Authors 2019. CC BY 4.0 License assigned to the specific second; otherwise, the value "0" is allocated. The algorithm detects MOS with 84% accuracy. More details about the algorithm can be found in (Kyriakou et al., 2019).

Spatial Analysis
To spatially analyse the detected moments of stress, we designed an algorithm following the problem statement in Section 3.

MOS Ratio
As it is described above, the spatial analysis of the physiological measurements from real-world field studies needs to consider number of available data points per defined sub-region. For this reason, we introduce the "MOS ratio", which works as a weighting parameter describing the nearby surroundings of a single MOS. The MOS ratio is calculated using aggregated data and includes the total number of detected MOS, together with all the available data points per sub-region. We define a subregion as the area within a circle with a given radius depending on the average walking speed and the minimum number of seconds that are required to detect a MOS. The following equation summarises this procedure: where pi designates each geo-located point with i = 1,2,… n, and m denotes a detected MOS. The maximum achievable MOS_r is 1, provided that all points have been detected as MOS, so it follows that 0 ≤ MOS_r_p ≤ 1.

Spatial Autocorrelation
According to Tobler's first law of geography, "Everything is related to everything else, but near things are more related than distant things" (Tobler, 1970, p. 234). This law is quantified by the presence of significant spatial autocorrelation (Getis & Ord, 1992). Thus, we perform a hot spot analysis using the i * method (Ord & Getis, 1995) based on the following equation: Where i * is the statistic that describes the spatial dependency of incident i over all n events, x j is the magnitude of Variable X at incident location j over all n, w ij is the weight value between even i and j that represents their spatial relationship. A close-to-zero i * value implies random distribution of the observed spatial events. Conversely, positive and negative i * statistics with high absolute values correspond to clusters of high-values events (hot spot) and low-values events (cold spot), respectively.

Results
We used our proposed approach to investigate urban walkability comparing the cities of Salzburg, Austria and Cologne, Germany. We recruited 56 participants (27 for Salzburg and 29 for Cologne) who were instructed to walk with sensors mounted on their bodies (Empatica e4 wristband, Zephyr BioHarness, plus GoPro ego-video camera). They were also asked to enter inputs into the eDiary app on a smartphone that they carried with them and to answer a customised questionnaire after their walk. Figure 2 shows the geospatially analysed (Getis Ord Gi*) physiological sensor data and the eDiary app entries for Salzburg. Red areas indicate hot spots (spatially clustered moments of stress) and blue areas cold spots (spatially clustered moments of relaxation). Red points represent eDiary entries correlated with stress states, while green points represent entries for calm states. The results show that suspected stress hot spots can be identified close to major tourist hubs (Hanuschplatz square, in front of Mozart's birthplace and Residenzplatz square). Participant's feedback indicate also these areas as stressful areas. Areas of relaxation are also detected and confirmed by eDiary entries. Figure 3 illustrates the spatial analysis of the same physiological sensor data using unmarked Kernel Density Estimation (KDE), as many previous approaches have done. Figure 4 presents the hot and cold spots for Cologne. Tourist places and shopping areas are identified as stressful areas. Figure 5 depicts the spatial KDE of physiological data for Cologne. Both methods can identify stressful areas for both cities which are in line with the subjective perceptions of the participants provided through the eDiary app and the questionnaire based on a visual inspection. However, as KDE disregards the speed of movement, it reveals a different picture concerning the "cold spots" (i.e. areas of relaxation). Unmarked KDE is not able to identify "cold spots" as Figure 6 and 7 present for Salzburg and Cologne respectively. However, these "cold spots" have been identified by using participants' feedback. This leads to the conclusion that unmarked KDE can identify only stressful areas. Despite this, KDE interpolates the entire area making it impossible to identify specific stressful areas, as Figure 8 depicts. More details on the qualitative conclusion can be found in (Doerrzapf et al., 2019)

Conclusions and Future Work
In this paper, we proposed a spatial analysis method for the geo-located MOS that have been detected using a rulebased algorithm that we developed in a previous study. The algorithm uses GSR and ST, recorded by low-cost wearable physiological sensors. Our approach introduces a MOS ratio for the geo-located physiological data and a spatial autocorrelation analysis. The proposed methodology detects spatial patterns in the physiological measurements, considering not only the detected MOS in the nearby area but also all the available data for the area. Also, it has been developed as generically as possible, allowing a wide implementation in real-world field studies. The comparison of our proposed approach with a traditional unmarked KDE analysis, reveals that our approach, which integrates the MOS ratio, reduces aggregation bias. Thus, the main contribution is that we identify high-intensity stress clusters with higher accuracy, and can detect both hot spots (areas of stress) and cold spots (areas of relaxation). Marked point pattern density analysis could provide similar to i * that is calculated by a    Despite this limitation, KDE can identify clusters but not their statistical significance as hot spot analysis does. The revealed dark red areas from hot spot analysis signify areas where there is intense clustering of high values with 99% confidence. These are areas where there are high numbers of MOS. Also, hot spot analysis allows us to estimate the density distribution of MOS at a local level, as present in Figure 8. Another limitation is the use of symmetric spatial An important limitation is that particular parameter choices of KDE (kernel bandwidth, output grid size, etc.) and hot spot analysis (cell size, search band) will lead to different outputs affecting the results. For our analysis, the parameters were automatically derived from the dataset's properties by the application that we used.   weights in hot spot analysis. The integration of asymmetric spatial weights shall be investigated in future work. This paper depicts a method that works efficiently for the detection of stress areas and can be further improved. First, we will integrate the temporal dimension to identify clusters in space and time in a spatiotemporal autocorrelation approach. These patterns will enhance urban analysis by identifying urban stressors that must be decreased to improve human wellbeing. Second, we aim to improve our algorithm for MOS detection, by integrating more physiological signals such as HRV and replacing the binary classifier (stress vs not stress). Thus, the algorithm will be able to detect a quasi-continuous intensity of stress for each detected MOS. Third, we will introduce a ratio to dynamically define the radius of the sub-region based on the average speed of subjects in the area around each point and the minimum time-window that our algorithm requires to detect a MOS. Last, we will use a statistical method to evaluate the correlation numerically amongst the hot spot and cold spot areas and participants' feedback.