Philipp MiedenAbstract— Measuring stress symptoms has been a challenge for many years, but recent technological advances have created newpossibilities, which do not only affect data collection but also data analysis on a large scale. Sensors are becoming more accurate,while manufacturing costs are shrinking, making them affordable also for private researchers. Additionally smartphones offer apowerful platform for data collection, since they are being used throughout the day. This paper summarizes different techniques forstress detection and discusses their applicability including use of phyisiological sensors, surveys and thermal video data analyis. It isshown that tradtional approaches have limitations and can deliver wrong results, which creates amplify need for new data collectionand analysis stragtegies.Index Terms—Stress, Emotions, Sensors1 O VERVIEW1.1 MotivationResearch in the area of stress recognition is constantly increasing. To-day there are many different strategies available, thus making it moredifficult which one to choose.Being consequently exposed to stress in our daily live imposes a riskto our physical health and personal well being. Therefore it is neces-sary being able to precisely measure and understand stress symptomsto fight its effects on the human body.Most of previous research has focused mainly on recognizing stressunder very controlled scenarios. While these advances have helpedidentify new directions and features that are useful, to date, thereare very few works that attempt to recognize stress in the wild or atleast close to conditions outside of the laboratory environment (JulianRamos, Jin-Hyuk Hong, Anind K. Dey, 2014) This paper aims at pro-viding an overview of existing strategies, their effetiveness and limita-tions, while validating new approaches.of being monitored. Researchers are working on strategies for noninvasive measurement of symptoms to get more realistic results.2D ATA C OLLECTIONWhile gathering data for analysis required a testing laboratory a fewyears ago, it can be nowadays be sufficient to use the subjects phone ordeploy wearable sensors. For meaningful data a stressing situation hasto be created, in the most realistic way possible. (Sioni R., Chittaro L.,2015) The actual collection of data points can either be accomplishedby sensors, surveys or questionaires. In general, reasearchers dividebetween intrusive and non intrusive techniques for data collection.Peternel et al. developed a system that is able collect and processdata from various sources and visualize the observations (see Figure 1)(Klemen Peternel, Matevž Poga?nik, Rudi Tav?ar, Andrej Kos, 2012).1.2 Related WorkVarious body sensors are being used in recent research, when trying toestimate stress. Electro Dermal Activity (EDA) is said to be one of themost robust physiological indices of stress (Bakker, J.; Pechenizkiy,M.; Sidorova, N., 2011).EDA sensors measure changes in the electrical conductivity of theskin surface, which can be produced by various physical and emo-tional stimuli that trigger variations in sweat-gland activity (Sioni R.,Chittaro L., 2015).Healey et al. presented methods for collecting and analyzing mul-tiple physiological data (ECG, electromyogram, EDA and respira-tion) during driving tasks to determine a driver’s relative stress level(Healey, J.A.; Picard, R.W., 2005).A new method of recognizing a user’s current emotional experienceincludes video data analysis, which recognizes human facial expres-sions based on selected features (Hogue, M.E.; McDuff, D.J.; Picard,R.W., 2012).1.3 ProblemsHowever the current techniques are not perfect – errors may occur forexample detecting an increased heart pulse due to physical activity asstress. Currently research evaluates input derived from physiologicalsensors, most of the time under controlled lab conditions (Jing Zhai,Armando Barreto, 2006). These results do not correspond to resultsunder natural conditions, as they leave the test person in knowledge• Philipp Mieden is studying Media Informatics at the University of Munich,Germany, E-mail: [email protected]• This research paper was written for the Media Informatics Proseminar on”Measures, Sensors and Techniques for Stress Recognition”, 2017/2018.Figure 1: Contextual data are provided by user devices. The prepro-cessed data is used to calculate a tension score which is then visualizedtogether with corresponding observations and timestamps (KlemenPeternel, Matevž Poga?nik, Rudi Tav?ar, Andrej Kos, 2012).2.1Stress GenerationTo generate stress virtual environments are being utilized such asvideogames or 3D simulations. Scenarios range from fire emergenciesto airplane crashes, but also the workplace in the subjects office (SioniR., Chittaro L., 2015). Often a low stress and a high stress scenarioare provided to compare the results afterwards (see Figure 2).Figure 3: Wearable wristband electrodermal activity sensor (KlemenPeternel, Matevž Poga?nik, Rudi Tav?ar, Andrej Kos, 2012).Figure 2: Two virtual environments: (a) low-stress and (b) high-stressversions of a school building, and (c) low-stress and (d) high-stressversions of a train station (Sioni R., Chittaro L., 2015).2.2Physiological IndicatorsPhysiological Indicators can be used to retrieve data from a patient,there are sensors for Blood Volume Pulse, Pupil Diameter, Skin Tem-perature, Heart Rate Variability and Galvanic Skin Response Time(Sioni R., Chittaro L., 2015).2.3Intrusive Measurement TechniquesIntrusive techniques let the subject know that its being monitored,which may impact the subjects behaviour during the experiments. Forexample the stress generated in a virtual scenario will not match thestress in a real world situation because the subject is aware of the situ-ation being not real.2.4 Non Intrusive Measurement TechniquesNon Intrusive Techniques are not noticable to the subject, which cre-ates a natural scenario for the most accurate results and overcomesprevious limitations created by invasive data collection strategies. Forexample, analysis of facial video data can deliver interesting insightsinto the subjects mental condition.2.4.1 Thermal Infrared CameraAnalysis of thermal video data has shown big potential, researchersfrom japan and malaysia have proven recently. It can be captured witha special thermal infrared camera, and is not noticable to the subject.The analysis reveals the heart beat rate, skin temperature and pupildiameter (Jing Zhai, Armando Barreto, 2006).Norzali et al. managed to measure the blood vessel activity by an-alyzing thermal video data (see Figure 4) (Mohd Norzali Haji Mohd,Masayuki Kashima, Kiminori SATO, Mutsumi Watanabe, 2015).Physiological SensorsPhysiological sensors deliver metric data points and are currentlybeing used thoughout the research. They can be worn as wear-ables, or directly attached to the human body.SurveysSurveys are a tradional way of retrieving feedback, however theyare very limited and not adaptive to the situation. Usage inmobile phone apps is becoming popular, as this allows askingthe subject at a reoccuring interval (Klemen Peternel, MatevžPoga?nik, Rudi Tav?ar, Andrej Kos, 2012).QuestionairesQuestionaires are useful because the moderator can react toevents and lead the conversation to a valuable point. Howeverthey are very time intensive and cannot easily performed on ahuge number of subjects.WearablesWearables are becoming more popular for tracking physiolog-ical indicators because the can be comfortably worn duringthe day (Basel Kikhia, Thanos G. Stavropoulos, Stelios An-dreadis, Niklas Karvonen, Ioannis Kompatsiaris, Stefan Säven-stedt, Marten Pijl, Catharina Melander, 2016). This enables theresearches to get a slightly less intrusive insight, since wearablescan be worn in daily life and are likely being forgotten about bythe subject.Peternel et al. used wearable wristband sensors to gatherdata from subjects (see Figure 3) (Klemen Peternel, MatevžPoga?nik, Rudi Tav?ar, Andrej Kos, 2012).Figure 4: (a) Forehead anatomy, (b) thermal facial during stress (MohdNorzali Haji Mohd, Masayuki Kashima, Kiminori SATO, MutsumiWatanabe, 2015).3 D ATA A NALYSISAs a consequence of a growing amount of available sensors and tech-niques, the amount of data that is gathered and needs to be analyzedgrows exponentially as well.Thankfully the techniques for analysis improve as well, which al-lows computers to deal with those huge amounts of data.Also the analysis of gathered metric data has changed drastically inrecent years – a consequence of increasing processing power and re-search in machine learning. Nowadays not only statistical approachesare used for metric analysis, especially neural networks and machinelearning techniques are becoming more and more popular as they de-liver promising results.3.1 PreprocessingThe first step in data analysis is always cleaning the data, maybe therehave been corrupt or misplaced sensors, that delivered data points thatdont make any sense (Jing Zhai, Armando Barreto, 2006).Feature SelectionFor the best results a meaningful feature set has to be chosen andextracted from the data (Julian Ramos, Jin-Hyuk Hong, AnindK. Dey, 2014).Data NormalizationNormalization is used to minimize the impact of individual sub-ject responses in the training of the learning systems (Jing Zhai,Armando Barreto, 2006).3.2 AnalysisFinally the gathered data can be evaluated to provide insights. In gen-eral we can differentiate between the classic statistical approach andnew ones such as machine and deep learning, which require lots ofcomputing power.The result is often visualized to provide a better insight into the data(see Figure 5).Figure 5: result of data analyis (Basel Kikhia, Thanos G. Stavropou-los, Stelios Andreadis, Niklas Karvonen, Ioannis Kompatsiaris, StefanSävenstedt, Marten Pijl, Catharina Melander, 2016).3.2.1 Statistical MethodsOutlier detection algorithms provide a way to detect abnormal values,which might indicate a stress situation. Alternatively frequency analy-sis can deliver insights into reoccuring events, such as daily stressors.Another popular evaluation strategy is the Naive Bayes method,which applies a simplified version of Baye’s rule to compute the pos-terior probability of a category. given the input attribute values of anexample situation. This classifier is based on probability models, thatincorporate class conditional independence assumptions (John, G.H.,Langley, P., 1995).Decision tree classification represents a ‘divide-and-conquer’ ap-proach. It consists of a tree structure, where each internal node testsa particular attribute. Each branch represents an outcome of the test,and leaf nodes represent classes or class distributions. The goal is tobreak up a complex decision into a union of several simpler decisions,in the hope that the final solution obtained will resemble the intendeddesired solution (Safavian, S.R., Landgrebe, D., 1991).3.2.2 Machine LearningMachine Learning describes the algorithmic procedures of letting aprogram learn a given model, and compare it with new data.Supervised models learn from labeled data and can thus be carefullyadjusted to a specific user, but then may deliver errornous results forothers. (Klemen Peternel, Matevž Poga?nik, Rudi Tav?ar, Andrej Kos,2012) Unsupervised models do not require labeled data and thus canhandle natural distribution (Julian Ramos, Jin-Hyuk Hong, Anind K.Dey, 2014).Support Vector Machines (SVM) are the computational machinelearning systems that use a hypothesis space of linear functions in ahigh dimensional feature space to perform supervised classification(Jing Zhai, Armando Barreto, 2006).4 D ISCUSSIONCurrent analyis strategies are delivering valuable results – but thereis still space for improvements. Limitations arise from misinterpre-tation of sensor values to the subject being aware of the monitoring.Research needs to focus on non invasive techniques, as those deliverinsights into stress under realistic conditions. From observing the de-velopment in research we can see a clear trend moving towards auto-mated systems and machine assisted evalutaion of gathered data. Ut-lizing todays computing power and networked systems is logical step- however we should not forget to protect these systems, as they storesensitive patient data, and make them resistant to tampering and ma-nipulation. Also the data must be anonymized to make the attributionto a specific patient impossible and conform to data protection laws.5 C ONCLUSIONUnderstanding stress symptoms and the danger it imposes to the hu-man body is as important as finding stragtegies to reduce stress. Theprevious research described in this paper gathered useful informationand will serve as a base for further research. Hopefully this will helpus to develop new strategies to prevent stress and relief its damageand make the techniques not only available to researchers, but also todoctors to aid them in treating their patients.